python - 如何用keras近似行列式

作为实验，我正在构建一个keras模型来近似矩阵的行列式。但是，当我运行它时，损失在每个时期都会下降，而验证损失会上升!例如:

8s - loss: 7573.9168 - val_loss: 21831.5428
Epoch 21/50
8s - loss: 7345.0197 - val_loss: 23594.8540
Epoch 22/50
13s - loss: 7087.7454 - val_loss: 24718.3967
Epoch 23/50
7s - loss: 6851.8714 - val_loss: 25624.8609
Epoch 24/50
6s - loss: 6637.8168 - val_loss: 26616.7835
Epoch 25/50
7s - loss: 6446.8898 - val_loss: 28856.9654
Epoch 26/50
7s - loss: 6255.7414 - val_loss: 30122.7924
Epoch 27/50
7s - loss: 6054.5280 - val_loss: 32458.5306
Epoch 28/50

这是完整的代码:

import numpy as np
import sys
from scipy.stats import pearsonr
from scipy.linalg import det
from sklearn.model_selection import train_test_split
from tqdm import tqdm
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
import math
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from keras import backend as K

def baseline_model():
# create model
        model = Sequential()
        model.add(Dense(200, input_dim=n**2, kernel_initializer='normal', activation='relu'))
        model.add(Dense(1, input_dim=n**2))
        #        model.add(Dense(1, kernel_initializer='normal'))
        # Compile model
        model.compile(loss='mean_squared_error', optimizer='adam')
        return model


n = 15

print("Making the input data using seed 7", file=sys.stderr)
np.random.seed(7)
U = np.random.choice([0, 1], size=(n**2,n))
#U is a random orthogonal matrix
X =[]
Y =[]
# print(U)
for i in tqdm(range(100000)):
        I = np.random.choice(n**2, size = n)
        # Pick out the random rows and sort the rows of the matrix lexicographically.
        A = U[I][np.lexsort(np.rot90(U[I]))]
        X.append(A.ravel())
        Y.append(det(A))

X = np.array(X)
Y = np.array(Y)

print("Data created")

estimators = []
estimators.append(('standardize', StandardScaler()))
estimators.append(('mlp', KerasRegressor(build_fn=baseline_model, epochs=50, batch_size=32, verbose=2)))
pipeline = Pipeline(estimators)
X_train, X_test, y_train, y_test = train_test_split(X, Y,
                                                    train_size=0.75, test_size=0.25)
pipeline.fit(X_train, y_train, mlp__validation_split=0.3)

更新1

我尝试添加更多层和L_2正则化。但是，它几乎没有区别。

def baseline_model():
# create model
        model = Sequential()
        model.add(Dense(n**2, input_dim=n**2, kernel_initializer='glorot_normal', activation='relu'))
        model.add(Dense(int((n**2)/2.0), kernel_initializer='glorot_normal', activation='relu', kernel_regularizer=regularizers.l2(0.01)))
        model.add(Dense(int((n**2)/2.0), kernel_initializer='glorot_normal', activation='relu', kernel_regularizer=regularizers.l2(0.01)))
        model.add(Dense(int((n**2)/2.0), kernel_initializer='glorot_normal', activation='relu', kernel_regularizer=regularizers.l2(0.01)))
        model.add(Dense(1, kernel_initializer='glorot_normal'))
        # Compile model
        model.compile(loss='mean_squared_error', optimizer='adam')
        return model

我将纪元数增加到100，结果是:

19s - loss: 788.9504 - val_loss: 18423.2807
Epoch 97/100
24s - loss: 760.2046 - val_loss: 18305.9273
Epoch 98/100
20s - loss: 806.0941 - val_loss: 18174.8706
Epoch 99/100
24s - loss: 780.0487 - val_loss: 18356.7482
Epoch 100/100
27s - loss: 749.2595 - val_loss: 18331.5859

最佳答案

我测试了您的代码，并得到了相同的结果。但是，让我们进入对矩阵行列式(DET)的基本理解。 DET由n组成!乘积，因此您无法在神经网络的几层中真正使用n * n权重对其进行近似。这要求权重数量从15开始就不会缩放到n = 15!是1307674368000的DET中的乘法项。

关于python - 如何用keras近似行列式，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/46734134/