如何使用自动编码器#2部分-深度自动编码器#3rd部分-堆叠式自动编码器初始化MLP的权重

如果您真正想做的是使用自动编码器学到的权重来初始化来自分类器的权重(我不太肯定，我推荐这种方法):您可以使用layer#get_weights提取权重矩阵，将其修剪(因为编码器有5个单位，而分类器只有1个单位)，最后设置分类器权重.以下几行内容: w, b = ae.get_layer('encoder').get_weights()# remove all units except by one.neuron_to_keep = 2w = w[:, neuron_to_keep:neuron_to_keep + 1]b = b[neuron_to_keep:neuron_to_keep + 1]classifier.get_layer('predictions').set_weights(w, b) I have built an autoencoder (1 encoder 8:5, 1 decoder 5:8) which takes the Pima-Indian-Diabetes dataset (https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv) and reduces its dimension (from 8 to 5). I would now like to use these reduced features to classify the data using an mlp. Now, here, I have some problems with the basic understanding of the architecture. How do I use the weights of the autoencoder and feed them into the mlp? I have checked these threads - https://github.com/keras-team/keras/issues/91 and https://www.codementor.io/nitinsurya/how-to-re-initialize-keras-model-weights-et41zre2g. The question here is which weight matrix should I consider? the one for the encoder part or the decoder part? When I add the layers for the mlp, how do I initialise the weights with these saved weights, not getting the exact syntax. Also, should my mlp start with 5 neurons since my reduced dimension is 5? What are the possible dimensions of the mlp for this binary classification problem? If anyone could elaborate please?The deep autoencoder code is as follows:# from keras.models import Sequentialfrom keras.layers import Input, Densefrom keras.models import Modelfrom sklearn.preprocessing import MinMaxScalerfrom sklearn.model_selection import train_test_splitimport numpy# Data pre-processing...# load pima indians datasetdataset = numpy.loadtxt("C:/Users/dibsa/Python Codes/pima.csv", delimiter=",")# split into input (X) and output (Y) variablesX = dataset[:, 0:8]Y = dataset[:, 8]# Split data into training and testing datasetsx_train, x_test, y_train, y_test = train_test_split( X, Y, test_size=0.2, random_state=42)# scale the data within [0-1] rangescalar = MinMaxScaler()x_train = scalar.fit_transform(x_train)x_test = scalar.fit_transform(x_test)# Autoencoder code begins here...encoding_dim1 = 5 # size of encoded representationsencoding_dim2 = 3 # size of encoded representations in the bottleneck layer# this is our input placeholderinput_data = Input(shape=(8,))# "encoded" is the first encoded representation of the inputencoded = Dense(encoding_dim1, activation='relu', name='encoder1')(input_data)# "enc" is the second encoded representation of the inputenc = Dense(encoding_dim2, activation='relu', name='encoder2')(encoded)# "dec" is the lossy reconstruction of the inputdec = Dense(encoding_dim1, activation='sigmoid', name='decoder1')(enc)# "decoded" is the final lossy reconstruction of the inputdecoded = Dense(8, activation='sigmoid', name='decoder2')(dec)# this model maps an input to its reconstructionautoencoder = Model(inputs=input_data, outputs=decoded)autoencoder.compile(optimizer='sgd', loss='mse')# trainingautoencoder.fit(x_train, x_train, epochs=300, batch_size=10, shuffle=True, validation_data=(x_test, x_test)) # need more tuning# test the autoencoder by encoding and decoding the test datasetreconstructions = autoencoder.predict(x_test)print('Original test data')print(x_test)print('Reconstructed test data')print(reconstructions)#The stacked autoencoder code is as follows:# from keras.models import Sequentialfrom keras.layers import Input, Densefrom keras.models import Modelfrom sklearn.preprocessing import MinMaxScalerfrom sklearn.model_selection import train_test_splitimport numpy# Data pre-processing...# load pima indians datasetdataset = numpy.loadtxt("C:/Users/dibsa/Python Codes/pima.csv", delimiter=",")# split into input (X) and output (Y) variablesX = dataset[:, 0:8]Y = dataset[:, 8]# Split data into training and testing datasetsx_train, x_test, y_train, y_test = train_test_split( X, Y, test_size=0.2, random_state=42)# scale the data within [0-1] rangescalar = MinMaxScaler()x_train = scalar.fit_transform(x_train)x_test = scalar.fit_transform(x_test)# Autoencoder code goes here...encoding_dim1 = 5 # size of encoded representationsencoding_dim2 = 3 # size of encoded representations in the bottleneck layer# this is our input placeholderinput_data1 = Input(shape=(8,))# the first encoded representation of the inputencoded1 = Dense(encoding_dim1, activation='relu', name='encoder1')(input_data1)# the first lossy reconstruction of the inputdecoded1 = Dense(8, activation='sigmoid', name='decoder1')(encoded1)# this model maps an input to its first layer of reconstructionsautoencoder1 = Model(inputs=input_data1, outputs=decoded1)# this is the first encoder modelenc1 = Model(inputs=input_data1, outputs=encoded1)autoencoder1.compile(optimizer='sgd', loss='mse')# trainingautoencoder1.fit(x_train, x_train, epochs=300, batch_size=10, shuffle=True, validation_data=(x_test, x_test))FirstAEoutput = autoencoder1.predict(x_train)input_data2 = Input(shape=(encoding_dim1,))# the second encoded representations of the inputencoded2 = Dense(encoding_dim2, activation='relu', name='encoder2')(input_data2)# the final lossy reconstruction of the inputdecoded2 = Dense(encoding_dim1, activation='sigmoid', name='decoder2')(encoded2)# this model maps an input to its second layer of reconstructionsautoencoder2 = Model(inputs=input_data2, outputs=decoded2)# this is the second encoderenc2 = Model(inputs=input_data2, outputs=encoded2)autoencoder2.compile(optimizer='sgd', loss='mse')# trainingautoencoder2.fit(FirstAEoutput, FirstAEoutput, epochs=300, batch_size=10, shuffle=True)# this is the overall autoencoder mapping an input to its final reconstructionsautoencoder = Model(inputs=input_data1, outputs=encoded2)# test the autoencoder by encoding and decoding the test datasetreconstructions = autoencoder.predict(x_test)print('Original test data')print(x_test)print('Reconstructed test data')print(reconstructions) 解决方案 So many questions. What have you tried so far? Code snippets?If your decoder is trying to reconstruct the input, then it doesn't really make sense to me to attach your classifier to its output. I mean, why not just attach it to the input in the first time? So if you are set on using an auto-encoder, I'd say it's pretty clear that you should attach your classifier to the output of the encoder pipe.I'm not quite sure what you mean with "use the weights of the autoencoder and feed them into the mlp". You don't feed a layer with another layer's weights, but with it's output signal. This is pretty easy to do on Keras. Let's say you defined your auto-encoder and trained it as such:from keras Input, Modelfrom keras import backend as Kfrom keras.layers import Densex = Input(shape=[8])y = Dense(5, activation='sigmoid' name='encoder')(x)y = Dense(8, name='decoder')(y)ae = Model(inputs=x, outputs=y)ae.compile(loss='mse', ...)ae.fit(x_train, x_train, ...)K.models.save_model(ae, './autoencoder.h5')Then you can attach a classifying layer at the encoder and create a classifier model with the following code:# load the model from the disk if you# are in a different execution.ae = K.models.load_model('./autoencoder.h5')y = ae.get_layer('encoder').outputy = Dense(1, activation='sigmoid', name='predictions')(y)classifier = Model(inputs=ae.inputs, outputs=y)classifier.compile(loss='binary_crossentropy', ...)classifier.fit(x_train, y_train, ...)That's it, really. The classifier model will now have the first embedding layer encoder of the ae model as its first layer, followed by a sigmoid decision layer predictions.If what you are really trying to do is to use the weights learned by the auto-encoder to initialize the weights from the classifier (I'm not positive I recommend this approach):You can take the weight matrices with layer#get_weights, prune it (because the encoder has 5 units and the classifier only has 1) and finally set the classifier weights. Something in the following lines:w, b = ae.get_layer('encoder').get_weights()# remove all units except by one.neuron_to_keep = 2w = w[:, neuron_to_keep:neuron_to_keep + 1]b = b[neuron_to_keep:neuron_to_keep + 1]classifier.get_layer('predictions').set_weights(w, b) 这篇关于如何使用自动编码器#2部分-深度自动编码器#3rd部分-堆叠式自动编码器初始化MLP的权重的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！ 上岸，阿里云！