Keras:在微调初始时准确度下降

我还能够在初始阶段叠加此模型以创建我的完整模型并使用该完整模型成功地对我的训练集进行分类.from keras 导入模型从 keras 导入优化器从 keras.callbacks 导入 EarlyStoppingimg_width, img_height = 150, 150top_model_weights_path = 'Inception_fc_model_v0.h5'train_data_dir = '../data/train2'验证数据目录 = '../data/train2'#我们有多少包容性的例子?inclusive_images = 1424nb_train_samples = 1424nb_validation_samples = 1424时代 = 50批量大小 = 16# 构建完整的评估网络base_model = applications.inception_v3.InceptionV3(weights='imagenet', include_top=False, input_shape=(img_width,img_height,3))top_model = 顺序()top_model.add(Flatten(input_shape=base_model.output_shape[1:]))top_model.add(Dense(1000, activation='relu'))top_model.add(Dense(inclusive_images, activation='softmax'))top_model.load_weights(top_model_weights_path)#结合基础模型和顶级模型fullModel = Model(input= base_model.input, output= top_model(base_model.output))#使用完整的训练数据集进行预测结果 = fullModel.predict_generator(ImageDataGenerator(rescale=1./255).flow_from_directory(train_data_dir,target_size=(img_width, img_height),批量大小=批量大小，class_mode='categorical',洗牌=假))检查对这个完整模型的处理结果与瓶颈生成的全连接模型的准确性相匹配.将 matplotlib.pyplot 导入为 plt进口经营者#从结果中检索基于 softmax 的类分配resultMaxClassIDs = [ max(enumerate(result), key=operator.itemgetter(1))[0] for result in results]#resultMaxClassIDs 应该等于 range(inclusive_images) 所以我们减去两者并绘制绝对值的对数#寻找表明值不相等的尖峰plt.plot([np.log(np.abs(x)+10) for x in (np.array(resultMaxClassIDs) - np.array(range(inclusive_images)))])问题来了:当我使用这个完整的模型并尝试训练它时，即使验证率保持在 99% 以上，准确率也会下降到 0.model2 = fullModel对于 model2.layers[:-2] 中的层:layer.trainable = 假# 使用 SGD/动量优化器编译模型# 和非常慢的学习率.#model.compile(loss='binary_crossentropy',optimizer=optimizers.SGD(lr=1e-4,momentum=0.9),metrics=['accuracy'])model2.compile(loss='categorical_crossentropy',优化器=优化器.SGD(lr=1e-4，动量=0.9)，指标=['准确度'])train_datagen = ImageDataGenerator(rescale=1./255)test_datagen = ImageDataGenerator(rescale=1./255)train_generator = train_datagen.flow_from_directory(train_data_dir,target_size=(img_height, img_width),批量大小=批量大小，class_mode='分类')验证生成器 = test_datagen.flow_from_directory(验证数据目录，target_size=(img_height, img_width),批量大小=批量大小，class_mode='分类')回调 = [EarlyStopping(monitor='acc', min_delta=0, Patient=3,verbose=0, mode='auto',baseline=None)]# 微调模型model2.fit_generator(#train_generator，验证生成器，step_per_epoch=nb_train_samples//batch_size,validation_steps = nb_validation_samples//batch_size,时代=时代，验证数据=验证生成器)纪元 1/5089/89 [==============================] - 388s 4s/step - 损失:13.5787 - acc:0.0000e+00 - val_loss: 0.0353 - val_acc: 0.9937随着事情的进展情况会变得更糟纪元 21/5089/89 [==============================] - 372s 4s/step - loss: 7.3850 - acc: 0.0035 - val_loss:0.5813 - val_acc:0.8272我唯一能想到的就是在这最后一列火车上以某种方式错误地分配了训练标签，但我之前使用 VGG16 使用类似的代码成功地做到了这一点.我搜索了代码，试图找到一个差异来解释为什么一个模型在 99% 以上的时间进行准确预测会降低其训练准确性，同时在微调期间保持验证准确性，但我无法弄清楚.任何帮助将不胜感激.关于代码和环境的信息:那些看起来很奇怪但本来就是这样的事情:每个类只有 1 张图片.这个神经网络旨在分类环境和方向条件为受控.每个班级只有一个可接受的图像对应正确的环境和旋转情况.测试集和验证集是相同的.这个 NN 是唯一的设计用于正在接受培训的课程.图片它将处理将是类示例的副本.这是我的意图将模型过度拟合到这些类我正在使用:Windows 10 Anaconda 客户端 1.6.14 下的 Python 3.5.6 Keras 2.2.2Tensorflow 1.10.0 作为后端CUDA 9.0CuDNN 8.0我已签出:微调模型中的 Keras 精度差异VGG16 Keras 微调:低准确率Keras:模型在达到 99% 的准确率和损失 0.01 后，准确率下降Keras inception v3 再训练和微调错误如何查找哪个版本TensorFlow 安装在我的系统中?但它们看起来无关. 解决方案 注意:由于您的问题有点奇怪并且在没有经过训练的模型和数据集的情况下很难调试，因此这个答案只是一个(最佳)猜测在考虑了许多可能会出错的事情之后.请提供您的反馈，如果它不起作用，我将删除此答案.由于 inception_V3 包含 BatchNormalization 层，当您将 trainable 参数设置为 False 时，问题可能是由于该层的(不知何故模糊或意外)行为造成的 (1, 2, 3，4).现在，让我们看看这是否是问题的根源:as@fchollet 建议，在定义模型进行微调时设置学习阶段:from keras import backend as KK.set_learning_phase(0)base_model = applications.inception_v3.InceptionV3(weights='imagenet', include_top=False, input_shape=(img_width,img_height,3))对于 base_model.layers 中的层:layer.trainable = 假K.set_learning_phase(1)top_model = 顺序()top_model.add(Flatten(input_shape=base_model.output_shape[1:]))top_model.add(Dense(1000, activation='relu'))top_model.add(Dense(inclusive_images, activation='softmax'))top_model.load_weights(top_model_weights_path)#结合基础模型和顶级模型fullModel = Model(input= base_model.input, output= top_model(base_model.output))fullModel.compile(loss='categorical_crossentropy',优化器=优化器.SGD(lr=1e-4，动量=0.9)，指标=['准确度'])################################################################### 在这里，定义生成器，然后像以前一样拟合模型###################################################################旁注:这不会对您的情况造成任何问题，但请记住，当您使用 top_model(base_model.output) 整个 Sequential 模型(即top_model) 存储为 fullModel 的一层.您可以使用 fullModel.summary() 或 print(fullModel.layers[-1]) 来验证这一点.因此，当您使用: for layer in model2.layers[:-2]:layer.trainable = 假您实际上并没有冻结 base_model 的最后一层.然而，由于它是一个 Concatenate 层，因此没有可训练的参数，所以不会出现问题，它会按照您的预期运行.I am having trouble fine tuning an Inception model with Keras.I have managed to use tutorials and documentation to generate a model of fully connected top layers that classifies my dataset into their proper categories with an accuracy over 99% using bottleneck features from Inception.import numpy as npfrom keras.preprocessing.image import ImageDataGeneratorfrom keras.models import Sequentialfrom keras.layers import Dropout, Flatten, Densefrom keras import applications# dimensions of our images.img_width, img_height = 150, 150#paths for saving weights and finding datasetstop_model_weights_path = 'Inception_fc_model_v0.h5'train_data_dir = '../data/train2'validation_data_dir = '../data/train2'#training related parameters?inclusive_images = 1424nb_train_samples = 1424nb_validation_samples = 1424epochs = 50batch_size = 16def save_bottlebeck_features(): datagen = ImageDataGenerator(rescale=1. / 255) # build bottleneck features model = applications.inception_v3.InceptionV3(include_top=False, weights='imagenet', input_shape=(img_width,img_height,3)) generator = datagen.flow_from_directory( train_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='categorical', shuffle=False) bottleneck_features_train = model.predict_generator( generator, nb_train_samples // batch_size) np.save('bottleneck_features_train', bottleneck_features_train) generator = datagen.flow_from_directory( validation_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='categorical', shuffle=False) bottleneck_features_validation = model.predict_generator( generator, nb_validation_samples // batch_size) np.save('bottleneck_features_validation', bottleneck_features_validation)def train_top_model(): train_data = np.load('bottleneck_features_train.npy') train_labels = np.array(range(inclusive_images)) validation_data = np.load('bottleneck_features_validation.npy') validation_labels = np.array(range(inclusive_images)) print('base size ', train_data.shape[1:]) model = Sequential() model.add(Flatten(input_shape=train_data.shape[1:])) model.add(Dense(1000, activation='relu')) model.add(Dense(inclusive_images, activation='softmax')) model.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy']) proceed = True #model.load_weights(top_model_weights_path) while proceed: history = model.fit(train_data, train_labels, epochs=epochs, batch_size=batch_size)#, #validation_data=(validation_data, validation_labels), verbose=1) if history.history['acc'][-1] > .99: proceed = False model.save_weights(top_model_weights_path)save_bottlebeck_features()train_top_model() Epoch 50/50 1424/1424 [==============================] - 17s 12ms/step - loss: 0.0398 - acc: 0.9909I have also been able to stack this model on top of inception to create my full model and use that full model to successfully classify my training set.from keras import Modelfrom keras import optimizersfrom keras.callbacks import EarlyStoppingimg_width, img_height = 150, 150top_model_weights_path = 'Inception_fc_model_v0.h5'train_data_dir = '../data/train2'validation_data_dir = '../data/train2'#how many inclusive examples do we have?inclusive_images = 1424nb_train_samples = 1424nb_validation_samples = 1424epochs = 50batch_size = 16# build the complete network for evaluationbase_model = applications.inception_v3.InceptionV3(weights='imagenet', include_top=False, input_shape=(img_width,img_height,3))top_model = Sequential()top_model.add(Flatten(input_shape=base_model.output_shape[1:]))top_model.add(Dense(1000, activation='relu'))top_model.add(Dense(inclusive_images, activation='softmax'))top_model.load_weights(top_model_weights_path)#combine base and top modelfullModel = Model(input= base_model.input, output= top_model(base_model.output))#predict with the full training datasetresults = fullModel.predict_generator(ImageDataGenerator(rescale=1. / 255).flow_from_directory( train_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='categorical', shuffle=False))inspection of the results from processing on this full model match the accuracy of the bottleneck generated fully connected model.import matplotlib.pyplot as pltimport operator#retrieve what the softmax based class assignments would be from resultsresultMaxClassIDs = [ max(enumerate(result), key=operator.itemgetter(1))[0] for result in results]#resultMaxClassIDs should be equal to range(inclusive_images) so we subtract the two and plot the log of the absolute value#looking for spikes that indicate the values aren't equalplt.plot([np.log(np.abs(x)+10) for x in (np.array(resultMaxClassIDs) - np.array(range(inclusive_images)))])Here is the problem:When I take this full model and attempt to train it, Accuracy drops to 0 even though validation remains above 99%. model2 = fullModelfor layer in model2.layers[:-2]: layer.trainable = False# compile the model with a SGD/momentum optimizer# and a very slow learning rate.#model.compile(loss='binary_crossentropy', optimizer=optimizers.SGD(lr=1e-4, momentum=0.9), metrics=['accuracy'])model2.compile(loss='categorical_crossentropy', optimizer=optimizers.SGD(lr=1e-4, momentum=0.9), metrics=['accuracy'])train_datagen = ImageDataGenerator(rescale=1. / 255)test_datagen = ImageDataGenerator(rescale=1. / 255)train_generator = train_datagen.flow_from_directory( train_data_dir, target_size=(img_height, img_width), batch_size=batch_size, class_mode='categorical')validation_generator = test_datagen.flow_from_directory( validation_data_dir, target_size=(img_height, img_width), batch_size=batch_size, class_mode='categorical')callback = [EarlyStopping(monitor='acc', min_delta=0, patience=3, verbose=0, mode='auto', baseline=None)]# fine-tune the modelmodel2.fit_generator( #train_generator, validation_generator, steps_per_epoch=nb_train_samples//batch_size, validation_steps = nb_validation_samples//batch_size, epochs=epochs, validation_data=validation_generator) Epoch 1/50 89/89 [==============================] - 388s 4s/step - loss: 13.5787 - acc: 0.0000e+00 - val_loss: 0.0353 - val_acc: 0.9937and it gets worse as things progress Epoch 21/50 89/89 [==============================] - 372s 4s/step - loss: 7.3850 - acc: 0.0035 - val_loss: 0.5813 - val_acc: 0.8272The only thing I could think of is that somehow the training labels are getting improperly assigned on this last train, but I've successfully done this with similar code using VGG16 before.I have searched over the code trying to find a discrepancy to explain why a model making accurate predictions over 99% of the time drops its training accuracy while maintaining validation accuracy during fine tuning, but I can't figure it out. Any help would be appreciated.Information about the code and environment:Things that are going to stand out as weird, but are meant to be that way:There is only 1 image per class. This NN is intended to classifyobjects whose environmental and orientation conditions arecontrolled. Their is only one acceptable image for each classcorresponding to the correct environmental and rotational situation.The test and validation set are the same. This NN is only everdesigned to be used on the classes it is being trained on. The imagesit will process will be carbon copies of the class examples. It is myintent to overfit the model to these classesI am using:Windows 10Python 3.5.6 under Anaconda client 1.6.14Keras 2.2.2Tensorflow 1.10.0 as the backendCUDA 9.0CuDNN 8.0I have checked out:Keras accuracy discrepancy in fine-tuned modelVGG16 Keras fine tuning: low accuracyKeras: model accuracy drops after reaching 99 percent accuracy and loss 0.01Keras inception v3 retraining and finetuning errorHow to find which version of TensorFlow is installed in my system?but they appear unrelated. 解决方案 Note: Since your problem is a bit strange and difficult to debug without having your trained model and dataset, this answer is just a (best) guess after considering many things that may have could go wrong. Please provide your feedback and I will delete this answer if it does not work.Since the inception_V3 contains BatchNormalization layers, maybe the problem is due to (somehow ambiguous or unexpected) behavior of this layer when you set trainable parameter to False (1, 2, 3, 4).Now, let's see if this is the root of the problem: as suggested by @fchollet, set the learning phase when defining the model for fine-tuning:from keras import backend as KK.set_learning_phase(0)base_model = applications.inception_v3.InceptionV3(weights='imagenet', include_top=False, input_shape=(img_width,img_height,3))for layer in base_model.layers: layer.trainable = FalseK.set_learning_phase(1)top_model = Sequential()top_model.add(Flatten(input_shape=base_model.output_shape[1:]))top_model.add(Dense(1000, activation='relu'))top_model.add(Dense(inclusive_images, activation='softmax'))top_model.load_weights(top_model_weights_path)#combine base and top modelfullModel = Model(input= base_model.input, output= top_model(base_model.output))fullModel.compile(loss='categorical_crossentropy', optimizer=optimizers.SGD(lr=1e-4, momentum=0.9), metrics=['accuracy'])###################################################################### Here, define the generators and then fit the model same as before ######################################################################Side Note: This is not causing any problem in your case, but keep in mind that when you use top_model(base_model.output) the whole Sequential model (i.e. top_model) is stored as one layer of fullModel. You can verify this by either using fullModel.summary() or print(fullModel.layers[-1]). Hence when you used:for layer in model2.layers[:-2]: layer.trainable = Falseyou are actually not freezing the last layer of base_model as well. However, since it is a Concatenate layer, and therefore does not have trainable parameters, no problem occurs and it would behave as you intended. 这篇关于Keras:在微调初始时准确度下降的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！ 上岸，阿里云！