我对im2txt的预训练Inception v3模型进行微调时遇到问题。出于某种原因，初始培训并没有减少太多损失，微调Inception v3并没有减少我的培训数据的任何损失。我试图找出原因，任何见解都会有所帮助。
im2txt是一个模型，它接收图像输入并打印出字幕列表作为输出。最初，im2txt将标题作为连贯的句子打印出来，以描述图像。为了适合我的项目，我更改了训练数据中的代码和标签，以便打印出与图像相关的单词列表。
例如，我的图像将如下所示。请注意，图像中的对象比普通Imagenet图像要多：
我的标签标题如下所示：
female lady woman clothes shop customer
我总共有40万张图片和相应的标签标题。我接受了130,000步的初始训练，并进行了170,000步的微调。词汇中总共有750个单词。初始训练+微调的损耗曲线（从步骤130,000开始）如下：
精度和召回率约为0.35〜40。
培训的配置文件如下：
# Copyright 2016 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Image-to-text model and training configurations."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
class ModelConfig(object):
"""Wrapper class for model hyperparameters."""
def __init__(self):
"""Sets the default model hyperparameters."""
# File pattern of sharded TFRecord file containing SequenceExample protos.
# Must be provided in training and evaluation modes.
self.input_file_pattern = None
# Image format ("jpeg" or "png").
self.image_format = "jpeg"
# Approximate number of values per input shard. Used to ensure sufficient
# mixing between shards in training.
self.values_per_input_shard = 2300
# Minimum number of shards to keep in the input queue.
self.input_queue_capacity_factor = 2
# Number of threads for prefetching SequenceExample protos.
self.num_input_reader_threads = 1
# Name of the SequenceExample context feature containing image data.
self.image_feature_name = "image/data"
# Name of the SequenceExample feature list containing integer captions.
self.caption_feature_name = "image/caption_ids"
# Number of unique words in the vocab (plus 1, for <UNK>).
# The default value is larger than the expected actual vocab size to allow
# for differences between tokenizer versions used in preprocessing. There is
# no harm in using a value greater than the actual vocab size, but using a
# value less than the actual vocab size will result in an error.
self.vocab_size = 750
# Number of threads for image preprocessing. Should be a multiple of 2.
self.num_preprocess_threads = 4
# Batch size.
self.batch_size = 32
# File containing an Inception v3 checkpoint to initialize the variables
# of the Inception model. Must be provided when starting training for the
# first time.
self.inception_checkpoint_file = None
# Dimensions of Inception v3 input images.
self.image_height = 299
self.image_width = 299
# Scale used to initialize model variables.
self.initializer_scale = 0.08
# LSTM input and output dimensionality, respectively.
self.embedding_size = 512
self.num_lstm_units = 512
# If < 1.0, the dropout keep probability applied to LSTM variables.
self.lstm_dropout_keep_prob = 0.7
class TrainingConfig(object):
"""Wrapper class for training hyperparameters."""
def __init__(self):
"""Sets the default training hyperparameters."""
# Number of examples per epoch of training data.
self.num_examples_per_epoch = 100000
# Optimizer for training the model.
self.optimizer = "SGD"
# Learning rate for the initial phase of training.
self.initial_learning_rate = 2.0
self.learning_rate_decay_factor = 0.5
self.num_epochs_per_decay = 1.0
# Learning rate when fine tuning the Inception v3 parameters.
self.train_inception_learning_rate = 0.005
# If not None, clip gradients to this value.
self.clip_gradients = 5.0
# How many model checkpoints to keep.
self.max_checkpoints_to_keep = 5
任何建议，见解或观察都将是很棒的。
最佳答案
请注意，im2txt在使句子易于阅读方面非常强大。在句子中，一个单词与相邻单词相关，这就是为什么它起作用的原因。在您的情况下，您正在更改模型以产生一组与顺序无关的标签。实际上，在im2txt模型中，单词“ female”（女性），“ woman”（女人）和“ lady”（女士）基本上是相同的概念，并且im2txt可以创建句子来刷写这些单词。例如，在im2txt中：“女士穿着漂亮的粉红色裙子”与“女士的裙子是粉红色”相同，或者应该非常相似。在您的情况下，如果您不对单词的顺序提供一些规则，则会使您的模型非常混乱，并且可能无法学习。
如果要从图像中获取标签列表，则应仅使用具有多标签分类的初始模型（将softmax层更改为S型，并使用S型交叉熵作为损失函数）。
关于python - 对im2txt的Inception v3进行微调时，损失不会减少，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/46637864/