keras - Keras:fit_generator的多个输入和多个输出使用flow_from_directory

多任务学习模型接受三个输入。我正在使用keras数据生成器。是否可以将三个数据生成器传递给model.fit_generator函数？



问题定义

我正在处理分类问题。我使用的数据集是“画家”（按编号），由kaggle主持。任务是确定给定绘画的画家，风格和体裁。

我已经开发了执行每个任务的个人模型。现在，我想结合多任务学习，看看它是否胜过单个模型。

 Model                       No of classes (Softmax)
------                     ------------------------
 Model predicting painter         8
 given paintings

 Model predicting style           10
 given paintings


 Model predicting genre           23
 given paintings

上表详细列出了各个模型，以及每个模型的输出类别编号。

现在，我想进行多任务学习，因此我想出了以下简单架构
Multi Task Learning Architecture

 style   = Input(shape=(64,64,3))
 genre   = Input(shape=(64,64,3))
 painter = Input(shape=(64,64,3))


 shared_conv = Convolution2D(
            filters = 5,# 5 feature maps
            kernel_size = (5,5),
            strides = 1)

 shared_conv_layer_A = shared_conv(style)
 shared_conv_layer_B = shared_conv(genre)
 shared_conv_layer_C = shared_conv(painter)

 merged_layer = keras.layers.concatenate([shared_conv_layer_A,shared_conv_layer_B,shared_conv_layer_C],axis=-1)

 pooling = MaxPooling2D(
        pool_size = (2,2),
        strides = 2
      )(merged_layer)

 dense = Flatten()(pooling)

 out_style = Dense(
        no_classes_style,
        kernel_initializer=glorot_normal(seed=seed_val),
        bias_initializer = 'zero',
        kernel_regularizer = l2(l=0.0001),
        activation = 'softmax',
    )(dense)

 out_genre = Dense(
        no_classes_genre,
        kernel_initializer=glorot_normal(seed=seed_val),
        bias_initializer = 'zero',
        kernel_regularizer = l2(l=0.0001),
        activation = 'softmax',
    )(dense)

 out_painter = Dense(
        no_classes_painter,
        kernel_initializer=glorot_normal(seed=seed_val),
        bias_initializer = 'zero',
        kernel_regularizer = l2(l=0.0001),
        activation = 'softmax',
    )(dense)


 multi_tasking_model = Model(inputs=[style,genre,painter],outputs=[out_style,out_genre,out_painter])
 multi_tasking_model.summary()


 multi_tasking_model.compile(
   loss = 'categorical_crossentropy',
   optimizer=Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=0.00000001 ),
   metrics=['accuracy']
 )

现在，我想传递三个keras图像数据生成器。因此，我想出了一个自定义数据生成器

 def create_data_generator(style_generator,genre_generator,painter_generator):
    # Input
    _style_generator = style_generator[0]
    _genre_generator = genre_generator[0]
    _painter_generator = painter_generator[0]

   # Label
   _lstyle_generator = style_generator[1]
   _lgenre_generator = genre_generator[1]
   _lpainter_generator = painter_generator[1]

return [_style_generator,_genre_generator,_painter_generator],[_lstyle_generator,_genre_generator,_painter_generator]

train_mulitle_data_generator = create_data_generator(trainStyleDataGenerator,trainGenreDataGenerator,trainPainterDataGenerator)
valid_mulitle_data_generator = create_data_generator(validationStyleDataGenerator,validationGenreDataGenerator,validationPainterDataGenerator)




history = multi_tasking_model.fit_generator(
     generator = train_mulitle_data_generator,
     steps_per_epoch= len(train_mulitle_data_generator),
     epochs = no_epoch,
     validation_data = valid_mulitle_data_generator,
 )

我遇到的错误

   'tuple' object has no attribute 'ndim'

有没有其他方法可以传递多个输入和多个输出。有什么建议或提示会很有帮助吗？

最佳答案

目前，create_data_generator尚未定义生成器。试试这个：

def create_data_generator(style_generator,genre_generator,painter_generator):

    while(True):
        _style_generator, _lstyle_generator = next(style_generator)
        _genre_generator, _lgenre_generator = next(genre_generator)
        _painter_generator, _lpainter_generator = next(painter_generator)

        yield [_style_generator,_genre_generator,_painter_generator], [_lstyle_generator,_genre_generator,_painter_generator]