我一直试图重新创建在this blog post中完成的工作。writeup非常全面,通过协作共享code。
我要做的是从预先训练好的VGG19网络中提取层,并创建一个以这些层作为输出的新网络。然而,当我组装新网络时,它与VGG19网络非常相似,似乎包含了我没有提取的层。下面是一个例子。
import tensorflow as tf
from tensorflow.python.keras import models
## Create network based on VGG19 arch with pretrained weights
vgg = tf.keras.applications.vgg19.VGG19(include_top=False, weights='imagenet')
vgg.trainable = False
当我们查看VGG19的概要时,我们看到了我们所期望的架构。
vgg.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) (None, None, None, 3) 0
_________________________________________________________________
block1_conv1 (Conv2D) (None, None, None, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, None, None, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, None, None, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, None, None, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, None, None, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, None, None, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, None, None, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_conv4 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, None, None, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, None, None, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_conv4 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, None, None, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_conv3 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_conv4 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, None, None, 512) 0
=================================================================
Total params: 20,024,384
Trainable params: 0
Non-trainable params: 20,024,384
_________________________________________________________________
然后,我们提取图层并创建一个新模型
## Layers to extract
content_layers = ['block5_conv2']
style_layers = ['block1_conv1','block2_conv1','block3_conv1','block4_conv1','block5_conv1']
## Get output layers corresponding to style and content layers
style_outputs = [vgg.get_layer(name).output for name in style_layers]
content_outputs = [vgg.get_layer(name).output for name in content_layers]
model_outputs = style_outputs + content_outputs
new_model = models.Model(vgg.input, model_outputs)
当创建
new_model
时,我相信我们应该有一个更小的模型。然而,对该模型的总结表明,新模型非常接近原始模型(它包含VGG19的22层中的19层),并且它包含了我们没有提取的层。new_model.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) (None, None, None, 3) 0
_________________________________________________________________
block1_conv1 (Conv2D) (None, None, None, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, None, None, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, None, None, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, None, None, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, None, None, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, None, None, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, None, None, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_conv4 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, None, None, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, None, None, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_conv4 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, None, None, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, None, None, 512) 2359808
=================================================================
Total params: 15,304,768
Trainable params: 15,304,768
Non-trainable params: 0
_________________________________________________________________
所以我的问题是。。。
为什么我没有提取的层会出现在
new_model
中。这些是由模型的实例化过程推断出来的吗?这似乎太接近VGG19架构,无法推断。根据我对Keras'per the docs的理解,传递多个输出层应该创建一个具有多个输出的模型,但是,新模型似乎是连续的,并且只有一个输出层。是这样吗?
最佳答案
为什么我没有提取的层会出现在new_model
中。
这是因为当您使用models.Model(vgg.input, model_outputs)
创建模型时,vgg.input
和输出层之间的“中间”层也包括在内。这是VGG按这种方式构建的预期方式。
例如,如果要以这种方式创建模型:models.Model(vgg.input, vgg.get_layer('block2_pool')
将包括input_1
和block2_pool
之间的每个中间层,因为输入必须在到达block2_pool
之前流过它们。下面是VGG的一个局部图,它可以帮助解决这个问题。
现在,-如果我没有误解-如果你想创建一个不包括那些中间层的模型(这可能会很糟糕),你必须自己创建一个。函数式API在这方面非常有用。documentation上有一些示例,但您要做的要点如下:
from keras.layers import Conv2D, Input
x_input = Input(shape=(28, 28, 1,))
block1_conv1 = Conv2D(64, (3, 3), padding='same')(x_input)
block2_conv2 = Conv2D(128, (3, 3), padding='same')(x_input)
...
new_model = models.Model(x_input, [block1_conv1, block2_conv2, ...])
... 然而,新模型似乎是连续的,只有一个输出层。是这样吗?
不,您的模型有您想要的多个输出。
model.summary()
应该显示哪些层连接到了什么(这将有助于理解结构),但我相信有些版本有一个小错误阻止了这一点。在任何情况下,通过检查new_model.output
,您都可以看到您的模型有多个输出,这将为您提供:[<tf.Tensor 'block1_conv1/Relu:0' shape=(?, ?, ?, 64) dtype=float32>,
<tf.Tensor 'block2_conv1/Relu:0' shape=(?, ?, ?, 128) dtype=float32>,
<tf.Tensor 'block3_conv1/Relu:0' shape=(?, ?, ?, 256) dtype=float32>,
<tf.Tensor 'block4_conv1/Relu:0' shape=(?, ?, ?, 512) dtype=float32>,
<tf.Tensor 'block5_conv1/Relu:0' shape=(?, ?, ?, 512) dtype=float32>,
<tf.Tensor 'block5_conv2/Relu:0' shape=(?, ?, ?, 512) dtype=float32>]
在
new_model.summary()
中按顺序打印它只是一种设计选择,因为它在复杂的模型中会变得毛茸茸的。