利用keras实现MobileNet，并以mnist数据集作为一个小例子进行识别。使用的环境是:tensorflow-gpu 2.0,python=3.7 , GTX-2070的GPU
1.导入数据
- 首先是导入两行魔法命令，可以多行显示.
%config InteractiveShell.ast_node_interactivity="all"
%pprint
- 加载keras中自带的mnist数据
import tensorflow as tf
import keras
tf.debugging.set_log_device_placement(True)
mnist = keras.datasets.mnist
(x_train,y_train),(x_test,y_test) = mnist.load_data()
上述tf.debugging.set_log_device_placement(True)的作用是将模型放在GPU上进行训练。
- 数据的转换
在mnist上下载的数据的分辨率是2828的，mobilenet用来训练的数据是ImageNet ，其图片的分辨率是224224，所以先将图片的维度调整为224*224.
from PIL import Image
import numpy as np
def convert_mnist_224pix(X):
img=Image.fromarray(X)
x=np.zeros((224,224))
img=np.array(img.resize((224,224)))
x[:,:]=img
return x
iteration = iter(x_train)
new_train =np.zeros((len(x_train),224,224),dtype=np.float32)
for i in range(len(x_train)):
data = next(iteration)
new_train[i]=convert_mnist_224pix(data)
if i%5000==0:
print(i)
new_train.shape
这里要注意一下，new_train中一定要注明dtype=np.float32，不然默认的是float64，这样数据就太大了，没有那么多存储空间装。最后输出的维度是（60000,224，224）
2.搭建模型
- 导入所有需要的函数和库
from keras.layers import Conv2D,DepthwiseConv2D,Dense,AveragePooling2D,BatchNormalization,Input
from keras import Model
from keras import Sequential
from keras.layers.advanced_activations import ReLU
from keras.utils import to_categorical
- 自己定义中间可以重复利用的层，将其放在一起，简化搭建网络的重复代码。
def depth_point_conv2d(x,s=[1,1,2,1],channel=[64,128]):
"""
s:the strides of the conv
channel: the depth of pointwiseconvolutions
"""
dw1 = DepthwiseConv2D((3,3),strides=s[0],padding='same')(x)
bn1 = BatchNormalization()(dw1)
relu1 = ReLU()(bn1)
pw1 = Conv2D(channel[0],(1,1),strides=s[1],padding='same')(relu1)
bn2 = BatchNormalization()(pw1)
relu2 = ReLU()(bn2)
dw2 = DepthwiseConv2D((3,3),strides=s[2],padding='same')(relu2)
bn3 = BatchNormalization()(dw2)
relu3 = ReLU()(bn3)
pw2 = Conv2D(channel[1],(1,1),strides=s[3],padding='same')(relu3)
bn4 = BatchNormalization()(pw2)
relu4 = ReLU()(bn4)
return relu4
def repeat_conv(x,s=[1,1],channel=512):
dw1 = DepthwiseConv2D((3,3),strides=s[0],padding='same')(x)
bn1 = BatchNormalization()(dw1)
relu1 = ReLU()(bn1)
pw1 = Conv2D(channel,(1,1),strides=s[1],padding='same')(relu1)
bn2 = BatchNormalization()(pw1)
relu2 = ReLU()(bn2)
return relu2
根据mobilenet论文中的结构进行模型的搭建在倒数第5行Conv/dw/s2中，我一直不理解如果strides=2,为什么最后生成图片尺寸没有变化，我感觉可能是笔误？，不过我这里将strides定义为1，因为这样才符合后面的整个输出。
- 搭建网络
h0=Input(shape=(224,224,1))
h1=Conv2D(32,(3,3),strides = 2,padding="same")(h0)
h2= BatchNormalization()(h1)
h3=ReLU()(h2)
h4 = depth_point_conv2d(h3,s=[1,1,2,1],channel=[64,128])
h5 = depth_point_conv2d(h4,s=[1,1,2,1],channel=[128,256])
h6 = depth_point_conv2d(h5,s=[1,1,2,1],channel=[256,512])
h7 = repeat_conv(h6)
h8 = repeat_conv(h7)
h9 = repeat_conv(h8)
h10 = repeat_conv(h9)
h11 = depth_point_conv2d(h10,s=[1,1,2,1],channel=[512,1024])
h12 = repeat_conv(h11,channel=1024)
h13 = AveragePooling2D((7,7))(h12)
h14 = Dense(10,activation='softmax')(h13)
model =Model(input=h0,output =h14)
model.summary()
Model: "model_4"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_11 (InputLayer) (None, 224, 224, 1) 0
_________________________________________________________________
conv2d_63 (Conv2D) (None, 112, 112, 32) 320
_________________________________________________________________
batch_normalization_120 (Bat (None, 112, 112, 32) 128
_________________________________________________________________
re_lu_120 (ReLU) (None, 112, 112, 32) 0
_________________________________________________________________
depthwise_conv2d_58 (Depthwi (None, 112, 112, 32) 320
_________________________________________________________________
batch_normalization_121 (Bat (None, 112, 112, 32) 128
_________________________________________________________________
re_lu_121 (ReLU) (None, 112, 112, 32) 0
_________________________________________________________________
conv2d_64 (Conv2D) (None, 112, 112, 64) 2112
_________________________________________________________________
batch_normalization_122 (Bat (None, 112, 112, 64) 256
_________________________________________________________________
re_lu_122 (ReLU) (None, 112, 112, 64) 0
_________________________________________________________________
depthwise_conv2d_59 (Depthwi (None, 56, 56, 64) 640
_________________________________________________________________
batch_normalization_123 (Bat (None, 56, 56, 64) 256
_________________________________________________________________
re_lu_123 (ReLU) (None, 56, 56, 64) 0
_________________________________________________________________
conv2d_65 (Conv2D) (None, 56, 56, 128) 8320
_________________________________________________________________
batch_normalization_124 (Bat (None, 56, 56, 128) 512
_________________________________________________________________
re_lu_124 (ReLU) (None, 56, 56, 128) 0
_________________________________________________________________
depthwise_conv2d_60 (Depthwi (None, 56, 56, 128) 1280
_________________________________________________________________
batch_normalization_125 (Bat (None, 56, 56, 128) 512
_________________________________________________________________
re_lu_125 (ReLU) (None, 56, 56, 128) 0
_________________________________________________________________
conv2d_66 (Conv2D) (None, 56, 56, 128) 16512
_________________________________________________________________
batch_normalization_126 (Bat (None, 56, 56, 128) 512
_________________________________________________________________
re_lu_126 (ReLU) (None, 56, 56, 128) 0
_________________________________________________________________
depthwise_conv2d_61 (Depthwi (None, 28, 28, 128) 1280
_________________________________________________________________
batch_normalization_127 (Bat (None, 28, 28, 128) 512
_________________________________________________________________
re_lu_127 (ReLU) (None, 28, 28, 128) 0
_________________________________________________________________
conv2d_67 (Conv2D) (None, 28, 28, 256) 33024
_________________________________________________________________
batch_normalization_128 (Bat (None, 28, 28, 256) 1024
_________________________________________________________________
re_lu_128 (ReLU) (None, 28, 28, 256) 0
_________________________________________________________________
depthwise_conv2d_62 (Depthwi (None, 28, 28, 256) 2560
_________________________________________________________________
batch_normalization_129 (Bat (None, 28, 28, 256) 1024
_________________________________________________________________
re_lu_129 (ReLU) (None, 28, 28, 256) 0
_________________________________________________________________
conv2d_68 (Conv2D) (None, 28, 28, 256) 65792
_________________________________________________________________
batch_normalization_130 (Bat (None, 28, 28, 256) 1024
_________________________________________________________________
re_lu_130 (ReLU) (None, 28, 28, 256) 0
_________________________________________________________________
depthwise_conv2d_63 (Depthwi (None, 14, 14, 256) 2560
_________________________________________________________________
batch_normalization_131 (Bat (None, 14, 14, 256) 1024
_________________________________________________________________
re_lu_131 (ReLU) (None, 14, 14, 256) 0
_________________________________________________________________
conv2d_69 (Conv2D) (None, 14, 14, 512) 131584
_________________________________________________________________
batch_normalization_132 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_132 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
depthwise_conv2d_64 (Depthwi (None, 14, 14, 512) 5120
_________________________________________________________________
batch_normalization_133 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_133 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
conv2d_70 (Conv2D) (None, 14, 14, 512) 262656
_________________________________________________________________
batch_normalization_134 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_134 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
depthwise_conv2d_65 (Depthwi (None, 14, 14, 512) 5120
_________________________________________________________________
batch_normalization_135 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_135 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
conv2d_71 (Conv2D) (None, 14, 14, 512) 262656
_________________________________________________________________
batch_normalization_136 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_136 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
depthwise_conv2d_66 (Depthwi (None, 14, 14, 512) 5120
_________________________________________________________________
batch_normalization_137 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_137 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
conv2d_72 (Conv2D) (None, 14, 14, 512) 262656
_________________________________________________________________
batch_normalization_138 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_138 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
depthwise_conv2d_67 (Depthwi (None, 14, 14, 512) 5120
_________________________________________________________________
batch_normalization_139 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_139 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
conv2d_73 (Conv2D) (None, 14, 14, 512) 262656
_________________________________________________________________
batch_normalization_140 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_140 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
depthwise_conv2d_68 (Depthwi (None, 14, 14, 512) 5120
_________________________________________________________________
batch_normalization_141 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_141 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
conv2d_74 (Conv2D) (None, 14, 14, 512) 262656
_________________________________________________________________
batch_normalization_142 (Bat (None, 14, 14, 512) 2048
_________________________________________________________________
re_lu_142 (ReLU) (None, 14, 14, 512) 0
_________________________________________________________________
depthwise_conv2d_69 (Depthwi (None, 7, 7, 512) 5120
_________________________________________________________________
batch_normalization_143 (Bat (None, 7, 7, 512) 2048
_________________________________________________________________
re_lu_143 (ReLU) (None, 7, 7, 512) 0
_________________________________________________________________
conv2d_75 (Conv2D) (None, 7, 7, 1024) 525312
_________________________________________________________________
batch_normalization_144 (Bat (None, 7, 7, 1024) 4096
_________________________________________________________________
re_lu_144 (ReLU) (None, 7, 7, 1024) 0
_________________________________________________________________
depthwise_conv2d_70 (Depthwi (None, 7, 7, 1024) 10240
_________________________________________________________________
batch_normalization_145 (Bat (None, 7, 7, 1024) 4096
_________________________________________________________________
re_lu_145 (ReLU) (None, 7, 7, 1024) 0
_________________________________________________________________
conv2d_76 (Conv2D) (None, 7, 7, 1024) 1049600
_________________________________________________________________
batch_normalization_146 (Bat (None, 7, 7, 1024) 4096
_________________________________________________________________
re_lu_146 (ReLU) (None, 7, 7, 1024) 0
_________________________________________________________________
average_pooling2d_5 (Average (None, 1, 1, 1024) 0
_________________________________________________________________
dense_4 (Dense) (None, 1, 1, 10) 10250
=================================================================
Total params: 3,249,482
Trainable params: 3,227,594
Non-trainable params: 21,888
_________________________________________________________________
因为这里的类别只有10类，所以最后的输出层只有10个神经元，原始的mobilenet要进行1000个类别分类，所以最后是1000个神经元。
model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])
上述代码定义优化算法和损失函数。
3、训练数据的整理与训练
将训练数据进行维度变换，标签进行one-hot编码并进行维度变换。
x_train = np.expand_dims(new_train,3)
y_train = to_categorical(y_train)
y=np.expand_dims(y_train,1)
y = np.expand_dims(y,1)
- 定义数据生成函数
def data_generate(x_train,y_train,batch_size,epochs):
for i in range(epochs):
batch_num = len(x_train)//batch_size
shuffle_index = np.arange(batch_num)
np.random.shuffle(shuffle_index)
for j in shuffle_index:
begin = j*batch_size
end =begin+batch_size
x = x_train[begin:end]
y = y_train[begin:end]
yield ({"input_11":x},{"dense_4":y})
上述命名和model中的第一层和最后一层名字一样，不然会报错。
- 开始训练
model.fit_generator(data_generate(x_train,y,100,11),step_per_epoch=600,epochs=10)
训练过程图如下：
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Epoch 1/10
Executing op __inference_keras_scratch_graph_22639 in device /job:localhost/replica:0/task:0/device:GPU:0
600/600 [==============================] - 411s 684ms/step - loss: 0.1469 - accuracy: 0.9529
Epoch 2/10
600/600 [==============================] - 398s 663ms/step - loss: 0.0375 - accuracy: 0.9884
Epoch 3/10
600/600 [==============================] - 401s 668ms/step - loss: 0.0283 - accuracy: 0.9909
Epoch 4/10
600/600 [==============================] - 399s 665ms/step - loss: 0.0211 - accuracy: 0.9936
Epoch 5/10
600/600 [==============================] - 400s 666ms/step - loss: 0.0216 - accuracy: 0.9932
Epoch 6/10
600/600 [==============================] - 401s 668ms/step - loss: 0.0208 - accuracy: 0.9935
Epoch 7/10
600/600 [==============================] - 401s 669ms/step - loss: 0.0174 - accuracy: 0.9945
Epoch 8/10
131/600 [=====>........................] - ETA: 5:13 - loss: 0.0091 - accuracy: 0.9973
模型卷积比较多，需要训练的时间有点长，参数不多，所以更新较快，收敛速度也很快。