《Deep Residual Learning for Image Recognition》是2016年 kaiming大神CVPR的最佳论文
原文：http://m.blog.csdn.net/justpsss/article/details/77103077
摘要
resNet主要解决一个问题，就是更深的神经网络如何收敛的问题，为了解决这个问题，论文提出了一个残差学习的框架。然后简单跟VGG比较了一下，152层的残差网络，比VGG深了8倍，但是比VGG复杂度更低，当然在ImageNet上的表现肯定比VGG更好，是2015年ILSVRC分类任务的冠军。
另外用resNet作为预训练模型的检测和分割效果也要更好，这个比较好理解，分类效果提升必然带来检测和分割的准确性提升。
介绍
在resNet之前，随着网络层数的增加，收敛越来越难，大家通常把其原因归结为梯度消失或者梯度爆炸，这是不对的。另外当训练网络的时候，也会有这样一个问题，当网络层数加深的时候，准确率可能会快速的下降，这当然也不是由过拟合导致的。我们可以这样理解，构造一个深度模型，我们把新加的层叫做identity mapping（这个mapping实在不知道怎么翻译好，尴尬……），而其他层从学好的浅层模型复制过来。现在我们需要保证这个构造的深度模型并不会比之前的浅层模型产生更高的训练错误，然而目前并没有好的比较方法。
从图上可以看到，层数越多，收敛越慢，且error更高。
在论文中，kaiming大佬提出了一个深度残差学习框架来解决网络加深之后准确率下降的问题。用公式来表示，假如我们需要的理想的mapping定义为H(x)，那么我们新加的非线性层就是F(x):=H(x)−x，原始的mapping就从x变成了F(x)+x。也就是说，如果我们之前的x是最优的，那么新加的identity mapping F(x)就应该都是0，而不会是其他的值。
这样整个残差网络是端对端（end-to-end）的，可以通过随机梯度下降反向传播，而且实现起来很简单（实际上就是两层求和，在Caffe中用Eltwise层实现）。至于它为什么收敛更快，error更低，我是这么理解的：
我们知道随机梯度下降就是用的链式求导法则，我们对H(x)求导，相当于对F(x)+x求导，那么这个梯度值就会在1附近(x的导数是1)，相比之前的plain网络，自然收敛更快。
深度残差学习
假设多个线性和非线性的组合层可以近似任意复杂函数（这是一个开放性的问题），那么当然也可以逼近残差函数H(x)−x（假设输入和输出的维度相同）。
论文中残差模块定义为：
y=F(x,wi)+x
其中，x代表输入，y代表输出，F(x,wi)代表需要学习的残差mapping。像上图firgure 2有两层网络，用F=W2σ(W1x)表示，这里σ表示ReLU激活层。这里Wx是卷积操作，是线性的，ReLU是非线性的。
其中x和F的维度一定要相同，如果不同的话，可以通过一个线性映射Ws来匹配维度：
y=F(x,Wi)+Wsx
这里F是比较灵活的，可以包含两层或者三层，甚至更多层。但是如果只有一层的话，就变成了y=Wix+x，这就是普通的线性函数了，就没有意义了。
接下来就是按照这个思路将网络结构加深了，下面列出几种结构：
最后是一个更深的瓶颈结构问题，论文中用三个1x1,3x3,1x1的卷积层代替前面说的两个3x3卷积层，第一个1x1用来降低维度，第三个1x1用来增加维度，这样可以保证中间的3x3卷积层拥有比较小的输入输出维度。
好了，resNet读到这里基本上差不多了，当然啦，后来又出了resNet的加宽版resNeXt，借鉴了GoogLeNet的思想，以后有机会再细读
最后附图：ResNet-20 和ResNet-50 模型结构，由于模型太大，图像显示不清晰，这里只黏贴很小的一部分：
name: "resnet20_cifar10"
layer {
name: "Input1"
type: "Input"
top: "data"
input_param {
shape {
dim:
dim:
dim:
dim:
}
}
} layer {
name: "conv_0"
type: "Convolution"
bottom: "data"
top: "conv_0"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_0"
type: "BatchNorm"
bottom: "conv_0"
top: "conv_0"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_0"
type: "Scale"
bottom: "conv_0"
top: "conv_0"
scale_param {
bias_term: true
}
}
layer {
name: "relu_0"
type: "ReLU"
bottom: "conv_0"
top: "conv_0"
}
layer {
name: "conv_1"
type: "Convolution"
bottom: "conv_0"
top: "conv_1"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_1"
type: "BatchNorm"
bottom: "conv_1"
top: "conv_1"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_1"
type: "Scale"
bottom: "conv_1"
top: "conv_1"
scale_param {
bias_term: true
}
}
layer {
name: "relu_1"
type: "ReLU"
bottom: "conv_1"
top: "conv_1"
}
layer {
name: "conv_2"
type: "Convolution"
bottom: "conv_1"
top: "conv_2"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_2"
type: "BatchNorm"
bottom: "conv_2"
top: "conv_2"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_2"
type: "Scale"
bottom: "conv_2"
top: "conv_2"
scale_param {
bias_term: true
}
}
layer {
name: "elem_2"
type: "Eltwise"
bottom: "conv_2"
bottom: "conv_0"
top: "elem_2"
eltwise_param {
operation: SUM
}
} layer {
name: "conv_3"
type: "Convolution"
bottom: "elem_2"
top: "conv_3"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_3"
type: "BatchNorm"
bottom: "conv_3"
top: "conv_3"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_3"
type: "Scale"
bottom: "conv_3"
top: "conv_3"
scale_param {
bias_term: true
}
}
layer {
name: "relu_3"
type: "ReLU"
bottom: "conv_3"
top: "conv_3"
}
layer {
name: "conv_4"
type: "Convolution"
bottom: "conv_3"
top: "conv_4"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_4"
type: "BatchNorm"
bottom: "conv_4"
top: "conv_4"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_4"
type: "Scale"
bottom: "conv_4"
top: "conv_4"
scale_param {
bias_term: true
}
}
layer {
name: "elem_4"
type: "Eltwise"
bottom: "conv_4"
bottom: "elem_2"
top: "elem_4"
eltwise_param {
operation: SUM
}
} layer {
name: "conv_5"
type: "Convolution"
bottom: "elem_4"
top: "conv_5"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_5"
type: "BatchNorm"
bottom: "conv_5"
top: "conv_5"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_5"
type: "Scale"
bottom: "conv_5"
top: "conv_5"
scale_param {
bias_term: true
}
}
layer {
name: "relu_5"
type: "ReLU"
bottom: "conv_5"
top: "conv_5"
}
layer {
name: "conv_6"
type: "Convolution"
bottom: "conv_5"
top: "conv_6"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_6"
type: "BatchNorm"
bottom: "conv_6"
top: "conv_6"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_6"
type: "Scale"
bottom: "conv_6"
top: "conv_6"
scale_param {
bias_term: true
}
}
layer {
name: "elem_6"
type: "Eltwise"
bottom: "conv_6"
bottom: "elem_4"
top: "elem_6"
eltwise_param {
operation: SUM
}
} layer {
name: "conv_7"
type: "Convolution"
bottom: "elem_6"
top: "conv_7"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_7"
type: "BatchNorm"
bottom: "conv_7"
top: "conv_7"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_7"
type: "Scale"
bottom: "conv_7"
top: "conv_7"
scale_param {
bias_term: true
}
}
layer {
name: "relu_7"
type: "ReLU"
bottom: "conv_7"
top: "conv_7"
}
layer {
name: "conv_8"
type: "Convolution"
bottom: "conv_7"
top: "conv_8"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_8"
type: "BatchNorm"
bottom: "conv_8"
top: "conv_8"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_8"
type: "Scale"
bottom: "conv_8"
top: "conv_8"
scale_param {
bias_term: true
}
} layer {
name: "proj_7"
type: "Convolution"
bottom: "elem_6"
top: "proj_7"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "proj_norm_7"
type: "BatchNorm"
bottom: "proj_7"
top: "proj_7"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "proj_scale_7"
type: "Scale"
bottom: "proj_7"
top: "proj_7"
scale_param {
bias_term: true
}
} layer {
name: "elem_8"
type: "Eltwise"
bottom: "conv_8"
bottom: "proj_7"
top: "elem_8"
eltwise_param {
operation: SUM
}
} layer {
name: "conv_9"
type: "Convolution"
bottom: "elem_8"
top: "conv_9"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_9"
type: "BatchNorm"
bottom: "conv_9"
top: "conv_9"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_9"
type: "Scale"
bottom: "conv_9"
top: "conv_9"
scale_param {
bias_term: true
}
}
layer {
name: "relu_9"
type: "ReLU"
bottom: "conv_9"
top: "conv_9"
}
layer {
name: "conv_10"
type: "Convolution"
bottom: "conv_9"
top: "conv_10"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_10"
type: "BatchNorm"
bottom: "conv_10"
top: "conv_10"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_10"
type: "Scale"
bottom: "conv_10"
top: "conv_10"
scale_param {
bias_term: true
}
}
layer {
name: "elem_10"
type: "Eltwise"
bottom: "conv_10"
bottom: "elem_8"
top: "elem_10"
eltwise_param {
operation: SUM
}
} layer {
name: "conv_11"
type: "Convolution"
bottom: "elem_10"
top: "conv_11"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_11"
type: "BatchNorm"
bottom: "conv_11"
top: "conv_11"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_11"
type: "Scale"
bottom: "conv_11"
top: "conv_11"
scale_param {
bias_term: true
}
}
layer {
name: "relu_11"
type: "ReLU"
bottom: "conv_11"
top: "conv_11"
}
layer {
name: "conv_12"
type: "Convolution"
bottom: "conv_11"
top: "conv_12"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_12"
type: "BatchNorm"
bottom: "conv_12"
top: "conv_12"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_12"
type: "Scale"
bottom: "conv_12"
top: "conv_12"
scale_param {
bias_term: true
}
}
layer {
name: "elem_12"
type: "Eltwise"
bottom: "conv_12"
bottom: "elem_10"
top: "elem_12"
eltwise_param {
operation: SUM
}
} layer {
name: "conv_13"
type: "Convolution"
bottom: "elem_12"
top: "conv_13"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_13"
type: "BatchNorm"
bottom: "conv_13"
top: "conv_13"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_13"
type: "Scale"
bottom: "conv_13"
top: "conv_13"
scale_param {
bias_term: true
}
}
layer {
name: "relu_13"
type: "ReLU"
bottom: "conv_13"
top: "conv_13"
}
layer {
name: "conv_14"
type: "Convolution"
bottom: "conv_13"
top: "conv_14"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_14"
type: "BatchNorm"
bottom: "conv_14"
top: "conv_14"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_14"
type: "Scale"
bottom: "conv_14"
top: "conv_14"
scale_param {
bias_term: true
}
}
layer {
name: "proj_13"
type: "Convolution"
bottom: "elem_12"
top: "proj_13"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "proj_norm_13"
type: "BatchNorm"
bottom: "proj_13"
top: "proj_13"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "proj_scale_13"
type: "Scale"
bottom: "proj_13"
top: "proj_13"
scale_param {
bias_term: true
}
}
layer {
name: "elem_14"
type: "Eltwise"
bottom: "conv_14"
bottom: "proj_13"
top: "elem_14"
eltwise_param {
operation: SUM
}
} layer {
name: "conv_15"
type: "Convolution"
bottom: "elem_14"
top: "conv_15"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_15"
type: "BatchNorm"
bottom: "conv_15"
top: "conv_15"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_15"
type: "Scale"
bottom: "conv_15"
top: "conv_15"
scale_param {
bias_term: true
}
}
layer {
name: "relu_15"
type: "ReLU"
bottom: "conv_15"
top: "conv_15"
}
layer {
name: "conv_16"
type: "Convolution"
bottom: "conv_15"
top: "conv_16"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_16"
type: "BatchNorm"
bottom: "conv_16"
top: "conv_16"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_16"
type: "Scale"
bottom: "conv_16"
top: "conv_16"
scale_param {
bias_term: true
}
}
layer {
name: "elem_16"
type: "Eltwise"
bottom: "conv_16"
bottom: "elem_14"
top: "elem_16"
eltwise_param {
operation: SUM
}
} layer {
name: "conv_17"
type: "Convolution"
bottom: "elem_16"
top: "conv_17"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_17"
type: "BatchNorm"
bottom: "conv_17"
top: "conv_17"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_17"
type: "Scale"
bottom: "conv_17"
top: "conv_17"
scale_param {
bias_term: true
}
}
layer {
name: "relu_17"
type: "ReLU"
bottom: "conv_17"
top: "conv_17"
}
layer {
name: "conv_18"
type: "Convolution"
bottom: "conv_17"
top: "conv_18"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
convolution_param {
num_output:
pad:
kernel_size:
stride:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
layer {
name: "norm_18"
type: "BatchNorm"
bottom: "conv_18"
top: "conv_18"
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
param {
lr_mult: 0.0
decay_mult: 0.0
}
batch_norm_param {
use_global_stats: true
moving_average_fraction: 0.95
}
}
layer {
name: "scale_18"
type: "Scale"
bottom: "conv_18"
top: "conv_18"
scale_param {
bias_term: true
}
}
layer {
name: "elem_18"
type: "Eltwise"
bottom: "conv_18"
bottom: "elem_16"
top: "elem_18"
eltwise_param {
operation: SUM
}
} layer {
name: "pool_19"
type: "Pooling"
bottom: "elem_18"
top: "pool_19"
pooling_param {
pool: AVE
global_pooling: true
}
}
layer {
name: "fc_19"
type: "InnerProduct"
bottom: "pool_19"
top: "fc_19"
param {
lr_mult: 1.0
decay_mult: 2.0
}
param {
lr_mult: 1.0
decay_mult: 0.0
}
inner_product_param {
num_output:
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value:
}
}
}
name: "ResNet-50"
input: "data"
input_dim:
input_dim:
input_dim:
input_dim: layer {
bottom: "data"
top: "conv1"
name: "conv1"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
}
} layer {
bottom: "conv1"
top: "conv1"
name: "bn_conv1"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "conv1"
top: "conv1"
name: "scale_conv1"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "conv1"
top: "conv1"
name: "conv1_relu"
type: "ReLU"
} layer {
bottom: "conv1"
top: "pool1"
name: "pool1"
type: "Pooling"
pooling_param {
kernel_size:
stride:
pool: MAX
}
} layer {
bottom: "pool1"
top: "res2a_branch1"
name: "res2a_branch1"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2a_branch1"
top: "res2a_branch1"
name: "bn2a_branch1"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2a_branch1"
top: "res2a_branch1"
name: "scale2a_branch1"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "pool1"
top: "res2a_branch2a"
name: "res2a_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2a_branch2a"
top: "res2a_branch2a"
name: "bn2a_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2a_branch2a"
top: "res2a_branch2a"
name: "scale2a_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2a_branch2a"
top: "res2a_branch2a"
name: "res2a_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res2a_branch2a"
top: "res2a_branch2b"
name: "res2a_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2a_branch2b"
top: "res2a_branch2b"
name: "bn2a_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2a_branch2b"
top: "res2a_branch2b"
name: "scale2a_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2a_branch2b"
top: "res2a_branch2b"
name: "res2a_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res2a_branch2b"
top: "res2a_branch2c"
name: "res2a_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2a_branch2c"
top: "res2a_branch2c"
name: "bn2a_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2a_branch2c"
top: "res2a_branch2c"
name: "scale2a_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2a_branch1"
bottom: "res2a_branch2c"
top: "res2a"
name: "res2a"
type: "Eltwise"
} layer {
bottom: "res2a"
top: "res2a"
name: "res2a_relu"
type: "ReLU"
} layer {
bottom: "res2a"
top: "res2b_branch2a"
name: "res2b_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2b_branch2a"
top: "res2b_branch2a"
name: "bn2b_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2b_branch2a"
top: "res2b_branch2a"
name: "scale2b_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2b_branch2a"
top: "res2b_branch2a"
name: "res2b_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res2b_branch2a"
top: "res2b_branch2b"
name: "res2b_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2b_branch2b"
top: "res2b_branch2b"
name: "bn2b_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2b_branch2b"
top: "res2b_branch2b"
name: "scale2b_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2b_branch2b"
top: "res2b_branch2b"
name: "res2b_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res2b_branch2b"
top: "res2b_branch2c"
name: "res2b_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2b_branch2c"
top: "res2b_branch2c"
name: "bn2b_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2b_branch2c"
top: "res2b_branch2c"
name: "scale2b_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2a"
bottom: "res2b_branch2c"
top: "res2b"
name: "res2b"
type: "Eltwise"
} layer {
bottom: "res2b"
top: "res2b"
name: "res2b_relu"
type: "ReLU"
} layer {
bottom: "res2b"
top: "res2c_branch2a"
name: "res2c_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2c_branch2a"
top: "res2c_branch2a"
name: "bn2c_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2c_branch2a"
top: "res2c_branch2a"
name: "scale2c_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2c_branch2a"
top: "res2c_branch2a"
name: "res2c_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res2c_branch2a"
top: "res2c_branch2b"
name: "res2c_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2c_branch2b"
top: "res2c_branch2b"
name: "bn2c_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2c_branch2b"
top: "res2c_branch2b"
name: "scale2c_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2c_branch2b"
top: "res2c_branch2b"
name: "res2c_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res2c_branch2b"
top: "res2c_branch2c"
name: "res2c_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res2c_branch2c"
top: "res2c_branch2c"
name: "bn2c_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res2c_branch2c"
top: "res2c_branch2c"
name: "scale2c_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2b"
bottom: "res2c_branch2c"
top: "res2c"
name: "res2c"
type: "Eltwise"
} layer {
bottom: "res2c"
top: "res2c"
name: "res2c_relu"
type: "ReLU"
} layer {
bottom: "res2c"
top: "res3a_branch1"
name: "res3a_branch1"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3a_branch1"
top: "res3a_branch1"
name: "bn3a_branch1"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3a_branch1"
top: "res3a_branch1"
name: "scale3a_branch1"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res2c"
top: "res3a_branch2a"
name: "res3a_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3a_branch2a"
top: "res3a_branch2a"
name: "bn3a_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3a_branch2a"
top: "res3a_branch2a"
name: "scale3a_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3a_branch2a"
top: "res3a_branch2a"
name: "res3a_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res3a_branch2a"
top: "res3a_branch2b"
name: "res3a_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3a_branch2b"
top: "res3a_branch2b"
name: "bn3a_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3a_branch2b"
top: "res3a_branch2b"
name: "scale3a_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3a_branch2b"
top: "res3a_branch2b"
name: "res3a_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res3a_branch2b"
top: "res3a_branch2c"
name: "res3a_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3a_branch2c"
top: "res3a_branch2c"
name: "bn3a_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3a_branch2c"
top: "res3a_branch2c"
name: "scale3a_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3a_branch1"
bottom: "res3a_branch2c"
top: "res3a"
name: "res3a"
type: "Eltwise"
} layer {
bottom: "res3a"
top: "res3a"
name: "res3a_relu"
type: "ReLU"
} layer {
bottom: "res3a"
top: "res3b_branch2a"
name: "res3b_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3b_branch2a"
top: "res3b_branch2a"
name: "bn3b_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3b_branch2a"
top: "res3b_branch2a"
name: "scale3b_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3b_branch2a"
top: "res3b_branch2a"
name: "res3b_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res3b_branch2a"
top: "res3b_branch2b"
name: "res3b_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3b_branch2b"
top: "res3b_branch2b"
name: "bn3b_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3b_branch2b"
top: "res3b_branch2b"
name: "scale3b_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3b_branch2b"
top: "res3b_branch2b"
name: "res3b_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res3b_branch2b"
top: "res3b_branch2c"
name: "res3b_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3b_branch2c"
top: "res3b_branch2c"
name: "bn3b_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3b_branch2c"
top: "res3b_branch2c"
name: "scale3b_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3a"
bottom: "res3b_branch2c"
top: "res3b"
name: "res3b"
type: "Eltwise"
} layer {
bottom: "res3b"
top: "res3b"
name: "res3b_relu"
type: "ReLU"
} layer {
bottom: "res3b"
top: "res3c_branch2a"
name: "res3c_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3c_branch2a"
top: "res3c_branch2a"
name: "bn3c_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3c_branch2a"
top: "res3c_branch2a"
name: "scale3c_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3c_branch2a"
top: "res3c_branch2a"
name: "res3c_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res3c_branch2a"
top: "res3c_branch2b"
name: "res3c_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3c_branch2b"
top: "res3c_branch2b"
name: "bn3c_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3c_branch2b"
top: "res3c_branch2b"
name: "scale3c_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3c_branch2b"
top: "res3c_branch2b"
name: "res3c_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res3c_branch2b"
top: "res3c_branch2c"
name: "res3c_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3c_branch2c"
top: "res3c_branch2c"
name: "bn3c_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3c_branch2c"
top: "res3c_branch2c"
name: "scale3c_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3b"
bottom: "res3c_branch2c"
top: "res3c"
name: "res3c"
type: "Eltwise"
} layer {
bottom: "res3c"
top: "res3c"
name: "res3c_relu"
type: "ReLU"
} layer {
bottom: "res3c"
top: "res3d_branch2a"
name: "res3d_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3d_branch2a"
top: "res3d_branch2a"
name: "bn3d_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3d_branch2a"
top: "res3d_branch2a"
name: "scale3d_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3d_branch2a"
top: "res3d_branch2a"
name: "res3d_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res3d_branch2a"
top: "res3d_branch2b"
name: "res3d_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3d_branch2b"
top: "res3d_branch2b"
name: "bn3d_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3d_branch2b"
top: "res3d_branch2b"
name: "scale3d_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3d_branch2b"
top: "res3d_branch2b"
name: "res3d_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res3d_branch2b"
top: "res3d_branch2c"
name: "res3d_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res3d_branch2c"
top: "res3d_branch2c"
name: "bn3d_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res3d_branch2c"
top: "res3d_branch2c"
name: "scale3d_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3c"
bottom: "res3d_branch2c"
top: "res3d"
name: "res3d"
type: "Eltwise"
} layer {
bottom: "res3d"
top: "res3d"
name: "res3d_relu"
type: "ReLU"
} layer {
bottom: "res3d"
top: "res4a_branch1"
name: "res4a_branch1"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4a_branch1"
top: "res4a_branch1"
name: "bn4a_branch1"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4a_branch1"
top: "res4a_branch1"
name: "scale4a_branch1"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res3d"
top: "res4a_branch2a"
name: "res4a_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4a_branch2a"
top: "res4a_branch2a"
name: "bn4a_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4a_branch2a"
top: "res4a_branch2a"
name: "scale4a_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4a_branch2a"
top: "res4a_branch2a"
name: "res4a_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res4a_branch2a"
top: "res4a_branch2b"
name: "res4a_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4a_branch2b"
top: "res4a_branch2b"
name: "bn4a_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4a_branch2b"
top: "res4a_branch2b"
name: "scale4a_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4a_branch2b"
top: "res4a_branch2b"
name: "res4a_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res4a_branch2b"
top: "res4a_branch2c"
name: "res4a_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4a_branch2c"
top: "res4a_branch2c"
name: "bn4a_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4a_branch2c"
top: "res4a_branch2c"
name: "scale4a_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4a_branch1"
bottom: "res4a_branch2c"
top: "res4a"
name: "res4a"
type: "Eltwise"
} layer {
bottom: "res4a"
top: "res4a"
name: "res4a_relu"
type: "ReLU"
} layer {
bottom: "res4a"
top: "res4b_branch2a"
name: "res4b_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4b_branch2a"
top: "res4b_branch2a"
name: "bn4b_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4b_branch2a"
top: "res4b_branch2a"
name: "scale4b_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4b_branch2a"
top: "res4b_branch2a"
name: "res4b_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res4b_branch2a"
top: "res4b_branch2b"
name: "res4b_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4b_branch2b"
top: "res4b_branch2b"
name: "bn4b_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4b_branch2b"
top: "res4b_branch2b"
name: "scale4b_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4b_branch2b"
top: "res4b_branch2b"
name: "res4b_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res4b_branch2b"
top: "res4b_branch2c"
name: "res4b_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4b_branch2c"
top: "res4b_branch2c"
name: "bn4b_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4b_branch2c"
top: "res4b_branch2c"
name: "scale4b_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4a"
bottom: "res4b_branch2c"
top: "res4b"
name: "res4b"
type: "Eltwise"
} layer {
bottom: "res4b"
top: "res4b"
name: "res4b_relu"
type: "ReLU"
} layer {
bottom: "res4b"
top: "res4c_branch2a"
name: "res4c_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4c_branch2a"
top: "res4c_branch2a"
name: "bn4c_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4c_branch2a"
top: "res4c_branch2a"
name: "scale4c_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4c_branch2a"
top: "res4c_branch2a"
name: "res4c_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res4c_branch2a"
top: "res4c_branch2b"
name: "res4c_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4c_branch2b"
top: "res4c_branch2b"
name: "bn4c_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4c_branch2b"
top: "res4c_branch2b"
name: "scale4c_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4c_branch2b"
top: "res4c_branch2b"
name: "res4c_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res4c_branch2b"
top: "res4c_branch2c"
name: "res4c_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4c_branch2c"
top: "res4c_branch2c"
name: "bn4c_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4c_branch2c"
top: "res4c_branch2c"
name: "scale4c_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4b"
bottom: "res4c_branch2c"
top: "res4c"
name: "res4c"
type: "Eltwise"
} layer {
bottom: "res4c"
top: "res4c"
name: "res4c_relu"
type: "ReLU"
} layer {
bottom: "res4c"
top: "res4d_branch2a"
name: "res4d_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4d_branch2a"
top: "res4d_branch2a"
name: "bn4d_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4d_branch2a"
top: "res4d_branch2a"
name: "scale4d_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4d_branch2a"
top: "res4d_branch2a"
name: "res4d_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res4d_branch2a"
top: "res4d_branch2b"
name: "res4d_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4d_branch2b"
top: "res4d_branch2b"
name: "bn4d_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4d_branch2b"
top: "res4d_branch2b"
name: "scale4d_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4d_branch2b"
top: "res4d_branch2b"
name: "res4d_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res4d_branch2b"
top: "res4d_branch2c"
name: "res4d_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4d_branch2c"
top: "res4d_branch2c"
name: "bn4d_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4d_branch2c"
top: "res4d_branch2c"
name: "scale4d_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4c"
bottom: "res4d_branch2c"
top: "res4d"
name: "res4d"
type: "Eltwise"
} layer {
bottom: "res4d"
top: "res4d"
name: "res4d_relu"
type: "ReLU"
} layer {
bottom: "res4d"
top: "res4e_branch2a"
name: "res4e_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4e_branch2a"
top: "res4e_branch2a"
name: "bn4e_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4e_branch2a"
top: "res4e_branch2a"
name: "scale4e_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4e_branch2a"
top: "res4e_branch2a"
name: "res4e_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res4e_branch2a"
top: "res4e_branch2b"
name: "res4e_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4e_branch2b"
top: "res4e_branch2b"
name: "bn4e_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4e_branch2b"
top: "res4e_branch2b"
name: "scale4e_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4e_branch2b"
top: "res4e_branch2b"
name: "res4e_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res4e_branch2b"
top: "res4e_branch2c"
name: "res4e_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4e_branch2c"
top: "res4e_branch2c"
name: "bn4e_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4e_branch2c"
top: "res4e_branch2c"
name: "scale4e_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4d"
bottom: "res4e_branch2c"
top: "res4e"
name: "res4e"
type: "Eltwise"
} layer {
bottom: "res4e"
top: "res4e"
name: "res4e_relu"
type: "ReLU"
} layer {
bottom: "res4e"
top: "res4f_branch2a"
name: "res4f_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4f_branch2a"
top: "res4f_branch2a"
name: "bn4f_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4f_branch2a"
top: "res4f_branch2a"
name: "scale4f_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4f_branch2a"
top: "res4f_branch2a"
name: "res4f_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res4f_branch2a"
top: "res4f_branch2b"
name: "res4f_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4f_branch2b"
top: "res4f_branch2b"
name: "bn4f_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4f_branch2b"
top: "res4f_branch2b"
name: "scale4f_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4f_branch2b"
top: "res4f_branch2b"
name: "res4f_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res4f_branch2b"
top: "res4f_branch2c"
name: "res4f_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res4f_branch2c"
top: "res4f_branch2c"
name: "bn4f_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res4f_branch2c"
top: "res4f_branch2c"
name: "scale4f_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4e"
bottom: "res4f_branch2c"
top: "res4f"
name: "res4f"
type: "Eltwise"
} layer {
bottom: "res4f"
top: "res4f"
name: "res4f_relu"
type: "ReLU"
} layer {
bottom: "res4f"
top: "res5a_branch1"
name: "res5a_branch1"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5a_branch1"
top: "res5a_branch1"
name: "bn5a_branch1"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5a_branch1"
top: "res5a_branch1"
name: "scale5a_branch1"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res4f"
top: "res5a_branch2a"
name: "res5a_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5a_branch2a"
top: "res5a_branch2a"
name: "bn5a_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5a_branch2a"
top: "res5a_branch2a"
name: "scale5a_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5a_branch2a"
top: "res5a_branch2a"
name: "res5a_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res5a_branch2a"
top: "res5a_branch2b"
name: "res5a_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5a_branch2b"
top: "res5a_branch2b"
name: "bn5a_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5a_branch2b"
top: "res5a_branch2b"
name: "scale5a_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5a_branch2b"
top: "res5a_branch2b"
name: "res5a_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res5a_branch2b"
top: "res5a_branch2c"
name: "res5a_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5a_branch2c"
top: "res5a_branch2c"
name: "bn5a_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5a_branch2c"
top: "res5a_branch2c"
name: "scale5a_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5a_branch1"
bottom: "res5a_branch2c"
top: "res5a"
name: "res5a"
type: "Eltwise"
} layer {
bottom: "res5a"
top: "res5a"
name: "res5a_relu"
type: "ReLU"
} layer {
bottom: "res5a"
top: "res5b_branch2a"
name: "res5b_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5b_branch2a"
top: "res5b_branch2a"
name: "bn5b_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5b_branch2a"
top: "res5b_branch2a"
name: "scale5b_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5b_branch2a"
top: "res5b_branch2a"
name: "res5b_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res5b_branch2a"
top: "res5b_branch2b"
name: "res5b_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5b_branch2b"
top: "res5b_branch2b"
name: "bn5b_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5b_branch2b"
top: "res5b_branch2b"
name: "scale5b_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5b_branch2b"
top: "res5b_branch2b"
name: "res5b_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res5b_branch2b"
top: "res5b_branch2c"
name: "res5b_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5b_branch2c"
top: "res5b_branch2c"
name: "bn5b_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5b_branch2c"
top: "res5b_branch2c"
name: "scale5b_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5a"
bottom: "res5b_branch2c"
top: "res5b"
name: "res5b"
type: "Eltwise"
} layer {
bottom: "res5b"
top: "res5b"
name: "res5b_relu"
type: "ReLU"
} layer {
bottom: "res5b"
top: "res5c_branch2a"
name: "res5c_branch2a"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5c_branch2a"
top: "res5c_branch2a"
name: "bn5c_branch2a"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5c_branch2a"
top: "res5c_branch2a"
name: "scale5c_branch2a"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5c_branch2a"
top: "res5c_branch2a"
name: "res5c_branch2a_relu"
type: "ReLU"
} layer {
bottom: "res5c_branch2a"
top: "res5c_branch2b"
name: "res5c_branch2b"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5c_branch2b"
top: "res5c_branch2b"
name: "bn5c_branch2b"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5c_branch2b"
top: "res5c_branch2b"
name: "scale5c_branch2b"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5c_branch2b"
top: "res5c_branch2b"
name: "res5c_branch2b_relu"
type: "ReLU"
} layer {
bottom: "res5c_branch2b"
top: "res5c_branch2c"
name: "res5c_branch2c"
type: "Convolution"
convolution_param {
num_output:
kernel_size:
pad:
stride:
bias_term: false
}
} layer {
bottom: "res5c_branch2c"
top: "res5c_branch2c"
name: "bn5c_branch2c"
type: "BatchNorm"
batch_norm_param {
use_global_stats: true
}
} layer {
bottom: "res5c_branch2c"
top: "res5c_branch2c"
name: "scale5c_branch2c"
type: "Scale"
scale_param {
bias_term: true
}
} layer {
bottom: "res5b"
bottom: "res5c_branch2c"
top: "res5c"
name: "res5c"
type: "Eltwise"
} layer {
bottom: "res5c"
top: "res5c"
name: "res5c_relu"
type: "ReLU"
} layer {
bottom: "res5c"
top: "pool5"
name: "pool5"
type: "Pooling"
pooling_param {
kernel_size:
stride:
pool: AVE
}
} layer {
bottom: "pool5"
top: "fc1000"
name: "fc1000"
type: "InnerProduct"
inner_product_param {
num_output:
}
} layer {
bottom: "fc1000"
top: "prob"
name: "prob"
type: "Softmax"
}
ResNet caffe 中为什么bn层要和scale层一起使用
https://zhidao.baidu.com/question/621624946902864092.html
这个问题首先你要理解batchnormal是做什么的。它其实做了两件事。
1) 输入归一化 x_norm = (x-u)/std, 其中u和std是个累计计算的均值和方差。
2）y=alpha×x_norm + beta，对归一化后的x进行比例缩放和位移。其中alpha和beta是通过迭代学习的。
那么caffe中的bn层其实只做了第一件事。scale层做了第二件事。
这样你也就理解了scale层里为什么要设置bias_term=True，这个偏置就对应2）件事里的beta。
网络结构文件中BatchNorm层的参数要注意
1.在训练时所有BN层要设置use_global_stats: false（也可以不写，caffe默认是false）
2.在测试时所有BN层要设置use_global_stats: true
影响：
1.训练如果不设为false，会导致模型不收敛
2.测试如果不设置为true，会导致准确率极低
(亲测，测试时为false时acc=0.05，为true时acc=0.91)
区别：
use_global_stats: false是使用了每个Batch里的数据的均值和方差；
use_global_stats: true是使用了所有数据的均值和方差。