如何在Pytorch中使用torch.nn.Sequential实现我自己的ResNet?

问题描述

我想实现一个ResNet网络(或更确切地说，是剩余块)，但是我真的希望它采用顺序网络形式.

I want to implement a ResNet network (or rather, residual blocks) but I really want it to be in the sequential network form.

依序网络形式的含义如下:

What I mean by sequential network form is the following:

## mdl5, from cifar10 tutorial
mdl5 = nn.Sequential(OrderedDict([
    ('pool1', nn.MaxPool2d(2, 2)),
    ('relu1', nn.ReLU()),
    ('conv1', nn.Conv2d(3, 6, 5)),
    ('pool1', nn.MaxPool2d(2, 2)),
    ('relu2', nn.ReLU()),
    ('conv2', nn.Conv2d(6, 16, 5)),
    ('relu2', nn.ReLU()),
    ('Flatten', Flatten()),
    ('fc1', nn.Linear(1024, 120)), # figure out equation properly
    ('relu4', nn.ReLU()),
    ('fc2', nn.Linear(120, 84)),
    ('relu5', nn.ReLU()),
    ('fc3', nn.Linear(84, 10))
]))

但是，当然，NN lego块是"ResNet".

but of course with the NN lego blocks being "ResNet".

我知道方程式是这样的:

I know the equation is something like:

但是我不确定如何在Pytorch AND Sequential中做到这一点.顺序是我的关键！

but I am not sure how to do it in Pytorch AND Sequential. Sequential is key for me!

交叉发布:

• https://discuss.pytorch.org/t/how-to-have-residual-network-using-only-sequential-blocks/51541
• https://www.quora.com/unanswered/How-does-one-implement-my-own-ResNet-with-torch-nn-Sequential-in-Pytorch

推荐答案

您不能仅使用 torch.nn.Sequential 来完成此操作，因为顾名思义，它需要按顺序进行操作，而您的是平行的.

You can't do it solely using torch.nn.Sequential as it requires operations to go, as the name suggests, sequentially, while yours are parallel.

原则上，您可以像这样轻松地构造自己的 block :

You could, in principle, construct your own block really easily like this:

import torch

class ResNet(torch.nn.Module):
    def __init__(self, module):
        super().__init__()
        self.module = module

    def forward(self, inputs):
        return self.module(inputs) + inputs

哪些人可以使用以下内容:

Which one can use something like this:

model = torch.nn.Sequential(
    torch.nn.Conv2d(3, 32, kernel_size=7),
    # 32 filters in and out, no max pooling so the shapes can be added
    ResNet(
        torch.nn.Sequential(
            torch.nn.Conv2d(32, 32, kernel_size=3),
            torch.nn.ReLU(),
            torch.nn.BatchNorm2d(32),
            torch.nn.Conv2d(32, 32, kernel_size=3),
            torch.nn.ReLU(),
            torch.nn.BatchNorm2d(32),
        )
    ),
    # Another ResNet block, you could make more of them
    # Downsampling using maxpool and others could be done in between etc. etc.
    ResNet(
        torch.nn.Sequential(
            torch.nn.Conv2d(32, 32, kernel_size=3),
            torch.nn.ReLU(),
            torch.nn.BatchNorm2d(32),
            torch.nn.Conv2d(32, 32, kernel_size=3),
            torch.nn.ReLU(),
            torch.nn.BatchNorm2d(32),
        )
    ),
    # Pool all the 32 filters to 1, you may need to use `torch.squeeze after this layer`
    torch.nn.AdaptiveAvgPool2d(1),
    # 32 10 classes
    torch.nn.Linear(32, 10),
)

通常被忽略的事实(当涉及浅层网络时没有实际后果)是应该保留跳过连接而没有任何非线性，例如 ReLU 或卷积层，这就是事实您可以在上方看到(来源:深层残留网络中的身份映射).

Fact being usually overlooked (without real consequences when it comes to shallowe networks) is that skip connection should be left without any nonlinearities like ReLU or convolutional layers and that's what you can see above (source: Identity Mappings in Deep Residual Networks).

这篇关于如何在Pytorch中使用torch.nn.Sequential实现我自己的ResNet?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！