通道注意力机制SENet

SENet(Squeeze-and-Excitation Networks，SENet)是ImageNet2017分类比赛的冠军模型，主要思想为对H×W×C的feature map进行处理，得到一个1×1×C的权重，再乘回去得到最终的feature map。具体结构如下图所示：

• Sequeeze：对H×W×C的feature map进行global average pooling，得到具有全局感受野的feature map（1×1×C）
• Excitation：通过全连接层和Sigmoid进行非线性变换，得到每个通道的权重
• Scale：H×W×C的feature map乘以1×1×C的权重，得到最终的feature map（H×W×C）

# SENet的Pytorch实现
class SELayer(nn.Module):
    def __init__(self, channel, reduction=16):
        super(SELayer, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Sequential(
            nn.Linear(channel, channel // reduction, bias=False),
            nn.ReLU(inplace=True),
            nn.Linear(channel // reduction, channel, bias=False),
            nn.Sigmoid()
        )

    def forward(self, x):
        b, c, _, _ = x.size()
        y = self.avg_pool(x).view(b, c) # view可改变Tensor的形状，但数据不变
        y = self.fc(y).view(b, c, 1, 1)
        return x * y.expand_as(x) # 通过expand_as将y拓展成和x想同的shape

通道+空间注意力机制CBAM

CBAM（Convolutional Block Attention Module，CBAM）发表于ECCV2018，其同时集成了通道注意力机制和空间注意力机制，具体结构如下图所示：

通道注意力模块

首先采用两种池化方法对feature map进行汇聚，

然后将生成的两种通道注意力向量输入共享的MLP（Multi-Layer Perception）网络，生成两个维数为C×1×1的注意力向量

最后将两种向量相加，载通道sigmoid函数，得到最终的通道注意力向量（C×1×1）

class ChannelAttention(nn.Module):
    def __init__(self, in_planes, rotio=16):
        super(ChannelAttention, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.max_pool = nn.AdaptiveMaxPool2d(1)

        self.sharedMLP = nn.Sequential(
            nn.Conv2d(in_planes, in_planes // ratio, 1, bias=False), nn.ReLU(),
            nn.Conv2d(in_planes // rotio, in_planes, 1, bias=False))
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        avgout = self.sharedMLP(self.avg_pool(x))
        maxout = self.sharedMLP(self.max_pool(x))
        return self.sigmoid(avgout + maxout)

空间注意力模块

首先同时采用两种池化方法，在通道维度对feature map进行汇聚，得到两个维度相同的feature map（1×H×W）

然后把得到的两个feature map拼接在一起，得到一个特殊的feature map

最后利用一个7*7的卷积层对feature map进行卷积，再通过sigmoid函数，得到最终的feature map

class SpatialAttention(nn.Module):
    def __init__(self, kernel_size=7):
        super(SpatialAttention, self).__init__()
        assert kernel_size in (3,7), "kernel size must be 3 or 7"   # 判断kernel的尺寸，不为3,7时输出语句
        padding = 3 if kernel_size == 7 else 1

        self.conv = nn.Conv2d(2,1,kernel_size, padding=padding, bias=False)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        avgout = torch.mean(x, dim=1, keepdim=True) # z在通道维度上进行mean和max操作
        maxout, _ = torch.max(x, dim=1, keepdim=True)
        x = torch.cat([avgout, maxout], dim=1)  # 按通道维度拼接在一起
        x = self.conv(x) # 卷积
        return self.sigmoid(x) # 激活

通道 + 空间注意力

class BasicBlock(nn.Module):
    expansion = 1
    def __init__(self, inplanes, planes, stride=1, downsample=None):
        super(BasicBlock, self).__init__()
        self.conv1 = conv3x3(inplanes, planes, stride)  # 自己定义的函数
        self.bn1 = nn.BatchNorm2d(planes)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = conv3x3(planes, planes)
        self.bn2 = nn.BatchNorm2d(planes)
        self.ca = ChannelAttention(planes)
        self.sa = SpatialAttention()
        self.downsample = downsample
        self.stride = stride
        
    def forward(self, x):
        residual = x
        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)
        out = self.conv2(out)
        out = self.bn2(out)
        out = self.ca(out) * out  # 广播机制
        out = self.sa(out) * out  # 广播机制
        if self.downsample is not None:
            residual = self.downsample(x)
        out += residual
        out = self.relu(out)
        return out