1.  official website https://pytorch.org/docs/master/generated/torch.nn.GroupNorm.html#groupnorm

       class torch.nn.GroupNorm(num_groups: int, num_channels: int, eps: float = 1e-05, affine: bool = True)

    params:  

• num_groups (int) – number of groups to separate the channels into

• num_channels (int) – number of channels expected in input

• eps – a value added to the denominator for numerical stability. Default: 1e-5

• affine – a boolean value that when set to True, this module has learnable per-channel affine parameters initialized to ones (for weights) and zeros (for biases). Default: True.

  =================

  Input_shape: (N, C, *)  where C=num_groups

  Output_shape: (N, C, *) same shape as the input_shape

  由以上可知，输入params只需输入num_groups与num_channels。

• 由原文知，num_groups对值不敏感

• num_channels 可由原来model获取  （具体如下）
  

(N, 

2. 将3D ResNet中的BN转为GN：1）找到BN layer；2）代替BN为GN

   def convert_BN2GN(model):  
   for name, module in model._modules.items():  #遍历layer
      if len(list(module.children())) > 0:  #对每个block内部的layers，调用原函数进行遍历
        self.convert_BN2GN(module)
      elif isinstance(module, torch.nn.modules.batchnorm.BatchNorm3d): #对每一个BN layer进行操作
        module_tmp = nn.GroupNorm(32, module.num_features)  
        #其中，32为num_groups; model.num_features为num_channels
        model._modules[name] = module_tmp
    return model

Ref： https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/104686

https://github.com/ppwwyyxx/GroupNorm-reproduce/blob/master/ImageNet-ResNet-PyTorch/resnet_gn.py

转载本文请联系原作者获取授权，同时请注明本文来自高琳琳科学网博客。

链接地址：https://wap.sciencenet.cn/blog-1969089-1242863.html?mobile=1

收藏

分享