torch.nn.Hardswish

原型

CLASS torch.nn.Hardswish(inplace=False)

参数

• inplace (bool) – 内部运算，默认为 False

定义

$$\text{Hardswish}(x)=\{\begin{array}{ll}0& \text{if }x\le -3,\\ x& \text{if }x\ge +3,\\ x\cdot (x+3)/6& \text{otherwise}\end{array}$$

图

代码

import torch
import torch.nn as nn

m = nn.Hardswish()
input = torch.randn(4)
output = m(input)
print("input: ", input)   # input:  tensor([-0.5567, -0.4911,  0.2918,  2.1492]) 
print("output: ", output) # output:  tensor([-0.2267, -0.2054,  0.1601,  1.8445])

【参考】

Hardswish — PyTorch 1.13 documentation