鹿晗新专辑《Imagination》开启预售 4.19正式发行

paddle.nn.functional. rrelu ( x: Tensor, lower: float = 0.125, upper: float = 0.3333333333333333, training: bool = True, name: str | None = None ) → Tensor [source]
百度 　　获知这一信息的几位传媒人表示，这个《暂行规定》的出台，展示了广西高层的胸怀与视野，也使人民网甚至所有传媒人都更加感受到所担负的责任。

rrelu activation.

Applies the randomized leaky rectified liner unit function to improve generalization performance, as described in the paper: Empirical Evaluation of Rectified Activations in Convolutional Network

During training, randomly samples the negative slope for activation values as described below:

\[\begin{split}rrelu(x)= \left\{ \begin{array}{rcl} x, & & if \ x >= 0 \\ a * x, & & otherwise \\ \end{array} \right.\end{split}\]

where \(x\) is the input tensor, \(a\) is randomly sampled from uniform distribution in range (\(lower\), \(upper\)),

In the test phase, the negative slope will take the average value of \(lower\) and \(upper\):

\[\begin{split}rrelu(x)= \left\{ \begin{array}{rcl} x, & & if \ x >= 0 \\ (lower + upper) * 0.5 * x, & & otherwise \\ \end{array} \right.\end{split}\]

where \(x\) is the input tensor, \(lower\) and \(upper\) are the bounds of uniform distribution.

Parameters

  • x (Tensor) – The input Tensor with data type float16, float32, float64.

  • lower (float, optional) – The lower bound of uniform distribution. Default: 0.125.

  • upper (float, optional) – The upper bound of uniform distribution. Default: 0.3333333333333333.

  • training (bool, optional) – Current mode is in training or others. Default is True.

  • name (str|None, optional) – For details, please refer to Name. Generally, no setting is required. Default: None.

Returns

A Tensor with the same data type and shape as x .

Examples

>>> import paddle
>>> import paddle.nn.functional as F
>>> paddle.seed(1)
>>> input_tensor = paddle.to_tensor([[[[-2.0,  3.0, -4.0,  5.0],
...                                    [ 3.0, -4.0,  5.0, -6.0],
...                                    [-7.0, -8.0,  8.0,  9.0]],
...                                   [[ 1.0, -2.0, -3.0,  4.0],
...                                    [-5.0,  6.0,  7.0, -8.0],
...                                    [ 6.0,  7.0,  8.0,  9.0]]]], dtype='float32')
>>> out = F.rrelu(input_tensor, 0.1, 0.3)
>>> print(out)
Tensor(shape=[1, 2, 3, 4], dtype=float32, place=Place(cpu), stop_gradient=True,
[[[[-0.20715050,  3.        , -1.01193857,  5.        ],
   [ 3.        , -0.94084597,  5.        , -0.65544695],
   [-1.24268556, -2.34339547,  8.        ,  9.        ]],
  [[ 1.        , -0.44942653, -0.68969047,  4.        ],
   [-1.03736508,  6.        ,  7.        , -0.95799232],
   [ 6.        ,  7.        ,  8.        ,  9.        ]]]])