Activations#

ivy.logit(x, /, *, eps=None, out=None)[source]#

Compute the logit of x.

logit(x) = log(x / (1 - x)).

Parameters:

  • x (Union[float, int, Array]) – Input data.

  • eps (Optional[float]) – When eps is None the function outpus NaN where x < 0 or x > 1. (default: None) and inf or -inf where x = 1 or x = 0, respectively. Otherwise if eps is defined, x is clamped to [eps, 1 - eps]

  • out (Optional[Array]) – Optional output array. (default: None)

Return type:

Array

Returns:

ret – Array containing elementwise logits of x.

Examples

>>> x = ivy.array([1, 0, 0.9])
>>> z = ivy.logit(x)
>>> print(z)
ivy.array([       inf,       -inf, 2.19722438])

>>> x = ivy.array([1, 2, -0.9])
>>> z = ivy.logit(x, eps=0.2)
>>> print(z)
ivy.array([ 1.38629448,  1.38629448, -1.38629436])
ivy.logsigmoid(input)[source]#

Apply element-wise Log-sigmoid of x.

logsigmoid(x) = log(1 / (1 + exp(-x)).

Parameters:

input (Union[NativeArray, Array]) – Input array.

Return type:

Array

Returns:

Array with same shape as input with Log-sigmoid applied to every element.

Examples

With ivy.Array input:

>>> x = ivy.array([-1., 0., 1.])
>>> z = x.logsigmoid()
>>> print(z)
ivy.array([-1.31326175, -0.69314718, -0.31326169])

>>> x = ivy.array([1.5, 0.7, -2.4])
>>> z = x.logsigmoid()
>>> print(z)
ivy.array([-0.20141329, -0.40318608, -2.48683619])

With ivy.Container input:

>>> x = ivy.Container(a=ivy.array([1.0, -1.2]), b=ivy.array([0.2, 0.6]))
>>> x = ivy.logsigmoid(x)
>>> print(x)
{
    a: ivy.array([-0.31326169, -1.46328247]),
    b: ivy.array([-0.59813893, -0.43748799])
}
ivy.prelu(x, slope, /, *, out=None)[source]#

Prelu takes input data (Array) and slope array as input,

and produces one output data (array) where the function f(x) = slope * x for x < 0, f(x) = x for x >= 0., is applied to the data array elementwise. This operator supports unidirectional broadcasting (array slope should be unidirectional broadcastable to input tensor X);

Parameters:

  • x (Union[NativeArray, Array]) – Input Array.

  • slope (Union[float, NativeArray, Array]) – Slope Array. The shape of slope can be smaller then first input X; if so, its shape must be unidirectional broadcastable to X.

  • out (Optional[Array]) – Optional output array. (default: None)

Return type:

Array

Returns:

ret – Array containing Parametrized relu values.

ivy.relu6(x, /, *, out=None)[source]#

Apply the rectified linear unit 6 function element-wise.

Parameters:

  • x (Union[Array, NativeArray]) – input array

  • out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that the (default: None) inputs broadcast to.

Return type:

Array

Returns:

ret – an array containing the rectified linear unit 6 activation of each element in x.

Examples

With ivy.Array input:

>>> x = ivy.array([-1.,  0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.])
>>> y = ivy.relu6(x)
>>> print(y)
ivy.array([0., 0., 1., 2., 3., 4., 5., 6., 6.])

>>> x = ivy.array([-1.,  0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.])
>>> y = ivy.zeros(9)
>>> ivy.relu6(x, out = y)
>>> print(y)
ivy.array([0., 0., 1., 2., 3., 4., 5., 6., 6.])

With ivy.Container input:

>>> x = {
            a: ivy.array([-3., -2., -1., 0., 1., 2., 3., 4., 5.]),
            b: ivy.array([1., 2., 3., 4., 5., 6., 7., 8., 9.])
        }
>>> x = ivy.relu6(x, out=x)
>>> print(x)
{
a: ivy.array([0., 0., 0., 0., 1., 2., 3., 4., 5.]),
b: ivy.array([1., 2., 3., 4., 5., 6., 6., 6., 6.])
}
ivy.selu(x, /, *, out=None)[source]#

Apply the scaled exponential linear unit function element-wise.

Parameters:

  • x (Union[Array, NativeArray]) – input array

  • out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that the (default: None) inputs broadcast to.

Return type:

Array

Returns:

ret – an array containing the scaled exponential linear unit activation of each element in x.

Examples

With ivy.Array input: >>> x = ivy.array([-1., 0., 1., 2., 3., 4., 5., 6., 7.]) >>> y = ivy.selu(x) >>> print(y) ivy.array([-1.11133075, 0., 1.05070102, 2.10140204, 3.15210295,

4.20280409, 5.25350523, 6.30420589, 7.35490704])

>>> x = ivy.array([-1.,  0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.])
>>> y = ivy.zeros(9)
>>> ivy.selu(x, out = y)
>>> print(y)
ivy.array([-1.11133075,  0.,  1.05070102,  2.10140204,  3.15210295,
            4.20280409,  5.25350523,  6.30420589,  7.35490704])

With ivy.Container input: >>> x = ivy.Container(

a=ivy.array([-3., -2., -1., 0., 1., 2., 3., 4., 5.]), b=ivy.array([1., 2., 3., 4., 5., 6., 7., 8., 9.])

)

>>> x = ivy.selu(x, out=x)
>>> print(x)
{
    a: ivy.array([-1.6705687, -1.52016652, -1.11133075, 0.,1.05070102,
                    2.10140204, 3.15210295, 4.20280409, 5.25350523]),
    b: ivy.array([1.05070102, 2.10140204, 3.15210295, 4.20280409, 5.25350523,
                    6.30420589, 7.35490704, 8.40560818, 9.45630932])
}
ivy.silu(x, /, *, out=None)[source]#

Apply the silu function element-wise.

Parameters:

  • x (Union[Array, NativeArray]) – input array.

  • out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that the (default: None) inputs broadcast to.

Return type:

Array

Returns:

ret – an array containing the silu activation of each element in x.

Examples

With ivy.Array input:

>>> x = ivy.array([-1.0, 1.0, 2.0])
>>> y = ivy.silu(x)
>>> print(y)
ivy.array([-0.2689,  0.7310,  1.7615])

>>> x = ivy.array([-1.0, 1.0, 2.0])
>>> y = x.silu()
>>> print(y)
ivy.array([-0.2689,  0.7310,  1.7615])

>>> x = ivy.array([[-1.3, 3.8, 2.1], [1.7, 4.2, -6.6]])
>>> y = ivy.silu(x)
>>> print(y)
ivy.array([[-0.2784,  3.7168,  1.8708], [ 1.4374,  4.1379, -0.0089]])
ivy.thresholded_relu(x, /, *, threshold=0, out=None)[source]#

Apply the rectified linear unit function with custom threshold.

Parameters:

  • x (Union[Array, NativeArray]) – input array

  • threshold (Union[int, float]) – threshold value above which the activation is linear. Default: 0. (default: 0)

  • out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that the (default: None) inputs broadcast to.

Return type:

Array

Returns:

ret – an array containing the rectified linear unit activation of each element in x. with custom threshold.

Examples

With ivy.Array input:

>>> x = ivy.array([-1., 0., 1.])
>>> y = ivy.thresholded_relu(x, threshold=0.5)
>>> print(y)
ivy.array([0.,  0. ,  1.])

>>> x = ivy.array([1.5, 0.7, -2.4])
>>> y = ivy.zeros(3)
>>> ivy.thresholded_relu(x, threshold=1, out = y)
>>> print(y)
ivy.array([ 1.5,  0., 0.])

With ivy.Container input:

>>> x = ivy.Container(a=ivy.array([1.0, -1.2]), b=ivy.array([0.2, 0.6]))
>>> x = ivy.thresholded_relu(x, threshold=0.5)
>>> print(x)
{
    a: ivy.array([1., 0.]),
    b: ivy.array([0., 0.6])
}