# Activations#

Collection of Ivy activation functions.

ivy.gelu(x, /, *, approximate=False, complex_mode='jax', out=None)[source]#

Apply the Gaussian error linear unit (GELU) activation function.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – Input array.

• approximate (`bool`, default: `False`) – Whether to approximate, default is `True`. An approximation is always used if the input array is complex.

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – The input array with gelu applied element-wise.

Examples

With `ivy.Array` input:

```>>> x = ivy.array([-1.2, -0.6, 1.5])
>>> y = ivy.gelu(x)
>>> y
ivy.array([-0.138, -0.165, 1.4])
```

With `ivy.NativeArray` input:

```>>> x = ivy.native_array([-1.3, 3.8, 2.1])
>>> y = ivy.gelu(x)
>>> y
ivy.array([-0.126, 3.8, 2.06])
```

With `ivy.Container` input:

```>>> x = ivy.Container(a=ivy.array([1., 2.]), b=ivy.array([-0.9, -1.]))
>>> y = ivy.gelu(x)
>>> y
{
a: ivy.array([0.841, 1.95]),
b: ivy.array([-0.166, -0.159])
}
```
ivy.hardswish(x, /, *, complex_mode='jax', out=None)[source]#

Apply the hardswish activation function element-wise.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – input array

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – an array containing the hardswish activation of each element in `x`.

Examples

With `ivy.Array` input:

```>>> x = ivy.array([0., 0., 4.])
>>> y = ivy.hardswish(x)
>>> y
ivy.array([0., 0., 4.])
```

With `ivy.Container` input:

```>>> x = ivy.Container(a=ivy.array([-3., 4., 5.]), b=ivy.array([0., 5.]))
>>> x = ivy.hardswish(x, out=x)
>>> x
{
a: ivy.array([-0.,  4.,  5.]),
b: ivy.array([0., 5.])
}
```
ivy.leaky_relu(x, /, *, alpha=0.2, complex_mode='jax', out=None)[source]#

Apply the leaky rectified linear unit function element-wise.

If the input is complex, then by default each element is scaled by alpha if either its real part is strictly negative or if its real part is zero and its imaginary part is negative. This behaviour can be changed by specifying a different complex_mode.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – Input array.

• alpha (`float`, default: `0.2`) – Negative slope for ReLU.

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – The input array with leaky relu applied element-wise.

Examples

With `ivy.Array` input:

```>>> x = ivy.array([0.39, -0.85])
>>> y = ivy.leaky_relu(x)
>>> print(y)
ivy.array([ 0.39, -0.17])
```
```>>> x = ivy.array([1.5, 0.7, -2.4])
>>> y = ivy.zeros(3)
>>> ivy.leaky_relu(x, out=y)
>>> print(y)
ivy.array([ 1.5 ,  0.7 , -0.48])
```
```>>> x = ivy.array([[1.1, 2.2, 3.3],
...                [-4.4, -5.5, -6.6]])
>>> ivy.leaky_relu(x, out=x)
>>> print(x)
ivy.array([[ 1.1 ,  2.2 ,  3.3 ],
[-0.88, -1.1 , -1.32]])
```

With `ivy.Container` input:

```>>> x = ivy.Container(a=ivy.array([0.0, -1.2]), b=ivy.array([0.4, -0.2]))
>>> x = ivy.leaky_relu(x, out=x)
>>> print(x)
{
a: ivy.array([0., -0.24000001]),
b: ivy.array([0.40000001, -0.04])
}
```
ivy.log_softmax(x, /, *, axis=None, complex_mode='jax', out=None)[source]#

Apply the log_softmax function element-wise.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – Input array.

• axis (`Optional`[`int`], default: `None`) – The dimension log_softmax would be performed on. The default is `None`.

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – The output array with log_softmax applied element-wise to input.

Examples

With `ivy.Array` input:

```>>> x = ivy.array([-1.0, -0.98])
>>> y = ivy.log_softmax(x)
>>> print(y)
ivy.array([-0.703, -0.683])
```
```>>> x = ivy.array([1.0, 2.0, 3.0])
>>> y = ivy.log_softmax(x)
>>> print(y)
ivy.array([-2.41, -1.41, -0.408])
```

With `ivy.NativeArray` input:

```>>> x = ivy.native_array([1.5, 0.5, 1.0])
>>> y = ivy.log_softmax(x)
>>> print(y)
ivy.array([-0.68, -1.68, -1.18])
```

With `ivy.Container` input:

```>>> x = ivy.Container(a=ivy.array([1.5, 0.5, 1.0]))
>>> y = ivy.log_softmax(x)
>>> print(y)
{
a: ivy.array([-0.68, -1.68, -1.18])
}
```
```>>> x = ivy.Container(a=ivy.array([1.0, 2.0]), b=ivy.array([0.4, -0.2]))
>>> y = ivy.log_softmax(x)
>>> print(y)
{
a: ivy.array([-1.31, -0.313]),
b: ivy.array([-0.437, -1.04])
}
```
ivy.mish(x, /, *, complex_mode='jax', out=None)[source]#

Apply the mish activation function element-wise.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – input array

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – an array containing the mish activation of each element in `x`.

Examples

With `ivy.Array` input:

```>>> x = ivy.array([-1., 0., 1.])
>>> y = ivy.mish(x)
>>> print(y)
ivy.array([-0.30340147,  0.        ,  0.86509842])
```
```>>> x = ivy.array([1.5, 0.7, -2.4])
>>> y = ivy.zeros(3)
>>> ivy.mish(x, out = y)
>>> print(y)
ivy.array([ 1.40337825,  0.56114835, -0.20788449])
```

With `ivy.Container` input:

```>>> x = ivy.Container(a=ivy.array([1.0, -1.2]), b=ivy.array([0.4, -0.2]))
>>> x = ivy.mish(x)
>>> print(x)
{
a: ivy.array([0.86509842, -0.30883577]),
b: ivy.array([0.28903052, -0.10714479])
}
```
ivy.relu(x, /, *, complex_mode='jax', out=None)[source]#

Apply the rectified linear unit function element-wise.

If the input is complex, then by default each element is set to zero if either its real part is strictly negative or if its real part is zero and its imaginary part is negative. This behaviour can be changed by specifying a different complex_mode.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – input array

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – an array containing the rectified linear unit activation of each element in `x`.

Examples

With `ivy.Array` input:

```>>> x = ivy.array([-1., 0., 1.])
>>> y = ivy.relu(x)
>>> print(y)
ivy.array([0., 0., 1.])
```
```>>> x = ivy.array([1.5, 0.7, -2.4])
>>> y = ivy.zeros(3)
>>> ivy.relu(x, out = y)
>>> print(y)
ivy.array([1.5, 0.7, 0.])
```

With `ivy.Container` input:

```>>> x = ivy.Container(a=ivy.array([1.0, -1.2]), b=ivy.array([0.4, -0.2]))
>>> x = ivy.relu(x, out=x)
>>> print(x)
{
a: ivy.array([1., 0.]),
b: ivy.array([0.40000001, 0.])
}
```
ivy.sigmoid(x, /, *, complex_mode='jax', out=None)[source]#

Apply the sigmoid function element-wise.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – input array.

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the input broadcast to. default: None

Return type:

`Array`

Returns:

ret – an array containing the sigmoid activation of each element in `x`. sigmoid activation of x is defined as 1/(1+exp(-x)).

Examples

With `ivy.Array` input:

```>>> x = ivy.array([-1.0, 1.0, 2.0])
>>> y = ivy.sigmoid(x)
>>> print(y)
ivy.array([0.2689414 , 0.7310586 , 0.88079703])
```
```>>> x = ivy.array([-1.0, 1.0, 2.0])
>>> y = ivy.zeros(3)
>>> ivy.sigmoid(x, out=y)
>>> print(y)
ivy.array([0.2689414 , 0.7310586 , 0.88079703])
```

With `ivy.Container` input:

```>>> x = ivy.Container(a=ivy.array([0.]),
...                   b=ivy.Container(c=ivy.array([1.]),
...                                   d=ivy.array([2.])))
>>> y = ivy.sigmoid(x)
>>> print(y)
{
a: ivy.array([0.5]),
b: {
c: ivy.array([0.7310586]),
d: ivy.array([0.88079703])
}
}
```
```>>> x = ivy.Container(a=ivy.array([0.]),
...                   b=ivy.Container(c=ivy.array([1.]),
...                                   d=ivy.array([2.])))
>>> y = ivy.Container(a=ivy.array([0.]),
...                   b=ivy.Container(c=ivy.array([0.]),
...                                   d=ivy.array([0.])))
>>> ivy.sigmoid(x, out=y)
>>> print(y)
{
a: ivy.array([0.5]),
b: {
c: ivy.array([0.7310586]),
d: ivy.array([0.88079703])
}
}
```
ivy.softmax(x, /, *, axis=None, complex_mode='jax', out=None)[source]#

Apply the softmax function element-wise.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – Input array.

• axis (`Optional`[`int`], default: `None`) – The dimension softmax would be performed on. The default is `None`.

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – The input array with softmax applied element-wise.

Examples

With `ivy.Array` input:

```>>> x = ivy.array([1.0, 0, 1.0])
>>> y = ivy.softmax(x)
>>> print(y)
ivy.array([0.422, 0.155, 0.422])
```
```>>> x = ivy.array([[1.1, 2.2, 3.3],
...                [4.4, 5.5, 6.6]])
>>> y = ivy.softmax(x, axis = 1)
>>> print(y)
ivy.array([[0.0768, 0.231 , 0.693 ],
[0.0768, 0.231 , 0.693 ]])
```
ivy.softplus(x, /, *, beta=None, threshold=None, complex_mode='jax', out=None)[source]#

Apply the softplus function element-wise.

If the input is complex, then by default we apply the softplus operation log(1+ exp(x)) to each element If threshold is set we check if either its real part is strictly negative or if its real part is zero and its imaginary part is negative then we apply input×β > threshold.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – input array.

• beta (`Optional`[`Union`[`int`, `float`]], default: `None`) – The beta value for the softplus formation. Default: `None`.

• threshold (`Optional`[`Union`[`int`, `float`]], default: `None`) – values above this revert to a linear function If the input is complex, only its real part is considered. Default: `None`

• complex_mode (`Literal`[`'split'`, `'magnitude'`, `'jax'`], default: `'jax'`) – optional specifier for how to handle complex data types. See `ivy.func_wrapper.handle_complex_input` for more detail.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – an array containing the softplus activation of each element in `x`.

Examples

With `ivy.Array` input:

```>>> x = ivy.array([-0.3461, -0.6491])
>>> y = ivy.softplus(x)
>>> print(y)
ivy.array([0.535,0.42])
```
```>>> x = ivy.array([-0.3461, -0.6491])
>>> y = ivy.softplus(x, beta=0.5)
>>> print(y)
ivy.array([1.22, 1.09])
```
```>>> x = ivy.array([1., 2., 3.])
>>> y = ivy.softplus(x, threshold=2)
>>> print(y)
ivy.array([1.31, 2.13, 3.  ])
```
ivy.softsign(x, /, out=None)[source]#

Apply the softsign function element-wise.

Parameters:
• x (`Union`[`Array`, `NativeArray`]) – Input array.

• out (`Optional`[`Array`], default: `None`) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

`Array`

Returns:

ret – The input array with softsign applied element-wise.

Examples

With `ivy.Array` input: >>> x = ivy.array([1.0, 2.0, 3.0]) >>> y = ivy.softsign(x) >>> print(y) ivy.array([0.5, 0.66666667, 0.75])

This should have hopefully given you an overview of the activations submodule, if you have any questions, please feel free to reach out on our discord in the activations channel!