Activations#
Collection of Ivy activation functions.
- ivy.deserialize(name, /, *, custom_objects=None)[source]#
Return activation function given a string identifier.
- Parameters:
name (
Optional
[str
]) – The name of the activation function.custom_objects (
Optional
[Dict
]) – Optional dictionary listing user-provided activation functions. (default:None
)
- Return type:
Optional
[Callable
]- Returns:
ret – Corresponding activation function.
Examples
With
str
input:>>> name = "sigmoid" >>> sigmoid = ivy.deserialize(name) >>> print(sigmoid) <function sigmoid at XXXXXXXXXXXXXX>
With
str
anddict
input:>>> name = "custom_fn" >>> objects = {"custom_fn": lambda x: x} >>> custom_fn = ivy.deserialize(name, custom_objects=objects) >>> print(custom_fn) <function custom_fn at XXXXXXXXXXXXXX>
- ivy.gelu(x, /, *, approximate=False, out=None)[source]#
Apply the Gaussian error linear unit (GELU) activation function.
- Parameters:
- Return type:
- Returns:
ret – The input array with gelu applied element-wise.
Examples
With
ivy.Array
input:>>> x = ivy.array([-1.2, -0.6, 1.5]) >>> y = ivy.gelu(x) >>> y ivy.array([-0.138, -0.165, 1.4])
With
ivy.NativeArray
input:>>> x = ivy.native_array([-1.3, 3.8, 2.1]) >>> y = ivy.gelu(x) >>> y ivy.array([-0.126, 3.8, 2.06])
With
ivy.Container
input:>>> x = ivy.Container(a=ivy.array([1., 2.]), b=ivy.array([-0.9, -1.])) >>> y = ivy.gelu(x) >>> y { a: ivy.array([0.841, 1.95]), b: ivy.array([-0.166, -0.159]) }
- ivy.get(name, /, *, custom_objects=None)[source]#
Return activation function given a string identifier.
- Parameters:
name (
Optional
[str
]) – The name of the activation function.custom_objects (
Optional
[Dict
]) – Optional dictionary listing user-provided activation functions. (default:None
)
- Return type:
Optional
[Callable
]- Returns:
ret – Corresponding activation function.
Examples
With
str
input:>>> name = "sigmoid" >>> sigmoid = ivy.get(name) >>> print(sigmoid) <function sigmoid at XXXXXXXXXXXXXX>
>>> name = None >>> linear = ivy.get(name) >>> print(linear) <function linear at XXXXXXXXXXXXXX>
With
str
anddict
input:>>> name = "custom_fn" >>> objects = {"custom_fn": lambda x: x} >>> custom_fn = ivy.get(name, custom_objects=objects) >>> print(custom_fn) <function custom_fn at XXXXXXXXXXXXXX>
- ivy.leaky_relu(x, /, *, alpha=0.2, out=None)[source]#
Apply the leaky rectified linear unit function element-wise.
- Parameters:
- Return type:
- Returns:
ret – The input array with leaky relu applied element-wise.
Examples
With
ivy.Array
input:>>> x = ivy.array([0.39, -0.85]) >>> y = ivy.leaky_relu(x) >>> print(y) ivy.array([ 0.39, -0.17])
>>> x = ivy.array([1.5, 0.7, -2.4]) >>> y = ivy.zeros(3) >>> ivy.leaky_relu(x, out=y) >>> print(y) ivy.array([ 1.5 , 0.7 , -0.48])
>>> x = ivy.array([[1.1, 2.2, 3.3], ... [-4.4, -5.5, -6.6]]) >>> ivy.leaky_relu(x, out=x) >>> print(x) ivy.array([[ 1.1 , 2.2 , 3.3 ], [-0.88, -1.1 , -1.32]])
With
ivy.Container
input:>>> x = ivy.Container(a=ivy.array([0.0, -1.2]), b=ivy.array([0.4, -0.2])) >>> x = ivy.leaky_relu(x, out=x) >>> print(x) { a: ivy.array([0., -0.24000001]), b: ivy.array([0.40000001, -0.04]) }
- ivy.log_softmax(x, /, *, axis=None, out=None)[source]#
Apply the log_softmax function element-wise.
- Parameters:
- Return type:
- Returns:
ret – The output array with log_softmax applied element-wise to input.
Examples
With
ivy.Array
input:>>> x = ivy.array([-1.0, -0.98]) >>> y = ivy.log_softmax(x) >>> print(y) ivy.array([-0.703, -0.683])
>>> x = ivy.array([1.0, 2.0, 3.0]) >>> y = ivy.log_softmax(x) >>> print(y) ivy.array([-2.41, -1.41, -0.408])
With
ivy.NativeArray
input:>>> x = ivy.native_array([1.5, 0.5, 1.0]) >>> y = ivy.log_softmax(x) >>> print(y) ivy.array([-0.68, -1.68, -1.18])
With
ivy.Container
input:>>> x = ivy.Container(a=ivy.array([1.5, 0.5, 1.0])) >>> y = ivy.log_softmax(x) >>> print(y) { a: ivy.array([-0.68, -1.68, -1.18]) }
>>> x = ivy.Container(a=ivy.array([1.0, 2.0]), b=ivy.array([0.4, -0.2])) >>> y = ivy.log_softmax(x) >>> print(y) { a: ivy.array([-1.31, -0.313]), b: ivy.array([-0.437, -1.04]) }
- ivy.mish(x, /, *, out=None)[source]#
Apply the mish activation function element-wise.
- Parameters:
- Return type:
- Returns:
ret – an array containing the mish activation of each element in
x
.
Examples
With
ivy.Array
input:>>> x = ivy.array([-1., 0., 1.]) >>> y = ivy.mish(x) >>> print(y) ivy.array([-0.30340147, 0. , 0.86509842])
>>> x = ivy.array([1.5, 0.7, -2.4]) >>> y = ivy.zeros(3) >>> ivy.mish(x, out = y) >>> print(y) ivy.array([ 1.40337825, 0.56114835, -0.20788449])
With
ivy.Container
input:>>> x = ivy.Container(a=ivy.array([1.0, -1.2]), b=ivy.array([0.4, -0.2])) >>> x = ivy.mish(x) >>> print(x) { a: ivy.array([0.86509842, -0.30883577]), b: ivy.array([0.28903052, -0.10714479]) }
- ivy.relu(x, /, *, out=None)[source]#
Apply the rectified linear unit function element-wise.
- Parameters:
- Return type:
- Returns:
ret – an array containing the rectified linear unit activation of each element in
x
.
Examples
With
ivy.Array
input:>>> x = ivy.array([-1., 0., 1.]) >>> y = ivy.relu(x) >>> print(y) ivy.array([0., 0., 1.])
>>> x = ivy.array([1.5, 0.7, -2.4]) >>> y = ivy.zeros(3) >>> ivy.relu(x, out = y) >>> print(y) ivy.array([1.5, 0.7, 0.])
With
ivy.Container
input:>>> x = ivy.Container(a=ivy.array([1.0, -1.2]), b=ivy.array([0.4, -0.2])) >>> x = ivy.relu(x, out=x) >>> print(x) { a: ivy.array([1., 0.]), b: ivy.array([0.40000001, 0.]) }
- ivy.sigmoid(x, /, *, out=None)[source]#
Apply the sigmoid function element-wise.
- Parameters:
- Return type:
- Returns:
ret – an array containing the sigmoid activation of each element in
x
.
Examples
With
ivy.Array
input:>>> x = ivy.array([-1.0, 1.0, 2.0]) >>> y = ivy.sigmoid(x) >>> print(y) ivy.array([0.269, 0.731, 0.881])
>>> x = ivy.array([-1.0, 1.0, 2.0]) >>> y = x.sigmoid() >>> print(y) ivy.array([0.269, 0.731, 0.881])
>>> x = ivy.array([[-1.3, 3.8, 2.1], [1.7, 4.2, -6.6]]) >>> y = ivy.sigmoid(x) >>> print(y) ivy.array([[0.214, 0.978, 0.891], [0.846,0.985,0.001]] )
- ivy.softmax(x, /, *, axis=None, out=None)[source]#
Apply the softmax function element-wise.
- Parameters:
- Return type:
- Returns:
ret – The input array with softmax applied element-wise.
Examples
With
ivy.Array
input:>>> x = ivy.array([1.0, 0, 1.0]) >>> y = ivy.softmax(x) >>> print(y) ivy.array([0.422, 0.155, 0.422])
>>> x = ivy.array([[1.1, 2.2, 3.3], ... [4.4, 5.5, 6.6]]) >>> y = ivy.softmax(x, axis = 1) >>> print(y) ivy.array([[0.0768, 0.231 , 0.693 ], [0.0768, 0.231 , 0.693 ]])
- ivy.softplus(x, /, *, beta=None, threshold=None, out=None)[source]#
Apply the softplus function element-wise.
- Parameters:
x (
Union
[Array
,NativeArray
]) – input array.beta (
Optional
[Union
[int
,float
]]) – The beta value for the softplus formation. Default:None
. (default:None
)threshold (
Optional
[Union
[int
,float
]]) – values above this revert to a linear function. Default:None
. (default:None
)out (
Optional
[Array
]) – optional output array, for writing the result to. It must have a shape that the (default:None
) inputs broadcast to.
- Return type:
- Returns:
ret – an array containing the softplus activation of each element in
x
.
Functional Examples
With
ivy.Array
input:>>> x = ivy.array([-0.3461, -0.6491]) >>> y = ivy.softplus(x) >>> print(y) ivy.array([0.535,0.42])
>>> x = ivy.array([-0.3461, -0.6491]) >>> y = ivy.softplus(x, beta=0.5) >>> print(y) ivy.array([1.22, 1.09])
>>> x = ivy.array([1., 2., 3.]) >>> y = ivy.softplus(x, threshold=2) >>> print(y) ivy.array([1.31, 2.13, 3. ])
This should have hopefully given you an overview of the activations submodule, if you have any questions, please feel free to reach out on our discord in the activations channel or in the activations forum!