Activations#

Collection of Ivy neural network activations as stateful classes.

class ivy.stateful.activations.ELU(*args, **kwargs)[source]#

Bases: Module

__init__(alpha=1.0)[source]#

Apply the ELU activation function.

class ivy.stateful.activations.GEGLU(*args, **kwargs)[source]#

Bases: Module

__init__()[source]#

Apply the GEGLU activation function.

class ivy.stateful.activations.GELU(*, approximate=False, complex_mode='jax')[source]#

Bases: Module

__init__(*, approximate=False, complex_mode='jax')[source]#

Apply the GELU activation function.

Parameters:

  • approximate (bool, default: False) – whether to use the gelu approximation algorithm or exact formulation.

  • complex_mode (Literal['split', 'magnitude', 'jax'], default: 'jax') – Specifies how to handle complex input. See ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.Hardswish(complex_mode='jax')[source]#

Bases: Module

__init__(complex_mode='jax')[source]#

Apply the HARDSWISH activation function.

Parameters:

complex_mode (Literal['split', 'magnitude', 'jax'], default: 'jax') –

Specifies how to handle complex input. See

ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.LeakyReLU(alpha=0.2, complex_mode='jax')[source]#

Bases: Module

__init__(alpha=0.2, complex_mode='jax')[source]#

Apply the LEAKY RELU activation function.

Parameters:

  • alpha (float, default: 0.2) – Negative slope for ReLU.

  • complex_mode (Literal['split', 'magnitude', 'jax'], default: 'jax') – Specifies how to handle complex input. See ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.LogSigmoid(complex_mode='jax')[source]#

Bases: Module

__init__(complex_mode='jax')[source]#

Apply the LogSigmoid activation function.

Parameter#

complex_mode

Specifies how to handle complex input. See ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.LogSoftmax(axis=-1, complex_mode='jax')[source]#

Bases: Module

__init__(axis=-1, complex_mode='jax')[source]#

Apply the LOG SOFTMAX activation function.

Parameters:

  • axis (Optional[int], default: -1) – The dimension log_softmax would be performed on. The default is None

  • complex_mode (Literal['split', 'magnitude', 'jax'], default: 'jax') – optional specifier for how to handle complex data types. See ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.Logit(*args, **kwargs)[source]#

Bases: Module

__init__(eps=None, complex_mode='jax')[source]#

Apply the LOGIT activation function.

Parameters:

  • eps (default: None) – The epsilon value for the logit formation. Default: None.

  • complex_mode (default: 'jax') – optional specifier for how to handle complex data types. See ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.Mish(*args, **kwargs)[source]#

Bases: Module

__init__()[source]#

Apply the MISH activation function.

class ivy.stateful.activations.PReLU(*args, **kwargs)[source]#

Bases: Module

__init__(slope)[source]#

Apply the PRELU activation function.

class ivy.stateful.activations.ReLU(complex_mode='jax')[source]#

Bases: Module

__init__(complex_mode='jax')[source]#

Apply the RELU activation function.

Parameters:

complex_mode (Literal['split', 'magnitude', 'jax'], default: 'jax') –

Specifies how to handle complex input. See

ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.ReLU6(complex_mode='jax')[source]#

Bases: Module

__init__(complex_mode='jax')[source]#

Apply the TANH activation function.

Parameters:

complex_mode (Literal['split', 'magnitude', 'jax'], default: 'jax') –

Specifies how to handle complex input. See

ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.SeLU(*args, **kwargs)[source]#

Bases: Module

__init__()[source]#

Apply the SELU activation function.

class ivy.stateful.activations.SiLU(*args, **kwargs)[source]#

Bases: Module

__init__()[source]#

Apply the SiLU activation function.

class ivy.stateful.activations.Sigmoid(complex_mode='jax')[source]#

Bases: Module

__init__(complex_mode='jax')[source]#

Apply the SIGMOID activation function.

Parameter#

complex_mode

Specifies how to handle complex input. See ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.Softmax(axis=-1, complex_mode='jax')[source]#

Bases: Module

__init__(axis=-1, complex_mode='jax')[source]#

Apply the SOFTMAX activation function.

Parameters:

  • axis (int, default: -1) – The axis which we apply softmax op on.

  • complex_mode (Literal['split', 'magnitude', 'jax'], default: 'jax') – Specifies how to handle complex input. See ivy.func_wrapper.handle_complex_input for more detail.

class ivy.stateful.activations.Softplus(*args, **kwargs)[source]#

Bases: Module

__init__(beta=1.0, threshold=None)[source]#

Apply the SOFTPLUS activation function.

class ivy.stateful.activations.Tanh(complex_mode='jax')[source]#

Bases: Module

__init__(complex_mode='jax')[source]#

Apply the TANH activation function.

Parameters:

complex_mode (Literal['split', 'magnitude', 'jax'], default: 'jax') –

Specifies how to handle complex input. See

ivy.func_wrapper.handle_complex_input for more detail.

This should have hopefully given you an overview of the activations submodule, if you have any questions, please feel free to reach out on our discord in the activations channel!