Activations
Collection of Ivy neural network activations as stateful classes.
-
class ivy.stateful.activations.GEGLU[source]
Bases: Module
-
__init__()[source]
Apply the GEGLU activation function.
-
class ivy.stateful.activations.GELU(*, approximate=False)[source]
Bases: Module
-
__init__(*, approximate=False)[source]
Apply the GELU activation function.
-
class ivy.stateful.activations.LeakyReLU(alpha=0.2)[source]
Bases: Module
-
__init__(alpha=0.2)[source]
Apply the LEAKY RELU activation function.
- Parameters:
alpha (float
) – Negative slope for ReLU. (default: 0.2
)
-
class ivy.stateful.activations.LogSoftmax[source]
Bases: Module
-
__init__()[source]
Apply the LOG SOFTMAX activation function.
-
class ivy.stateful.activations.Mish[source]
Bases: Module
-
__init__()[source]
Apply the MISH activation function.
-
class ivy.stateful.activations.ReLU[source]
Bases: Module
-
__init__()[source]
Apply the RELU activation function.
-
class ivy.stateful.activations.SiLU[source]
Bases: Module
-
__init__()[source]
Apply the SiLU activation function.
-
class ivy.stateful.activations.Softmax[source]
Bases: Module
-
__init__()[source]
Apply the SOFTMAX activation function.
-
class ivy.stateful.activations.Softplus[source]
Bases: Module
-
__init__()[source]
Apply the SOFTPLUS activation function.
This should have hopefully given you an overview of the activations submodule, if you have any questions, please feel free to reach out on our discord in the activations channel or in the activations forum!