Helpers#
- class ivy_models.transformers.helpers.FeedForward(*args, **kwargs)[source]#
Bases:
Module
- __init__(dim, dropout=0.0, device=None, v=None)[source]#
Initialize Ivy layer, which is a stateful object consisting of trainable variables.
- Parameters:
args – Positional arguments to the _build method.
v (default:
None
) – Ivy container of trainable variables. Created internally by default.buffers – Ivy container of buffers/non-trainable arrays in the state_dict.
build_mode – How the Module is built, either on initialization (now), explicitly by the user by calling build(), or the first time the __call__ method is run. Default is on initialization.
store_vars – Whether or not to store the variables created. Default is
True
.with_partial_v – Whether to allow partial specification of variables. Default is
False
.dynamic_backend – When the value is true, allow conversion of arrays from a different backend to the current backend if v passed in the input contains arrays created with different backend.
training – specifies whether the module is in training or evaluation mode. Default is
True
.dtype – Data type to be used for creating model variables. (Default value = None).
device (default:
None
) – Device on which to create the module’s variables ‘cuda:0’, ‘cuda:1’, ‘cpu’ etc. (Default value = None).kwargs – Keyword arguments to the _build method.
- class ivy_models.transformers.helpers.PreNorm(*args, **kwargs)[source]#
Bases:
Module
- __init__(dim, fn, key_dim=None, value_dim=None, eps=1e-05, device=None, v=None)[source]#
Initialize Ivy layer, which is a stateful object consisting of trainable variables.
- Parameters:
args – Positional arguments to the _build method.
v (default:
None
) – Ivy container of trainable variables. Created internally by default.buffers – Ivy container of buffers/non-trainable arrays in the state_dict.
build_mode – How the Module is built, either on initialization (now), explicitly by the user by calling build(), or the first time the __call__ method is run. Default is on initialization.
store_vars – Whether or not to store the variables created. Default is
True
.with_partial_v – Whether to allow partial specification of variables. Default is
False
.dynamic_backend – When the value is true, allow conversion of arrays from a different backend to the current backend if v passed in the input contains arrays created with different backend.
training – specifies whether the module is in training or evaluation mode. Default is
True
.dtype – Data type to be used for creating model variables. (Default value = None).
device (default:
None
) – Device on which to create the module’s variables ‘cuda:0’, ‘cuda:1’, ‘cpu’ etc. (Default value = None).kwargs – Keyword arguments to the _build method.