Helpers#

class ivy_models.transformers.helpers.FeedForward(*args, **kwargs)[source]#

Bases: Module

__init__(dim, dropout=0.0, device=None, v=None)[source]#

Initialize Ivy layer, which is a stateful object consisting of trainable variables.

Parameters:
  • args – Positional arguments to the _build method.

  • v (default: None) – Ivy container of trainable variables. Created internally by default.

  • buffers – Ivy container of buffers/non-trainable arrays in the state_dict.

  • build_mode – How the Module is built, either on initialization (now), explicitly by the user by calling build(), or the first time the __call__ method is run. Default is on initialization.

  • store_vars – Whether or not to store the variables created. Default is True.

  • with_partial_v – Whether to allow partial specification of variables. Default is False.

  • dynamic_backend – When the value is true, allow conversion of arrays from a different backend to the current backend if v passed in the input contains arrays created with different backend.

  • training – specifies whether the module is in training or evaluation mode. Default is True.

  • dtype – Data type to be used for creating model variables. (Default value = None).

  • device (default: None) – Device on which to create the module’s variables ‘cuda:0’, ‘cuda:1’, ‘cpu’ etc. (Default value = None).

  • kwargs – Keyword arguments to the _build method.

class ivy_models.transformers.helpers.PreNorm(*args, **kwargs)[source]#

Bases: Module

__init__(dim, fn, key_dim=None, value_dim=None, eps=1e-05, device=None, v=None)[source]#

Initialize Ivy layer, which is a stateful object consisting of trainable variables.

Parameters:
  • args – Positional arguments to the _build method.

  • v (default: None) – Ivy container of trainable variables. Created internally by default.

  • buffers – Ivy container of buffers/non-trainable arrays in the state_dict.

  • build_mode – How the Module is built, either on initialization (now), explicitly by the user by calling build(), or the first time the __call__ method is run. Default is on initialization.

  • store_vars – Whether or not to store the variables created. Default is True.

  • with_partial_v – Whether to allow partial specification of variables. Default is False.

  • dynamic_backend – When the value is true, allow conversion of arrays from a different backend to the current backend if v passed in the input contains arrays created with different backend.

  • training – specifies whether the module is in training or evaluation mode. Default is True.

  • dtype – Data type to be used for creating model variables. (Default value = None).

  • device (default: None) – Device on which to create the module’s variables ‘cuda:0’, ‘cuda:1’, ‘cpu’ etc. (Default value = None).

  • kwargs – Keyword arguments to the _build method.