Ctrl+K

Norms#

Collection of Ivy normalization classes.

class ivy.stateful.norms.BatchNorm2D(num_features, /, *, eps=1e-05, momentum=0.1, data_format='NSC', affine=True, track_running_stats=True, device=None, v=None, dtype=None, training=True)[source]#

Bases: Module

__init__(num_features, /, *, eps=1e-05, momentum=0.1, data_format='NSC', affine=True, track_running_stats=True, device=None, v=None, dtype=None, training=True)[source]#

Class for applying Layer Normalization over a mini-batch of inputs.

Parameters:

  • num_features – Trailing shape to applying the normalization to.

  • epsilon – small constant to add to the denominator, use global ivy.min_base by default.

  • data_format (str, default: 'NSC') – The ordering of the dimensions in the input, one of “NSC” or “NCS”, where N is the batch dimension, S represents any number of spatial dimensions and C is the channel dimension. Default is “NSC”.

  • affine (bool, default: True) – Whether to include learnable affine parameters, default is True.

  • track_running_stats (bool, default: True) – is a boolean flag that determines whether the running statistics should be updated during training in batch normalization.

  • momentum (float, default: 0.1) –

    The value used for the running_mean and running_var computation.

    Default is 0.1.

  • device (default: None) – device on which to create the layer’s variables ‘cuda:0’, ‘cuda:1’, ‘cpu’ etc. (Default value = None)

  • v (default: None) – the variables for each submodule in the sequence, constructed internally by default.

  • training (default: True) – If true, calculate and use the mean and variance of x. Otherwise, use the internal mean and variance when affine is True.

class ivy.stateful.norms.LayerNorm(normalized_shape, /, *, eps=1e-05, elementwise_affine=True, new_std=1.0, device=None, v=None, dtype=None)[source]#

Bases: Module

__init__(normalized_shape, /, *, eps=1e-05, elementwise_affine=True, new_std=1.0, device=None, v=None, dtype=None)[source]#

Class for applying Layer Normalization over a mini-batch of inputs.

Parameters:

  • normalized_shape – Trailing shape to applying the normalization to.

  • epsilon – small constant to add to the denominator, use global ivy.min_base by default.

  • elementwise_affine (bool, default: True) – Whether to include learnable affine parameters, default is True.

  • new_std (float, default: 1.0) – The standard deviation of the new normalized values. Default is 1.

  • device (default: None) – device on which to create the layer’s variables ‘cuda:0’, ‘cuda:1’, ‘cpu’ etc. (Default value = None)

  • v (default: None) – the variables for each submodule in the sequence, constructed internally by default.

This should have hopefully given you an overview of the norms submodule, if you have any questions, please feel free to reach out on our discord in the norms channel!

On this page