Layers#

class ivy.data_classes.container.layers._ContainerWithLayers(dict_in=None, queues=None, queue_load_sizes=None, container_combine_method='list_join', queue_timeout=None, print_limit=10, key_length_limit=None, print_indent=4, print_line_spacing=0, ivyh=None, default_key_color='green', keyword_color_dict=None, rebuild_child_containers=False, types_to_iteratively_nest=None, alphabetical_keys=True, dynamic_backend=None, build_callable=False, **kwargs)[source]#

Bases: ContainerBase

_abc_impl = <_abc._abc_data object>#

static _static_conv1d(x, filters, strides, padding, /, *, data_format='NWC', filter_format='channel_last', x_dilations=1, dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container static method variant of ivy.conv1d. This method simply wraps the function, and so the docstring for ivy.conv1d also applies to this method with minimal changes.

Parameters:

x (Union[Array, NativeArray, Container]) – Input image [batch_size,w, d_in].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fw,d_in, d_out]. (d_in must be the same as d from x)
strides (Union[int, Tuple[int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str, default: 'NWC') – “NWC” or “NCW”. Defaults to “NWC”.
filter_format (str, default: 'channel_last') – Either “channel_first” or “channel_last”. Defaults to “channel_last”.
x_dilations (Union[int, Tuple[int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
dilations (Union[int, Tuple[int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
key_chains (Optional[Union[List[str], Dict[str, str]]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (bool, default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (bool, default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (bool, default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
bias (Optional[Container], default: None) –
[d_out]. (Bias array of shape) –
out (Optional[Container], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a=ivy.array([[[2., 3., 4.], [5., 6., 7.]]]),
...                   b=ivy.array([[[7., 8., 9.], [10., 11., 12]]]))
>>> filters = ivy.array([[[0., 0.5, 1.], [0.25, 0.5, 0.75], [-0.5, 0., 0.5 ]]])
>>> result= ivy.Container.static_conv1d(x,filters,(1,),'VALID')
>>> print(result)
{
    ... a: ivy.array([[[-1.25, 2.5, 6.25],
    ...                [-2., 5.5, 13.]]]),
    ... b: ivy.array([[[-2.5, 7.5, 17.5],
    ...                [-3.25, 10.5, 24.2]]])
}

static _static_conv1d_transpose(x, filters, strides, padding, /, *, output_shape=None, data_format='NWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container static method variant of ivy.conv1d_transpose. This method simply wraps the function, and so the docstring for ivy.conv1d_transpose also applies to this method with minimal changes.

Parameters:

x (Union[Array, NativeArray, Container]) – Input image [batch_size,w,d_in] or [batch_size,d_in,w].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fw,d_in,d_out].
strides (Union[int, Tuple[int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – either the string ‘SAME’ (padding with zeros evenly), the string ‘VALID’ (no padding), or a sequence of n (low, high) integer pairs that give the padding to apply before and after each spatial dimension.
output_shape (Optional[Union[Array, NativeArray, Container]], default: None) – Shape of the output (Default value = None)
data_format (str, default: 'NWC') – The ordering of the dimensions in the input, one of “NWC” or “NCW”. “NWC” corresponds to input with shape (batch_size, width, channels), while “NCW” corresponds to input with shape (batch_size, channels, width).
dilations (Union[int, Tuple[int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
key_chains (Optional[Union[List[str], Dict[str, str]]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (bool, default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (bool, default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (bool, default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Container], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the transpose convolution operation.

Examples

>>> x = ivy.Container(a=ivy.random_normal(mean=0, std=1, shape=[1, 28, 3]),
...                   b=ivy.random_normal(mean=0, std=1, shape=[1, 56, 3]))
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 6])
>>> y = ivy.Container.static_conv1d_transpose(x, filters, 2, 'SAME')
>>> print(y.shape)
{
    a: [1,56,6],
    b: [1,112,6]
}

static _static_conv2d(x, filters, strides, padding, /, *, data_format='NHWC', filter_format='channel_last', x_dilations=1, dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container static method variant of ivy.conv2d. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

x (Union[Array, NativeArray, Container]) – Input image [batch_size,h,w,d_in].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str, default: 'NHWC') – “NHWC” or “NCHW”. Defaults to “NHWC”.
dilations (Union[int, Tuple[int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
key_chains (Optional[Union[List[str], Dict[str, str]]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (bool, default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (bool, default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (bool, default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a = ivy.eye(3, 3).reshape((1, 3, 3, 1)),
...                   b = ivy.eye(5, 5).reshape((1, 5, 5, 1)))
>>> filters = ivy.array([[2., 0., 1.],
...                      [1., 3., 1.],
...                      [0., 1., 1.]]).reshape((3, 3, 1, 1))
>>> result = ivy.Container.static_conv2d(x, filters, (2,), 'SAME')
>>> print(result)
{
    a:ivy.array([[[[4.],[0.]],[[1.],[5.]]]]),
    b:ivy.array([[[[4.],[0.],[0.]],[[1.],[6.],[0.]],[[0.],[1.],[5.]]]])
}

static _static_conv2d_transpose(x, filters, strides, padding, /, *, output_shape=None, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container static method variant of ivy.conv2d_transpose. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

x (Union[Array, NativeArray, Container]) – Input image [batch_size,h,w,d_in].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
output_shape (Optional[Union[Array, NativeArray, Container]], default: None) – Shape of the output (Default value = None)
data_format (str, default: 'NHWC') – “NHWC” or “NCHW”. Defaults to “NHWC”.
dilations (Union[int, Tuple[int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
key_chains (Optional[Union[List[str], Dict[str, str]]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (bool, default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (bool, default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (bool, default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> a = ivy.random_normal(mean=0, std=1, shape=[1, 14, 14, 3])
>>> b = ivy.random_normal(mean=0, std=1, shape=[1, 28, 28, 3])
>>> c = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 6])
>>> d = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 6])
>>> x = ivy.Container(a=a, b=b)
>>> filters = ivy.Container(c=c, d=d)
>>> y = ivy.Container.static_conv2d_transpose(x, filters, 2, 'SAME')
>>> print(y.shape)
{
    a: {
        c: [1,28,28,6],
        d: [1,28,28,6]
    },
    b: {
        c: [1,56,56,6],
        d: [1,56,56,6]
    }
}

static _static_conv3d(x, filters, strides, padding, /, *, data_format='NDHWC', filter_format='channel_last', x_dilations=1, dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container static method variant of ivy.conv3d. This method simply wraps the function, and so the docstring for ivy.conv3d also applies to this method with minimal changes.

Parameters:

x (Union[Array, NativeArray, Container]) – Input volume [batch_size,d,h,w,d_in].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fdfh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str, default: 'NDHWC') – “NDHWC” or “NCDHW”. Defaults to “NDHWC”.
filter_format (str, default: 'channel_last') – Either “channel_first” or “channel_last”. Defaults to “channel_last”.
x_dilations (Union[int, Tuple[int, int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
dilations (Union[int, Tuple[int, int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a = ivy.full((1, 2, 3, 3, 1),0.5),                              b = ivy.full((1, 2, 5, 5, 1),1.))

>>> filters = ivy.ones((3, 3, 3, 1, 1))

>>> result = ivy.Container.static_conv3d(x, filters, 2, 'SAME')
>>> print(result)
{
    a: ivy.array([[[[[4.],[4.]],[[4.],[4.]]]]]),
    b: ivy.array([[[[[8.],[12.],[8.]],[[12.],[18.],[12.]],[[8.],[12.],[8.]]]]])
}

static _static_conv3d_transpose(x, filters, strides, padding, /, *, output_shape=None, data_format='NDHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container static method variant of ivy.conv3d_transpose. This method simply wraps the function, and so the docstring for ivy.conv3d_transpose also applies to this method with minimal changes.

Parameters:

x (Union[Array, NativeArray, Container]) – Input container with leaves of volume [batch_size,d,h,w,d_in] or [batch_size,d_in,d,h,w].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fd,fh,fw,d_in,d_out].
strides (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, List[int], Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
output_shape (Optional[Union[Array, NativeArray, Container]], default: None) – Shape of the output (Default value = None)
data_format (str, default: 'NDHWC') –
The ordering of the dimensions in the input, one of “NDHWC” or “NCDHW”. “NDHWC” corresponds to inputs with shape (batch_size,

depth, height, width, channels), while “NCDHW” corresponds to input with shape (batch_size, channels, depth, height, width).
dilations (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Container], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the transpose convolution operation in a container.
>>> a = ivy.random_normal(mean=0, std=1, shape=[1, 3, 14, 14, 3])
>>> b = ivy.random_normal(mean=0, std=1, shape=[1, 3, 28, 28, 3]))
>>> c = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 3, 6])
>>> d = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 3, 6]))
>>> x = ivy.Container(a=a, b=b)
>>> filters = ivy.Container(c=c, d=d)
>>> y = ivy.Container.static_conv3d_transpose(x, filters, 2, ‘SAME’)
>>> print(y.shape)
{ –

a: {
c: [1, 6, 28, 28, 6], d: [1, 6, 28, 28, 6]

}, b: {

c: [1, 6, 56, 56, 6], d: [1, 6, 56, 56, 6]

}
}

static _static_depthwise_conv2d(x, filters, strides, padding, /, *, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.depthwise_conv2d. This method simply wraps the function, and so the docstring for ivy.depthwise_conv2d also applies to this method with minimal changes.

Parameters:

x (Container) – Input image [batch_size,h,w,d].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in]. (d_in must be the same as d from x)
strides (Union[int, Tuple[int], Tuple[int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, List[int], Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (Union[str, Container], default: 'NHWC') – “NHWC” or “NCHW”. Defaults to “NHWC”.
dilations (Union[int, Tuple[int], Tuple[int, int], Container], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
out (Optional[Container], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> a = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> b = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> inp = ivy.Container(a=a, b=b)
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3])
>>> y = ivy.Container.static_depthwise_conv2d(
...                                            inp,
...                                            filters,
...                                            strides=2,
...                                            padding='SAME')
>>> print(y.shape)
[1, 64, 64, 3]

static _static_dropout(x, prob, /, *, scale=True, dtype=None, training=True, seed=None, noise_shape=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.dropout. This method simply wraps the function, and so the docstring for ivy.dropout also applies to this method with minimal changes.

Parameters:

x (Union[Array, NativeArray, Container]) – The input container x to perform dropout on.
prob (Union[float, Container]) – The probability of zeroing out each array element, float between 0 and 1.
scale (Union[bool, Container], default: True) – Whether to scale the output by 1/(1-prob), default is True.
dtype (Optional[Union[Dtype, Container]], default: None) – Output array data type. If dtype is None, the output array data type must be inferred from x. Default: None.
training (Union[bool, Container], default: True) – Turn on dropout if training, turn off otherwise. Default is True.
seed (Optional[Union[int, Container]], default: None) – Set a default seed for random number generating (for reproducibility). Default is None.
noise_shape (Optional[Union[Sequence[int], Container]], default: None) – a sequence representing the shape of the binary dropout mask that will be multiplied with the input.
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result array of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([[1., 2., 3.], [4., 5., 6.]]),
...                   b=ivy.array([7., 8., 9.]))
>>> y = ivy.Container.static_dropout(x, 0.3)
>>> print(y)
{
    a: ivy.array([[0., 0., 4.28571415],
                  [5.71428585, 7.14285755, 0.]]),
    b: ivy.array([0., 11.4285717, 12.8571434])
}

static _static_dropout1d(x, prob, /, *, training=True, data_format='NWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.dropout1d. This method simply wraps the function, and so the docstring for ivy.dropout1d also applies to this method with minimal changes.

Parameters:

x (Container) – The input container to perform dropout on.
prob (Union[float, Container]) – The probability of zeroing out each array element, float between 0 and 1.
training (Union[bool, Container], default: True) – Turn on dropout if training, turn off otherwise. Default is True.
data_format (Union[str, Container], default: 'NWC') – “NWC” or “NCW”. Default is "NCW".
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([1, 2, 3]).reshape([1, 1, 3]),
...                   b=ivy.array([4, 5, 6]).reshape([1, 1, 3]))
>>> y = ivy.Container.static_dropout1d(x, 0.5)
>>> print(y)
{
    a: ivy.array([[[0., 4., 0.]]]),
    b: ivy.array([[[0., 0., 12.]]])
}

static _static_dropout2d(x, prob, /, *, training=True, data_format='NHWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.dropout2d. This method simply wraps the function, and so the docstring for ivy.dropout2d also applies to this method with minimal changes.

Parameters:

x (Container) – The input container to perform dropout on.
prob (Union[float, Container]) – The probability of zeroing out each array element, float between 0 and 1.
training (Union[bool, Container], default: True) – Turn on dropout if training, turn off otherwise. Default is True.
data_format (Union[str, Container], default: 'NHWC') – “NHWC” or “NCHW”. Default is "NHWC".
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

static _static_dropout3d(x, prob, /, *, training=True, data_format='NDHWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.dropout3d. This method simply wraps the function, and so the docstring for ivy.dropout3d also applies to this method with minimal changes.

Parameters:

x (Container) – The input container to perform dropout on.
prob (Union[float, Container]) – The probability of zeroing out each array element, float between 0 and 1.
training (Union[bool, Container], default: True) – Turn on dropout if training, turn off otherwise. Default is True.
data_format (Union[str, Container], default: 'NDHWC') – “NDHWC” or “NCDHW”. Default is "NDHWC".
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

static _static_linear(x, weight, /, *, bias=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.linear. This method simply wraps the function, and so the docstring for ivy.linear also applies to this method with minimal changes.

Parameters:

x (Container) – The input x to compute linear transformation on. [outer_batch_shape,inner_batch_shape,in_features]
weight (Union[Array, NativeArray, Container]) – The weight matrix. [outer_batch_shape,out_features,in_features]
bias (Optional[Union[Array, NativeArray, Container]], default: None) – The bias vector, default is None. [outer_batch_shape,out_features]
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result array of the linear transformation. [outer_batch_shape,inner_batch_shape,out_features]

Examples

>>> x = ivy.Container(a=ivy.array([[1.1, 2.2, 3.3],                                            [11., 22., 33.]]),                               b=ivy.array([[1.245, 0.278, 4.105],                                            [7., 13., 17.]]))
>>> w = ivy.array([[1., 2., 3.],                            [4., 5., 6.],                            [7., 8., 9.]])
>>> b = ivy.array([1., 0., -1.])
>>> y = ivy.Container.static_linear(x, w, bias=b)
>>> print(y)
{
    a: ivy.array([[16.4, 35.2, 54.],
                  [155., 352., 549.]]),
    b: ivy.array([[15.1, 31., 46.9],
                  [85., 195., 305.]])
}

>>> x = ivy.Container(a=ivy.array([[1.1, 2.2, 3.3],                                            [.0, .1, .2]]),                               b=ivy.array([[1.245, 0.278, 4.105],                                            [.7, .8, .9]]))
>>> w = ivy.Container(a=ivy.array([[1., 2., 3.]]),                               b=ivy.array([[.1, .2, .3]]))
>>> b = ivy.Container(a=ivy.array([1.]), b=ivy.array([-1.]))
>>> y = ivy.Container.static_linear(x, w, bias=b)
>>> print(y)
{
    a: ivy.array([[16.4],
                  [1.8]]),
    b: ivy.array([[0.412],
                  [-0.5]])
}

static _static_lstm_update(x, init_h, init_c, kernel, recurrent_kernel, /, *, bias=None, recurrent_bias=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False)[source]#

Return type:: Tuple[Container, Container]

static _static_multi_head_attention(query, /, *, key=None, value=None, num_heads=8, scale=None, attention_mask=None, in_proj_weights=None, q_proj_weights=None, k_proj_weights=None, v_proj_weights=None, out_proj_weights=None, in_proj_bias=None, out_proj_bias=None, is_causal=False, key_padding_mask=None, bias_k=None, bias_v=None, static_k=None, static_v=None, add_zero_attn=False, return_attention_weights=False, average_attention_weights=True, dropout=0.0, training=False, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

Return type:: Container

static _static_reduce_window(operand, init_value, computation, window_dimensions, /, *, window_strides=1, padding='VALID', base_dilation=1, window_dilation=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False)[source]#

Return type:: Container

static _static_scaled_dot_product_attention(query, key, value, /, *, scale, mask=None, dropout_p=0.0, is_causal=False, training=False, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.scaled_dot_product_attention. This method simply wraps the function, and so the docstring for ivy.scaled_dot_product_attention also applies to this method with minimal changes.

Parameters:

query (Union[Array, NativeArray, Container]) – The queries input container. The shape of queries input array leaves should be in [batch_shape,num_queries,feat_dim]. The queries input array leaves should have the same size as keys and values.
key (Union[Array, NativeArray, Container]) – The keys input array container. The shape of keys input array leaves should be in [batch_shape,num_keys,feat_dim]. The keys input array leaves should have the same size as queries and values.
value (Union[Array, NativeArray, Container]) – The values input array container. The shape of values input array leaves should be in [batch_shape,num_keys,feat_dim]. The values input array leaves should have the same size as queries and keys.
scale (Union[float, Container]) – The scale float value. The scale float value is used to scale the query-key pairs before softmax.
mask (Optional[Union[Array, NativeArray, Container]], default: None) – The mask input array/container. The mask to apply to the query-key values. Default is None. The shape of mask input array leaves should be in [batch_shape,num_queries,num_keys].
dropout_p (Optional[float], default: 0.0) – Specifies the dropout probablity, if greater than 0.0, dropout is applied
is_causal (Optional[bool], default: False) – If true, assumes causal attention masking and errors if both mask and is_causal are set.
training (Optional[bool], default: False) – If True, dropout is used, otherwise dropout is not activated.
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The output container following applications of scaled dot-product attention. The output array is the weighted sum produced by the attention score and value. The shape of output array is [batch_shape,num_queries,feat_dim] .

Examples

With ivy.Container input:

>>> q = ivy.Container(a=ivy.array([[[0.2, 1.], [2.7, 3.], [4.4, 5.6]]]),
...                   b=ivy.array([[[1.2, 1.], [2.2, 3.], [4.4, 5.6]]]))
>>> k = ivy.Container(a=ivy.array([[[4.2, 1.], [2.2, 3.3],[4.4, 5.6]]]),
...                   b=ivy.array([[[3.2, 1.], [2.2, 3.6], [4.0, 5.6]]]))
>>> v = ivy.Container(a=ivy.array([[[5.2, 1.], [2.1, 3.],[4.4, 5.6]]]),
...                   b=ivy.array([[[0.2, 1.], [2.2, 3.],[4.4, 5.6]]]))
>>> mask =
... ivy.Container(a=ivy.array([[[1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0],
...                             [1.0, 1.0,1.0]]]),
...               b=ivy.array([[[1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0],
...                             [1.0, 1.0,1.0]]]))
>>> result = ivy.Container.static_scaled_dot_product_attention(q,
...                                                            k,
...                                                            v,
...                                                            1,
...                                                            mask=mask)
>>> print(result)
{
    a: ivy.array([[[4.27, 5.4],
                [4.4, 5.6],
                [4.4, 5.6]]]),
    b: ivy.array([[[4.35, 5.54],
                [4.4, 5.6],
                [4.4, 5.6]]])
}

conv1d(filters, strides, padding, /, *, data_format='NWC', filter_format='channel_last', x_dilations=1, dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container instance method variant of ivy.conv1d. This method simply wraps the function, and so the docstring for ivy.conv1d also applies to this method with minimal changes.

Parameters:

self (Container) – Input image [batch_size,w, d_in].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fw,d_in, d_out]. (d_in must be the same as d from x)
strides (Union[int, Tuple[int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str, default: 'NWC') – “NWC” or “NCW”. Defaults to “NWC”.
filter_format (str, default: 'channel_last') – Either “channel_first” or “channel_last”. Defaults to “channel_last”.
x_dilations (Union[int, Tuple[int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
dilations (Union[int, Tuple[int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
key_chains (Optional[Union[List[str], Dict[str, str]]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (bool, default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (bool, default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (bool, default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Container], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a=ivy.array([[[2., 3., 4.], [5., 6., 7.]]]),
...                   b=ivy.array([[[7., 8., 9.], [10., 11., 12]]]))
>>> filters = ivy.array([[[0., 0.5, 1.], [0.25, 0.5, 0.75], [-0.5, 0., 0.5 ]]])
>>> result= x.conv1d(filters, (1,), 'VALID')
>>> print(result)
{
    ... a: ivy.array([[[-1.25, 2.5, 6.25],
    ...                [-2., 5.5, 13.]]]),
    ... b: ivy.array([[[-2.5, 7.5, 17.5],
    ...                [-3.25, 10.5, 24.2]]])
}

conv1d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container instance method variant of ivy.conv1d_transpose. This method simply wraps the function, and so the docstring for ivy.conv1d_transpose also applies to this method with minimal changes.

Parameters:

self (Container) – Input image [batch_size,w,d_in] or [batch_size,d_in,w].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fw,d_in,d_out].
strides (Union[int, Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – either the string ‘SAME’ (padding with zeros evenly), the string ‘VALID’ (no padding), or a sequence of n (low, high) integer pairs that give the padding to apply before and after each spatial dimension.
output_shape (Optional[Union[Array, NativeArray, Container]], default: None) – Shape of the output (Default value = None)
data_format (str, default: 'NWC') – The ordering of the dimensions in the input, one of “NWC” or “NCW”. “NWC” corresponds to input with shape (batch_size, width, channels), while “NCW” corresponds to input with shape (batch_size, channels, width).
dilations (int, default: 1) – The dilation factor for each dimension of input. (Default value = 1)
key_chains (Optional[Union[List[str], Dict[str, str]]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (bool, default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (bool, default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (bool, default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Union[Array, Container]], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Union[Array, NativeArray, Container]

Returns:

ret – The result of the transpose convolution operation.

Examples

>>> x = ivy.Container(a=ivy.random_normal(mean=0, std=1, shape=[1, 28, 3]),
...                   b=ivy.random_normal(mean=0, std=1, shape=[1, 56, 3]))
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 6])
>>> y = x.conv1d_transpose(filters, 2, 'SAME')
>>> print(y.shape)
{
    a: ivy.Shape(1, 56, 6),
    b: ivy.Shape(1, 112, 6)
}

conv2d(filters, strides, padding, /, *, data_format='NHWC', filter_format='channel_last', x_dilations=1, dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container instance method variant of ivy.conv2d. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

self (Container) – Input image [batch_size,h,w,d_in].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str, default: 'NHWC') – “NHWC” or “NCHW”. Defaults to “NHWC”.
dilations (Union[int, Tuple[int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
key_chains (Optional[Union[List[str], Dict[str, str]]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (bool, default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (bool, default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (bool, default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a = ivy.eye(3, 3).reshape((1, 3, 3, 1)),
...                   b = ivy.eye(5, 5).reshape((1, 5, 5, 1)))
>>> filters = ivy.array([[2, 0, 1],
...                      [1, 3, 1],
...                      [0, 1, 1]], dtype=ivy.float32).reshape((3, 3, 1, 1))
>>> result = x.conv2d(filters, 2, 'SAME')
>>> print(result)
{
    a:ivy.array([[[[4.],[0.]],[[1.],[5.]]]]),
    b:ivy.array([[[[4.],[0.],[0.]],[[1.],[6.],[0.]],[[0.],[1.],[5.]]]])
}

conv2d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container instance method variant of ivy.conv2d_transpose. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

self (Container) – Input image [batch_size,h,w,d_in].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
output_shape (Optional[Union[Array, NativeArray, Container]], default: None) – Shape of the output (Default value = None)
data_format (str, default: 'NHWC') – “NHWC” or “NCHW”. Defaults to “NHWC”.
dilations (Union[int, Tuple[int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
key_chains (Optional[Union[List[str], Dict[str, str]]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (bool, default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (bool, default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (bool, default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> a = ivy.random_normal(mean=0, std=1, shape=[1, 14, 14, 3])
>>> b = ivy.random_normal(mean=0, std=1, shape=[1, 28, 28, 3])
>>> c = ivy.random_normal(mean=0, std=1, shape=[6, 3, 3, 3])
>>> d = ivy.random_normal(mean=0, std=1, shape=[6, 3, 3, 3])
>>> x = ivy.Container(a=a, b=b)
>>> filters = ivy.Container(c=c, d=d)
>>> y = x.conv2d_transpose(filters,2,'SAME')
>>> print(y.shape)
{
    a: {
        c: ivy.Shape(1, 28, 28, 3),
        d: ivy.Shape(1, 28, 28, 3)
    },
    b: {
        c: ivy.Shape(1, 56, 56, 3),
        d: ivy.Shape(1, 56, 56, 3)
    },
    c: {
        c: ivy.Shape(6, 6, 6, 3),
        d: ivy.Shape(6, 6, 6, 3)
    },
    d: {
        c: ivy.Shape(6, 6, 6, 3),
        d: ivy.Shape(6, 6, 6, 3)
    }
}

conv3d(filters, strides, padding, /, *, data_format='NDHWC', filter_format='channel_last', x_dilations=1, dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container instance method variant of ivy.conv3d. This method simply wraps the function, and so the docstring for ivy.conv3d also applies to this method with minimal changes.

Parameters:

x – Input volume [batch_size,d,h,w,d_in].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fdfh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str, default: 'NDHWC') – “NDHWC” or “NCDHW”. Defaults to “NDHWC”.
filter_format (str, default: 'channel_last') – Either “channel_first” or “channel_last”. Defaults to “channel_last”.
x_dilations (Union[int, Tuple[int, int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
dilations (Union[int, Tuple[int, int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a = ivy.full((1, 2, 3, 3, 1),0.5),                              b = ivy.full((1, 2, 5, 5, 1),1.))

>>> filters = ivy.ones((3, 3, 3, 1, 1))

>>> result = x.conv3d(filters, 2, 'SAME')
>>> print(result)
{
    a: ivy.array([[[[[4.],[4.]],[[4.],[4.]]]]]),
    b: ivy.array([[[[[8.],[12.],[8.]],[[12.],[18.],[12.]],[[8.],[12.],[8.]]]]])
}

conv3d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NDHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, bias=None, out=None)[source]#

ivy.Container instance method variant of ivy.conv3d_transpose. This method simply wraps the function, and so the docstring for ivy.conv3d_transpose also applies to this method with minimal changes.

Parameters:

self (Container) – Input container with leaves of volume [batch_size,d,h,w,d_in] or [batch_size,d_in,d,h,w].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fd,fh,fw,d_in,d_out].
strides (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, List[int], Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
output_shape (Optional[Union[Array, NativeArray, Container]], default: None) – Shape of the output (Default value = None)
data_format (str, default: 'NDHWC') –
The ordering of the dimensions in the input, one of “NDHWC” or “NCDHW”. “NDHWC” corresponds to inputs with shape (batch_size,

depth, height, width, channels), while “NCDHW” corresponds to input with shape (batch_size, channels, depth, height, width).
dilations (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int]], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
bias (Optional[Container], default: None) – Bias array of shape [d_out].
out (Optional[Container], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the transpose convolution operation in a container.

Examples

>>> x = ivy.Container(a = ivy.ones((1, 3, 3, 3, 1)).astype(ivy.float32) )
>>> filters = ivy.ones((3, 3, 3, 1, 1)).astype(ivy.float32)
>>> result = x.conv3d(filters, 2, 'SAME')
>>> print(result)
{
    a: ivy.array([[[[[8.],
                     [8.]],
                    [[8.],
                     [8.]]],
                  [[[8.],
                     [8.]],
                    [[8.],
                     [8.]]]]])
}

depthwise_conv2d(filters, strides, padding, /, *, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.depthwise_conv2d. This method simply wraps the function, and so the docstring for ivy.depthwise_conv2d also applies to this method with minimal changes.

Parameters:

self (Container) – Input image [batch_size,h,w,d].
filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in]. (d_in must be the same as d from self)
strides (Union[int, Tuple[int], Tuple[int, int], Container]) – The stride of the sliding window for each dimension of input.
padding (Union[str, List[int], Container]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (Union[str, Container], default: 'NHWC') – “NHWC” or “NCHW”. Defaults to “NHWC”.
dilations (Union[int, Tuple[int], Tuple[int, int], Container], default: 1) – The dilation factor for each dimension of input. (Default value = 1)
out (Optional[Container], default: None) – optional output container, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> a = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> b = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> inp = ivy.Container(a=a, b=b)
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3])
>>> y = inp.depthwise_conv2d(filters, 2, 'SAME')
>>> print(y.shape)
[1, 64, 64, 3]

dropout(prob, /, *, scale=True, dtype=None, training=True, seed=None, noise_shape=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.dropout. This method simply wraps the function, and so the docstring for ivy.dropout also applies to this method with minimal changes.

Parameters:

self (Container) – The input container to perform dropout on.
prob (Union[float, Container]) – The probability of zeroing out each array element, float between 0 and 1.
scale (Union[bool, Container], default: True) – Whether to scale the output by 1/(1-prob), default is True.
dtype (Optional[Union[Dtype, Container]], default: None) – output array data type. If dtype is None, the output array data type must be inferred from x. Default: None.
training (Union[bool, Container], default: True) – Turn on dropout if training, turn off otherwise. Default is True.
seed (Optional[Union[int, Container]], default: None) – Set a default seed for random number generating (for reproducibility). Default is None.
noise_shape (Optional[Union[Sequence[int], Container]], default: None) – a sequence representing the shape of the binary dropout mask that will be multiplied with the input.
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result array of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([[1., 2., 3.], [4., 5., 6.]]),
...                   b=ivy.array([7., 8., 9.]))
>>> y = x.dropout(0.3)
>>> print(y)
{
    a: ivy.array([[0., 0., 4.28571415],
                  [5.71428585, 7.14285755, 0.]]),
    b: ivy.array([0., 11.4285717, 12.8571434])
}

dropout1d(prob, /, *, training=True, data_format='NWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.dropout1d. This method simply wraps the function, and so the docstring for ivy.dropout1d also applies to this method with minimal changes.

Parameters:

self (Container) – The input container to perform dropout on.
prob (Union[float, Container]) – The probability of zeroing out each array element, float between 0 and 1.
training (Union[bool, Container], default: True) – Turn on dropout if training, turn off otherwise. Default is True.
data_format (Union[str, Container], default: 'NWC') – “NWC” or “NCW”. Default is "NCW".
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([1, 2, 3]).reshape([1, 1, 3]),
...                   b=ivy.array([4, 5, 6]).reshape([1, 1, 3]))
>>> y = x.dropout1d(x, 0.5)
>>> print(y)
{
    a: ivy.array([[[0., 4., 0.]]]),
    b: ivy.array([[[0., 0., 12.]]])
}

dropout2d(prob, /, *, training=True, data_format='NHWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.dropout2d. This method simply wraps the function, and so the docstring for ivy.dropout2d also applies to this method with minimal changes.

Parameters:

self (Container) – The input container to perform dropout on.
prob (Union[float, Container]) – The probability of zeroing out each array element, float between 0 and 1.
training (Union[bool, Container], default: True) – Turn on dropout if training, turn off otherwise. Default is True.
data_format (Union[str, Container], default: 'NHWC') – “NHWC” or “NCHW”. Default is "NHWC".
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([[100, 200, 300]]),
...                   b=ivy.array([[400, 500, 600]]))
>>> y = x.dropout2d(0.5)
>>> print(y)
{
    a: ivy.array([[200., 0., 600.]]),
    b: ivy.array([[0., 0., 0.]])
}

dropout3d(prob, /, *, training=True, data_format='NDHWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.dropout3d. This method simply wraps the function, and so the docstring for ivy.dropout3d also applies to this method with minimal changes.

Parameters:

self (Container) – The input container to perform dropout on.
prob (Union[float, Container]) – The probability of zeroing out each array element, float between 0 and 1.
training (Union[bool, Container], default: True) – Turn on dropout if training, turn off otherwise. Default is True.
data_format (Union[str, Container], default: 'NDHWC') – “NDHWC” or “NCDHW”. Default is "NDHWC".
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

linear(weight, /, *, bias=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.linear. This method simply wraps the function, and so the docstring for ivy.linear also applies to this method with minimal changes.

Parameters:

self (Container) – The input container to compute linear transformation on. [outer_batch_shape,inner_batch_shape,in_features]
weight (Union[Array, NativeArray, Container]) – The weight matrix. [outer_batch_shape,out_features,in_features]
bias (Optional[Union[Array, NativeArray, Container]], default: None) – The bias vector, default is None. [outer_batch_shape,out_features]
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result array of the linear transformation. [outer_batch_shape,inner_batch_shape,out_features]

Examples

>>> x = ivy.Container(a=ivy.array([[1.1, 2.2, 3.3],
...                                [11., 22., 33.]]),
...                   b=ivy.array([[1.245, 0.278, 4.105],
...                                [7., 13., 17.]]))
>>> w = ivy.array([[1., 2., 3.],
...                [4., 5., 6.],
...                [7., 8., 9.]])
>>> b = ivy.Container(a=ivy.array([1., 0., -1.]),
...                   b=ivy.array([1., 1., 0.]))
>>> y = x.linear(w, bias=b, out=x)
>>> print(y)
{
    a: ivy.array([[16.39999962, 35.19999695, 54.],
                  [155., 352., 549.]]),
    b: ivy.array([[15.11600018, 32., 47.88399887],
                  [85., 196., 306.]])
}

lstm_update(init_h, init_c, kernel, recurrent_kernel, /, *, bias=None, recurrent_bias=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False)[source]#

ivy.Container instance method variant of ivy.lstm_update. This method simply wraps the function, and so the docstring for ivy.lstm_update also applies to this method with minimal changes.

Parameters:

init_h (Union[Array, NativeArray, Container]) – initial state tensor for the cell output [batch_shape, out].
init_c (Union[Array, NativeArray, Container]) – initial state tensor for the cell hidden state [batch_shape, out].
kernel (Union[Array, NativeArray, Container]) – weights for cell kernel [in, 4 x out].
recurrent_kernel (Union[Array, NativeArray, Container]) – weights for cell recurrent kernel [out, 4 x out].
bias (Optional[Union[Array, NativeArray, Container]], default: None) – bias for cell kernel [4 x out]. (Default value = None)
recurrent_bias (Optional[Union[Array, NativeArray, Container]], default: None) – bias for cell recurrent kernel [4 x out]. (Default value = None)

Return type:

Tuple[Container, Container]

Returns:

ret – hidden state for all timesteps [batch_shape,t,out] and cell state for last timestep [batch_shape,out]

Examples

>>> x = ivy.Container(
...     a=ivy.random_normal(shape=(5, 20, 3)),
...     b=ivy.random_normal(shape=(5, 20, 3))
... )
>>> h_i = ivy.random_normal(shape=(5, 6))
>>> c_i = ivy.random_normal(shape=(5, 6))

>>> kernel = ivy.random_normal(shape=(3, 4 * 6))
>>> rc = ivy.random_normal(shape=(6, 4 * 6))
>>> x.lstm_update(h_i, c_i, kernel, rc)
{
    a: (tuple(2), <class ivy.array.array.Array>, shape=[5, 20, 6]),
    b: (tuple(2), <class ivy.array.array.Array>, shape=[5, 20, 6])
}

multi_head_attention(*, key=None, value=None, num_heads=8, scale=None, attention_mask=None, in_proj_weights=None, q_proj_weights=None, k_proj_weights=None, v_proj_weights=None, out_proj_weights=None, in_proj_bias=None, out_proj_bias=None, is_causal=False, key_padding_mask=None, bias_k=None, bias_v=None, static_k=None, static_v=None, add_zero_attn=False, return_attention_weights=False, average_attention_weights=True, dropout=0.0, training=False, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

Return type:: Container

reduce_window(init_value, computation, window_dimensions, /, *, window_strides=1, padding='VALID', base_dilation=1, window_dilation=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False)[source]#

ivy.Container instance method variant of ivy.reduce_window. This method simply wraps the function, and so the docstring for ivy.reduce_window also applies to this method with minimal changes.

Parameters:

self (Container) – A container representing the base areas on which the window is going to slide over.
init_value (Union[int, float, Container]) – The starting value for the reduction.
computation (Union[Callable, Container]) – The reduction function to apply to elements in each window.
window_dimensions (Union[int, Sequence[int], Container]) – A sequence containing the window dimensions.
window_strides (Union[int, Sequence[int], Container], default: 1) – A sequence containing the window strides.
padding (Union[str, int, Sequence[Tuple[int, int]], Container], default: 'VALID') – Either the string ‘SAME’ (padding with zeros evenly), the string ‘VALID’ (no padding), or a sequence of n (low, high) integer pairs that give the padding to apply before and after each spatial dimension.
base_dilation (Union[int, Sequence[int], Container], default: 1) – A sequence containing the base dilation values.
window_dilation (Union[int, Sequence[int], Container], default: 1) – A sequence containing the window dilation values.

Return type:

Container

Returns:

ret – The result of the pooling-like operation.

Examples

>>> x = ivy.Container(
...     a=ivy.array([[1, 2, 3, 4],
...                  [5, 6, 7, 8],
...                  [9, 10, 11, 12]]),
...     b=ivy.array([[13, 14, 15, 16],
...                  [17, 18, 19, 20],
...                  [21, 22, 23, 24]])
... )
>>> x.reduce_window(0, ivy.sum, (2, 2))
{
    a: ivy.array([[21 25 29]
                  [33 37 41]
                  [45 49 53]]),
    b: ivy.array([[63 67 71]
                  [75 79 83]
                  [87 91 95]])
}

scaled_dot_product_attention(key, value, /, *, scale, mask=None, dropout_p=0.0, is_causal=False, training=False, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.scaled_dot_product_attention. This method simply wraps the function, and so the docstring for ivy.scaled_dot_product_attention also applies to this method with minimal changes.

Parameters:

self (Container) – The queries input container. The shape of queries input array leaves should be in [batch_shape,num_queries,feat_dim]. The queries input array leaves should have the same size as keys and values.
key (Union[Array, NativeArray, Container]) – The keys input array container. The shape of keys input array leaves should be in [batch_shape,num_keys,feat_dim]. The keys input array leaves should have the same size as queries and values.
value (Union[Array, NativeArray, Container]) – The values input array container. The shape of values input array leaves should be in [batch_shape,num_keys,feat_dim]. The values input array leaves should have the same size as queries and keys.
scale (Union[float, Container]) – The scale float value. The scale float value is used to scale the query-key pairs before softmax.
mask (Optional[Union[Array, NativeArray, Container]], default: None) – The mask input array/container. The mask to apply to the query-key values. Default is None. The shape of mask input array leaves should be in [batch_shape,num_queries,num_keys].
dropout_p (Optional[float], default: 0.0) – Specifies the dropout probablity, if greater than 0.0, dropout is applied
is_causal (Optional[bool], default: False) – If true, assumes causal attention masking and errors if both mask and is_causal are set.
training (Optional[bool], default: False) – If True, dropout is used, otherwise dropout is not activated.
key_chains (Optional[Union[List[str], Dict[str, str], Container]], default: None) – The key-chains to apply or not apply the method to. Default is None.
to_apply (Union[bool, Container], default: True) – If True, the method will be applied to key_chains, otherwise key_chains will be skipped. Default is True.
prune_unapplied (Union[bool, Container], default: False) – Whether to prune key_chains for which the function was not applied. Default is False.
map_sequences (Union[bool, Container], default: False) – Whether to also map method to sequences (lists, tuples). Default is False.
out (Optional[Container], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Container

Returns:

Examples

With ivy.Container input:

>>> q = ivy.Container(a=ivy.array([[[0.2, 1.], [2.7, 3.], [4.4, 5.6]]]),
...                   b=ivy.array([[[1.2, 1.], [2.2, 3.], [4.4, 5.6]]]))
>>> k = ivy.Container(a=ivy.array([[[4.2, 1.], [2.2, 3.3], [4.4, 5.6]]]),
...                   b=ivy.array([[[3.2, 1.], [2.2, 3.6], [4.0, 5.6]]]))

>>> v = ivy.Container(a=ivy.array([[[5.2, 1.], [2.1, 3.], [4.4, 5.6]]]),
...                   b=ivy.array([[[0.2, 1.], [2.2, 3.], [4.4, 5.6]]]))
>>> result = ivy.scaled_dot_product_attention(q, k, v, scale=1, dropout_p=0.1,
...                                           is_causal=True, training=True)
>>> print(result)
{
    a: ivy.array([[[5.19999981, 1.],
                   [2.59249449, 2.68226194],
                   [4.4000001, 5.5999999]]]),
    b: ivy.array([[[0.2, 1.],
                   [2.19603825, 2.9960382],
                   [4.4000001, 5.5999999]]])
}

>>> q = ivy.Container(a=ivy.array([[[0.2, 1.], [2.7, 3.], [4.4, 5.6]]]),
...                   b=ivy.array([[[1.2, 1.], [2.2, 3.], [4.4, 5.6]]]))
>>> k = ivy.Container(a=ivy.array([[[4.2, 1.], [2.2, 3.3], [4.4, 5.6]]]),
...                   b=ivy.array([[[3.2, 1.], [2.2, 3.6], [4.0, 5.6]]]))
>>> v = ivy.Container(a=ivy.array([[[5.2, 1.], [2.1, 3.], [4.4, 5.6]]]),
...                   b=ivy.array([[[0.2, 1.], [2.2, 3.], [4.4, 5.6]]]))
>>> mask =
... ivy.Container(a=ivy.array([[[1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0]]]),
...               b=ivy.array([[[1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0],
...                             [1.0, 1.0,1.0]]]))
>>> result = ivy.scaled_dot_product_attention(q,k,v,scale=1,mask=mask)
>>> print(result)
{
    a: ivy.array([[[4.26894283, 5.40236187],
                   [4.39999437, 5.59999037],
                   [4.4000001, 5.5999999]]]),
    b: ivy.array([[[4.35046196, 5.54282808],
                   [4.39989519, 5.5998764],
                   [4.4000001, 5.5999999]]])

}