Layers#

class ivy.data_classes.container.layers._ContainerWithLayers(dict_in=None, queues=None, queue_load_sizes=None, container_combine_method='list_join', queue_timeout=None, print_limit=10, key_length_limit=None, print_indent=4, print_line_spacing=0, ivyh=None, default_key_color='green', keyword_color_dict=None, rebuild_child_containers=False, types_to_iteratively_nest=None, alphabetical_keys=True, dynamic_backend=None, **kwargs)[source]#

Bases: ContainerBase

_abc_impl = <_abc_data object>#
static _static_conv1d(x, filters, strides, padding, /, *, data_format='NWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.conv1d. This method simply wraps the function, and so the docstring for ivy.conv1d also applies to this method with minimal changes.

Parameters:

  • x (Union[Array, NativeArray, Container]) – Input image [batch_size,w, d_in].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fw,d_in, d_out]. (d_in must be the same as d from x)

  • strides (Union[int, Tuple[int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • data_format (str) – “NWC” or “NCW”. Defaults to “NWC”. (default: 'NWC')

  • dilations (Union[int, Tuple[int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output container, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a=ivy.array([[[2., 3., 4.], [5., 6., 7.]]]),
...                   b=ivy.array([[[7., 8., 9.], [10., 11., 12]]]))
>>> filters = ivy.array([[[0., 0.5, 1.], [0.25, 0.5, 0.75], [-0.5, 0., 0.5 ]]])
>>> result= ivy.Container.static_conv1d(x,filters,(1,),'VALID')
>>> print(result)
{
    ... a: ivy.array([[[-1.25, 2.5, 6.25],
    ...                [-2., 5.5, 13.]]]),
    ... b: ivy.array([[[-2.5, 7.5, 17.5],
    ...                [-3.25, 10.5, 24.2]]])
}
static _static_conv1d_transpose(x, filters, strides, padding, /, *, output_shape=None, data_format='NWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.conv1d_transpose. This method simply wraps the function, and so the docstring for ivy.conv1d_transpose also applies to this method with minimal changes.

Parameters:

  • x (Union[Array, NativeArray, Container]) – Input image [batch_size,w,d_in] or [batch_size,d_in,w].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fw,d_in,d_out].

  • strides (Union[int, Tuple[int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – either the string ‘SAME’ (padding with zeros evenly), the string ‘VALID’ (no padding), or a sequence of n (low, high) integer pairs that give the padding to apply before and after each spatial dimension.

  • output_shape (Optional[Union[Array, NativeArray, Container]]) – Shape of the output (Default value = None) (default: None)

  • data_format (str) – The ordering of the dimensions in the input, one of “NWC” or “NCW”. “NWC” (default: 'NWC') corresponds to input with shape (batch_size, width, channels), while “NCW” corresponds to input with shape (batch_size, channels, width).

  • dilations (Union[int, Tuple[int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output container, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the transpose convolution operation.

Examples

>>> x = ivy.Container(a=ivy.random_normal(mean=0, std=1, shape=[1, 28, 3]),
...                   b=ivy.random_normal(mean=0, std=1, shape=[1, 56, 3]))
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 6])
>>> y = ivy.Container.static_conv1d_transpose(x, filters, 2, 'SAME')
>>> print(y.shape)
{
    a: [1,56,6],
    b: [1,112,6]
}
static _static_conv2d(x, filters, strides, padding, /, *, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.conv2d. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

  • x (Union[Array, NativeArray, Container]) – Input image [batch_size,h,w,d_in].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in,d_out].

  • strides (Union[int, Tuple[int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • data_format (str) – “NHWC” or “NCHW”. Defaults to “NHWC”. (default: 'NHWC')

  • dilations (Union[int, Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a = ivy.eye(3, 3).reshape((1, 3, 3, 1)),
...                   b = ivy.eye(5, 5).reshape((1, 5, 5, 1)))
>>> filters = ivy.array([[2., 0., 1.],
...                      [1., 3., 1.],
...                      [0., 1., 1.]]).reshape((3, 3, 1, 1))
>>> result = ivy.Container.static_conv2d(x, filters, (2,), 'SAME')
>>> print(result)
{
    a:ivy.array([[[[4.],[0.]],[[1.],[5.]]]]),
    b:ivy.array([[[[4.],[0.],[0.]],[[1.],[6.],[0.]],[[0.],[1.],[5.]]]])
}
static _static_conv2d_transpose(x, filters, strides, padding, /, *, output_shape=None, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.conv2d_transpose. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

  • x (Union[Array, NativeArray, Container]) – Input image [batch_size,h,w,d_in].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in,d_out].

  • strides (Union[int, Tuple[int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • output_shape (Optional[Union[Array, NativeArray, Container]]) – Shape of the output (Default value = None) (default: None)

  • data_format (str) – “NHWC” or “NCHW”. Defaults to “NHWC”. (default: 'NHWC')

  • dilations (Union[int, Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> a = ivy.random_normal(mean=0, std=1, shape=[1, 14, 14, 3])
>>> b = ivy.random_normal(mean=0, std=1, shape=[1, 28, 28, 3])
>>> c = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 6])
>>> d = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 6])
>>> x = ivy.Container(a=a, b=b)
>>> filters = ivy.Container(c=c, d=d)
>>> y = ivy.Container.static_conv2d_transpose(x, filters, 2, 'SAME')
>>> print(y.shape)
{
    a: {
        c: [1,28,28,6],
        d: [1,28,28,6]
    },
    b: {
        c: [1,56,56,6],
        d: [1,56,56,6]
    }
}
static _static_conv3d(x, filters, strides, padding, /, *, data_format='NDHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.conv3d. This method simply wraps the function, and so the docstring for ivy.conv3d also applies to this method with minimal changes.

Parameters:

  • x (Union[Array, NativeArray, Container]) – Input volume [batch_size,d,h,w,d_in].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fdfh,fw,d_in,d_out].

  • strides (Union[int, Tuple[int, int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • data_format (str) – “NDHWC” or “NCDHW”. Defaults to “NDHWC”. (default: 'NDHWC')

  • dilations (Union[int, Tuple[int, int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a = ivy.full((1, 2, 3, 3, 1),0.5),                              b = ivy.full((1, 2, 5, 5, 1),1.))

>>> filters = ivy.ones((3, 3, 3, 1, 1))

>>> result = ivy.Container.static_conv3d(x, filters, 2, 'SAME')
>>> print(result)
{
    a: ivy.array([[[[[4.],[4.]],[[4.],[4.]]]]]),
    b: ivy.array([[[[[8.],[12.],[8.]],[[12.],[18.],[12.]],[[8.],[12.],[8.]]]]])
}
static _static_conv3d_transpose(x, filters, strides, padding, /, *, output_shape=None, data_format='NDHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.conv3d_transpose. This method simply wraps the function, and so the docstring for ivy.conv3d_transpose also applies to this method with minimal changes.

Parameters:

  • x (Union[Array, NativeArray, Container]) – Input container with leaves of volume [batch_size,d,h,w,d_in] or [batch_size,d_in,d,h,w].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fd,fh,fw,d_in,d_out].

  • strides (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (Union[str, List[int]]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • output_shape (Optional[Union[Array, NativeArray, Container]]) – Shape of the output (Default value = None) (default: None)

  • data_format (str) –

    The ordering of the dimensions in the input, one of “NDHWC” or (default: 'NDHWC') “NCDHW”. “NDHWC” corresponds to inputs with shape (batch_size,

    depth, height, width, channels), while “NCDHW” corresponds to input with shape (batch_size, channels, depth, height, width).

  • dilations (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • out (Optional[Container]) – optional output container, for writing the result to. It must (default: None) have a shape that the inputs broadcast to.

Return type:

Container

Returns:

  • ret – The result of the transpose convolution operation in a container.

  • >>> a = ivy.random_normal(mean=0, std=1, shape=[1, 3, 14, 14, 3])

  • >>> b = ivy.random_normal(mean=0, std=1, shape=[1, 3, 28, 28, 3]))

  • >>> c = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 3, 6])

  • >>> d = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 3, 6]))

  • >>> x = ivy.Container(a=a, b=b)

  • >>> filters = ivy.Container(c=c, d=d)

  • >>> y = ivy.Container.static_conv3d_transpose(x, filters, 2, ‘SAME’)

  • >>> print(y.shape)

  • { –

    a: {

    c: [1, 6, 28, 28, 6], d: [1, 6, 28, 28, 6]

    }, b: {

    c: [1, 6, 56, 56, 6], d: [1, 6, 56, 56, 6]

    }

  • }

static _static_depthwise_conv2d(x, filters, strides, padding, /, *, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.depthwise_conv2d. This method simply wraps the function, and so the docstring for ivy.depthwise_conv2d also applies to this method with minimal changes.

Parameters:

  • x (Container) – Input image [batch_size,h,w,d].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in]. (d_in must be the same as d from x)

  • strides (Union[int, Tuple[int], Tuple[int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (Union[str, List[int]]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • data_format (str) – “NHWC” or “NCHW”. Defaults to “NHWC”. (default: 'NHWC')

  • dilations (Union[int, Tuple[int], Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • out (Optional[Container]) – optional output container, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> a = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> b = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> inp = ivy.Container(a=a, b=b)
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3])
>>> y = ivy.Container.static_depthwise_conv2d(
...                                            inp,
...                                            filters,
...                                            strides=2,
...                                            padding='SAME')
>>> print(y.shape)
[1, 64, 64, 3]
static _static_dropout(x, prob, /, *, scale=True, dtype=None, training=True, seed=None, noise_shape=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.dropout. This method simply wraps the function, and so the docstring for ivy.dropout also applies to this method with minimal changes.

Parameters:

  • x (Union[Array, NativeArray, Container]) – The input container x to perform dropout on.

  • prob (float) – The probability of zeroing out each array element, float between 0 and 1.

  • scale (bool) – Whether to scale the output by 1/(1-prob), default is True. (default: True)

  • dtype (Optional[Dtype]) – Output array data type. If dtype is None, the output array data type (default: None) must be inferred from x. Default: None.

  • training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)

  • seed (Optional[int]) – Set a default seed for random number generating (for reproducibility). (default: None) Default is None.

  • noise_shape (Optional[Sequence[int]]) – a sequence representing the shape of the binary dropout mask that will be (default: None) multiplied with the input.

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output container, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result array of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([[1., 2., 3.], [4., 5., 6.]]),
...                   b=ivy.array([7., 8., 9.]))
>>> y = ivy.Container.static_dropout(x, 0.3)
>>> print(y)
{
    a: ivy.array([[0., 0., 4.28571415],
                  [5.71428585, 7.14285755, 0.]]),
    b: ivy.array([0., 11.4285717, 12.8571434])
}
static _static_dropout1d(x, prob, /, *, training=True, data_format='NWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.dropout1d. This method simply wraps the function, and so the docstring for ivy.dropout1d also applies to this method with minimal changes.

Parameters:

  • x (Container) – The input container to perform dropout on.

  • prob (float) – The probability of zeroing out each array element, float between 0 and 1.

  • training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)

  • data_format (str) – “NWC” or “NCW”. Default is "NCW". (default: 'NWC')

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([1, 2, 3]).reshape([1, 1, 3]),
...                   b=ivy.array([4, 5, 6]).reshape([1, 1, 3]))
>>> y = ivy.Container.static_dropout1d(x, 0.5)
>>> print(y)
{
    a: ivy.array([[[0., 4., 0.]]]),
    b: ivy.array([[[0., 0., 12.]]])
}
static _static_dropout3d(x, prob, /, *, training=True, data_format='NDHWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.dropout3d. This method simply wraps the function, and so the docstring for ivy.dropout3d also applies to this method with minimal changes.

Parameters:

  • x (Container) – The input container to perform dropout on.

  • prob (float) – The probability of zeroing out each array element, float between 0 and 1.

  • training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)

  • data_format (str) – “NDHWC” or “NCDHW”. Default is "NDHWC". (default: 'NDHWC')

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

static _static_linear(x, weight, /, *, bias=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.linear. This method simply wraps the function, and so the docstring for ivy.linear also applies to this method with minimal changes.

Parameters:

  • x (Container) – The input x to compute linear transformation on. [outer_batch_shape,inner_batch_shape,in_features]

  • weight (Union[Array, NativeArray]) – The weight matrix. [outer_batch_shape,out_features,in_features]

  • bias (Optional[Union[Array, NativeArray]]) – The bias vector, default is None. [outer_batch_shape,out_features] (default: None)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result array of the linear transformation. [outer_batch_shape,inner_batch_shape,out_features]

Examples

>>> x = ivy.Container(a=ivy.array([[1.1, 2.2, 3.3],                                            [11., 22., 33.]]),                               b=ivy.array([[1.245, 0.278, 4.105],                                            [7., 13., 17.]]))
>>> w = ivy.array([[1., 2., 3.],                            [4., 5., 6.],                            [7., 8., 9.]])
>>> b = ivy.array([1., 0., -1.])
>>> y = ivy.Container.static_linear(x, w, bias=b)
>>> print(y)
{
    a: ivy.array([[16.4, 35.2, 54.],
                  [155., 352., 549.]]),
    b: ivy.array([[15.1, 31., 46.9],
                  [85., 195., 305.]])
}

>>> x = ivy.Container(a=ivy.array([[1.1, 2.2, 3.3],                                            [.0, .1, .2]]),                               b=ivy.array([[1.245, 0.278, 4.105],                                            [.7, .8, .9]]))
>>> w = ivy.Container(a=ivy.array([[1., 2., 3.]]),                               b=ivy.array([[.1, .2, .3]]))
>>> b = ivy.Container(a=ivy.array([1.]), b=ivy.array([-1.]))
>>> y = ivy.Container.static_linear(x, w, bias=b)
>>> print(y)
{
    a: ivy.array([[16.4],
                  [1.8]]),
    b: ivy.array([[0.412],
                  [-0.5]])
}
static _static_lstm_update(x, init_h, init_c, kernel, recurrent_kernel, /, *, bias=None, recurrent_bias=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False)[source]#
Return type:

Tuple[Container, Container]

static _static_multi_head_attention(x, scale, num_heads, /, *, context=None, mask=None, to_q_fn=None, to_kv_fn=None, to_out_fn=None, to_q_v=None, to_kv_v=None, to_out_v=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#
Return type:

Union[Array, NativeArray, Container]

static _static_reduce_window(operand, init_value, computation, window_dimensions, /, *, window_strides=1, padding='VALID', base_dilation=1, window_dilation=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False)[source]#
Return type:

Container

static _static_scaled_dot_product_attention(q, k, v, scale, /, *, mask=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container static method variant of ivy.scaled_dot_product_attention. This method simply wraps the function, and so the docstring for ivy.scaled_dot_product_attention also applies to this method with minimal changes.

Parameters:

  • q (Union[Array, NativeArray, Container]) – The queries input container. The shape of queries input array leaves should be in [batch_shape,num_queries,feat_dim]. The queries input array leaves should have the same size as keys and values.

  • k (Union[Array, NativeArray, Container]) – The keys input array container. The shape of keys input array leaves should be in [batch_shape,num_keys,feat_dim]. The keys input array leaves should have the same size as queries and values.

  • v (Union[Array, NativeArray, Container]) – The values input array container. The shape of values input array leaves should be in [batch_shape,num_keys,feat_dim]. The values input array leaves should have the same size as queries and keys.

  • scale (float) – The scale float value. The scale float value is used to scale the query-key pairs before softmax.

  • mask (Optional[Union[Array, NativeArray, Container]]) – The mask input array/container. The mask to apply to the query-key values. (default: None) Default is None. The shape of mask input array leaves should be in [batch_shape,num_queries,num_keys].

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The output container following applications of scaled dot-product attention. The output array is the weighted sum produced by the attention score and value. The shape of output array is [batch_shape,num_queries,feat_dim] .

Examples

With ivy.Container input:

>>> q = ivy.Container(a=ivy.array([[[0.2, 1.], [2.7, 3.], [4.4, 5.6]]]),
...                   b=ivy.array([[[1.2, 1.], [2.2, 3.], [4.4, 5.6]]]))
>>> k = ivy.Container(a=ivy.array([[[4.2, 1.], [2.2, 3.3],[4.4, 5.6]]]),
...                   b=ivy.array([[[3.2, 1.], [2.2, 3.6], [4.0, 5.6]]]))
>>> v = ivy.Container(a=ivy.array([[[5.2, 1.], [2.1, 3.],[4.4, 5.6]]]),
...                   b=ivy.array([[[0.2, 1.], [2.2, 3.],[4.4, 5.6]]]))
>>> mask =
... ivy.Container(a=ivy.array([[[1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0],
...                             [1.0, 1.0,1.0]]]),
...               b=ivy.array([[[1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0],
...                             [1.0, 1.0,1.0]]]))
>>> result = ivy.Container.static_scaled_dot_product_attention(q,
                                                               k,
                                                               v,
                                                               1,
                                                               mask=mask)
>>> print(result)
{
    a: ivy.array([[[4.27, 5.4],
                [4.4, 5.6],
                [4.4, 5.6]]]),
    b: ivy.array([[[4.35, 5.54],
                [4.4, 5.6],
                [4.4, 5.6]]])
}
conv1d(filters, strides, padding, /, *, data_format='NWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.conv1d. This method simply wraps the function, and so the docstring for ivy.conv1d also applies to this method with minimal changes.

Parameters:

  • self (Container) – Input image [batch_size,w, d_in].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fw,d_in, d_out]. (d_in must be the same as d from x)

  • strides (Union[int, Tuple[int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • data_format (str) – “NWC” or “NCW”. Defaults to “NWC”. (default: 'NWC')

  • dilations (Union[int, Tuple[int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output container, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a=ivy.array([[[2., 3., 4.], [5., 6., 7.]]]),
...                   b=ivy.array([[[7., 8., 9.], [10., 11., 12]]]))
>>> filters = ivy.array([[[0., 0.5, 1.], [0.25, 0.5, 0.75], [-0.5, 0., 0.5 ]]])
>>> result= x.conv1d(filters, (1,), 'VALID')
>>> print(result)
{
    ... a: ivy.array([[[-1.25, 2.5, 6.25],
    ...                [-2., 5.5, 13.]]]),
    ... b: ivy.array([[[-2.5, 7.5, 17.5],
    ...                [-3.25, 10.5, 24.2]]])
}
conv1d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.conv1d_transpose. This method simply wraps the function, and so the docstring for ivy.conv1d_transpose also applies to this method with minimal changes.

Parameters:

  • self (Container) – Input image [batch_size,w,d_in] or [batch_size,d_in,w].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fw,d_in,d_out].

  • strides (int) – The stride of the sliding window for each dimension of input.

  • padding (str) – either the string ‘SAME’ (padding with zeros evenly), the string ‘VALID’ (no padding), or a sequence of n (low, high) integer pairs that give the padding to apply before and after each spatial dimension.

  • output_shape (Optional[Union[Array, NativeArray, Container]]) – Shape of the output (Default value = None) (default: None)

  • data_format (str) – The ordering of the dimensions in the input, one of “NWC” or “NCW”. “NWC” (default: 'NWC') corresponds to input with shape (batch_size, width, channels), while “NCW” corresponds to input with shape (batch_size, channels, width).

  • dilations (int) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Union[Array, Container]]) – optional output container, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Union[Array, NativeArray, Container]

Returns:

ret – The result of the transpose convolution operation.

Examples

>>> x = ivy.Container(a=ivy.random_normal(mean=0, std=1, shape=[1, 28, 3]),
...                   b=ivy.random_normal(mean=0, std=1, shape=[1, 56, 3]))
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 6])
>>> y = x.conv1d_transpose(filters, 2, 'SAME')
>>> print(y.shape)
{
    a: [1,56,6],
    b: [1,112,6]
}
conv2d(filters, strides, padding, /, *, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.conv2d. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

  • self (Container) – Input image [batch_size,h,w,d_in].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in,d_out].

  • strides (Union[int, Tuple[int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • data_format (str) – “NHWC” or “NCHW”. Defaults to “NHWC”. (default: 'NHWC')

  • dilations (Union[int, Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a = ivy.eye(3, 3).reshape((1, 3, 3, 1)),
...                   b = ivy.eye(5, 5).reshape((1, 5, 5, 1)))
>>> filters = ivy.array([[2, 0, 1],
...                      [1, 3, 1],
...                      [0, 1, 1]], dtype=ivy.float32).reshape((3, 3, 1, 1))
>>> result = x.conv2d(filters, 2, 'SAME')
>>> print(result)
{
    a:ivy.array([[[[4.],[0.]],[[1.],[5.]]]]),
    b:ivy.array([[[[4.],[0.],[0.]],[[1.],[6.],[0.]],[[0.],[1.],[5.]]]])
}
conv2d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.conv2d_transpose. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

  • self (Container) – Input image [batch_size,h,w,d_in].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in,d_out].

  • strides (Union[int, Tuple[int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • output_shape (Optional[Union[Array, NativeArray, Container]]) – Shape of the output (Default value = None) (default: None)

  • data_format (str) – “NHWC” or “NCHW”. Defaults to “NHWC”. (default: 'NHWC')

  • dilations (Union[int, Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> a = ivy.random_normal(mean=0, std=1, shape=[1, 14, 14, 3])
>>> b = ivy.random_normal(mean=0, std=1, shape=[1, 28, 28, 3])
>>> c = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 6])
>>> d = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 6])
>>> x = ivy.Container(a=a, b=b)
>>> filters = ivy.Container(c=c, d=d)
>>> y = x.conv2d_transpose(x, filters, 2, 'SAME')
>>> print(y.shape)
{
    a: {
        c: [1,28,28,6],
        d: [1,28,28,6]
    },
    b: {
        c: [1,56,56,6],
        d: [1,56,56,6]
    }
}
conv3d(filters, strides, padding, /, *, data_format='NDHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.conv3d. This method simply wraps the function, and so the docstring for ivy.conv3d also applies to this method with minimal changes.

Parameters:

  • x – Input volume [batch_size,d,h,w,d_in].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fdfh,fw,d_in,d_out].

  • strides (Union[int, Tuple[int, int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • data_format (str) – “NDHWC” or “NCDHW”. Defaults to “NDHWC”. (default: 'NDHWC')

  • dilations (Union[int, Tuple[int, int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.Container(a = ivy.full((1, 2, 3, 3, 1),0.5),                              b = ivy.full((1, 2, 5, 5, 1),1.))

>>> filters = ivy.ones((3, 3, 3, 1, 1))

>>> result = x.conv3d(filters, 2, 'SAME')
>>> print(result)
{
    a: ivy.array([[[[[4.],[4.]],[[4.],[4.]]]]]),
    b: ivy.array([[[[[8.],[12.],[8.]],[[12.],[18.],[12.]],[[8.],[12.],[8.]]]]])
}
conv3d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NDHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.conv3d_transpose. This method simply wraps the function, and so the docstring for ivy.conv3d_transpose also applies to this method with minimal changes.

Parameters:

  • self (Container) – Input container with leaves of volume [batch_size,d,h,w,d_in] or [batch_size,d_in,d,h,w].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fd,fh,fw,d_in,d_out].

  • strides (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (Union[str, List[int]]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • output_shape (Optional[Union[Array, NativeArray, Container]]) – Shape of the output (Default value = None) (default: None)

  • data_format (str) –

    The ordering of the dimensions in the input, one of “NDHWC” or (default: 'NDHWC') “NCDHW”. “NDHWC” corresponds to inputs with shape (batch_size,

    depth, height, width, channels), while “NCDHW” corresponds to input with shape (batch_size, channels, depth, height, width).

  • dilations (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • out (Optional[Container]) – optional output container, for writing the result to. It must (default: None) have a shape that the inputs broadcast to.

Return type:

Container

Returns:

  • ret – The result of the transpose convolution operation in a container.

  • >>> a = ivy.random_normal(mean=0, std=1, shape=[1, 3, 14, 14, 3])

  • >>> b = ivy.random_normal(mean=0, std=1, shape=[1, 3, 28, 28, 3]))

  • >>> c = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 3, 6])

  • >>> d = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 3, 6]))

  • >>> x = ivy.Container(a=a, b=b)

  • >>> filters = ivy.Container(c=c, d=d)

  • >>> y = x.conv3d_transpose(filters, 2, ‘SAME’)

  • >>> print(y.shape)

  • { –

    a: {

    c: [1, 6, 28, 28, 6], d: [1, 6, 28, 28, 6]

    }, b: {

    c: [1, 6, 56, 56, 6], d: [1, 6, 56, 56, 6]

    }

  • }

depthwise_conv2d(filters, strides, padding, /, *, data_format='NHWC', dilations=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.depthwise_conv2d. This method simply wraps the function, and so the docstring for ivy.depthwise_conv2d also applies to this method with minimal changes.

Parameters:

  • self (Container) – Input image [batch_size,h,w,d].

  • filters (Union[Array, NativeArray, Container]) – Convolution filters [fh,fw,d_in]. (d_in must be the same as d from self)

  • strides (Union[int, Tuple[int], Tuple[int, int]]) – The stride of the sliding window for each dimension of input.

  • padding (Union[str, List[int]]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.

  • data_format (str) – “NHWC” or “NCHW”. Defaults to “NHWC”. (default: 'NHWC')

  • dilations (Union[int, Tuple[int], Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)

  • out (Optional[Container]) – optional output container, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The result of the convolution operation.

Examples

>>> a = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> b = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> inp = ivy.Container(a=a, b=b)
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3])
>>> y = inp.depthwise_conv2d(filters, 2, 'SAME')
>>> print(y.shape)
[1, 64, 64, 3]
dropout(prob, /, *, scale=True, dtype=None, training=True, seed=None, noise_shape=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.dropout. This method simply wraps the function, and so the docstring for ivy.dropout also applies to this method with minimal changes.

Parameters:

  • self (Container) – The input container to perform dropout on.

  • prob (float) – The probability of zeroing out each array element, float between 0 and 1.

  • scale (bool) – Whether to scale the output by 1/(1-prob), default is True. (default: True)

  • dtype (Optional[Dtype]) – output array data type. If dtype is None, the output array data type (default: None) must be inferred from x. Default: None.

  • training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)

  • seed (Optional[int]) – Set a default seed for random number generating (for reproducibility). (default: None) Default is None.

  • noise_shape (Optional[Sequence[int]]) – a sequence representing the shape of the binary dropout mask that will be (default: None) multiplied with the input.

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result array of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([[1., 2., 3.], [4., 5., 6.]]),
...                   b=ivy.array([7., 8., 9.]))
>>> y = x.dropout(0.3)
>>> print(y)
{
    a: ivy.array([[0., 0., 4.28571415],
                  [5.71428585, 7.14285755, 0.]]),
    b: ivy.array([0., 11.4285717, 12.8571434])
}
dropout1d(prob, /, *, training=True, data_format='NWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.dropout1d. This method simply wraps the function, and so the docstring for ivy.dropout1d also applies to this method with minimal changes.

Parameters:

  • self (Container) – The input container to perform dropout on.

  • prob (float) – The probability of zeroing out each array element, float between 0 and 1.

  • training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)

  • data_format (str) – “NWC” or “NCW”. Default is "NCW". (default: 'NWC')

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

Examples

>>> x = ivy.Container(a=ivy.array([1, 2, 3]).reshape([1, 1, 3]),
...                   b=ivy.array([4, 5, 6]).reshape([1, 1, 3]))
>>> y = x.dropout1d(x, 0.5)
>>> print(y)
{
    a: ivy.array([[[0., 4., 0.]]]),
    b: ivy.array([[[0., 0., 12.]]])
}
dropout3d(prob, /, *, training=True, data_format='NDHWC', key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.dropout3d. This method simply wraps the function, and so the docstring for ivy.dropout3d also applies to this method with minimal changes.

Parameters:

  • self (Container) – The input container to perform dropout on.

  • prob (float) – The probability of zeroing out each array element, float between 0 and 1.

  • training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)

  • data_format (str) – “NDHWC” or “NCDHW”. Default is "NDHWC". (default: 'NDHWC')

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result container of the output after dropout is performed.

linear(weight, /, *, bias=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.linear. This method simply wraps the function, and so the docstring for ivy.linear also applies to this method with minimal changes.

Parameters:

  • self (Container) – The input container to compute linear transformation on. [outer_batch_shape,inner_batch_shape,in_features]

  • weight (Union[Array, NativeArray]) – The weight matrix. [outer_batch_shape,out_features,in_features]

  • bias (Optional[Union[Array, NativeArray]]) – The bias vector, default is None. [outer_batch_shape,out_features] (default: None)

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – Result array of the linear transformation. [outer_batch_shape,inner_batch_shape,out_features]

Examples

>>> x = ivy.Container(a=ivy.array([[1.1, 2.2, 3.3],                                            [11., 22., 33.]]),                               b=ivy.array([[1.245, 0.278, 4.105],                                            [7., 13., 17.]]))
>>> w = ivy.array([[1., 2., 3.],                            [4., 5., 6.],                            [7., 8., 9.]])
>>> b = ivy.array([1, 0, -1])
>>> y = x.linear(w, bias=b)
>>> print(y)
{
    a: ivy.array([[16.4, 35.2, 54.],                           [155., 352., 549.]]),             b: ivy.array([[15.1, 31., 46.9],                           [85., 195., 305.]])
}
lstm_update(init_h, init_c, kernel, recurrent_kernel, /, *, bias=None, recurrent_bias=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False)[source]#

ivy.Container instance method variant of ivy.lstm_update. This method simply wraps the function, and so the docstring for ivy.lstm_update also applies to this method with minimal changes.

Parameters:

  • init_h (Union[Array, NativeArray, Container]) – initial state tensor for the cell output [batch_shape, out].

  • init_c (Union[Array, NativeArray, Container]) – initial state tensor for the cell hidden state [batch_shape, out].

  • kernel (Union[Array, NativeArray, Container]) – weights for cell kernel [in, 4 x out].

  • recurrent_kernel (Union[Array, NativeArray, Container]) – weights for cell recurrent kernel [out, 4 x out].

  • bias (Optional[Union[Array, NativeArray, Container]]) – bias for cell kernel [4 x out]. (Default value = None) (default: None)

  • recurrent_bias (Optional[Union[Array, NativeArray, Container]]) – bias for cell recurrent kernel [4 x out]. (Default value = None) (default: None)

Return type:

Tuple[Container, Container]

Returns:

ret – hidden state for all timesteps [batch_shape,t,out] and cell state for last timestep [batch_shape,out]

Examples

>>> x = ivy.Container(
...     a=ivy.random_normal(shape=(5, 20, 3)),
...     b=ivy.random_normal(shape=(5, 20, 3))
... )
>>> h_i = ivy.random_normal(shape=(5, 6))
>>> c_i = ivy.random_normal(shape=(5, 6))

>>> kernel = ivy.random_normal(shape=(3, 4 * 6))
>>> rc = ivy.random_normal(shape=(6, 4 * 6))
>>> x.lstm_update(h_i, c_i, kernel, rc)
{
    a: (tuple(2), <class ivy.array.array.Array>, shape=[5, 20, 6]),
    b: (tuple(2), <class ivy.array.array.Array>, shape=[5, 20, 6])
}
multi_head_attention(scale, num_heads, /, *, context=None, mask=None, to_q_fn=None, to_kv_fn=None, to_out_fn=None, to_q_v=None, to_kv_v=None, to_out_v=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#
Return type:

Union[Array, NativeArray, Container]

reduce_window(init_value, computation, window_dimensions, /, *, window_strides=1, padding='VALID', base_dilation=1, window_dilation=1, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False)[source]#

ivy.Container instance method variant of ivy.reduce_window. This method simply wraps the function, and so the docstring for ivy.reduce_window also applies to this method with minimal changes.

Parameters:

  • self (Container) – A container representing the base areas on which the window is going to slide over.

  • init_value (Union[int, float]) – The starting value for the reduction.

  • computation (Callable) – The reduction function to apply to elements in each window.

  • window_dimensions (Union[int, Sequence[int]]) – A sequence containing the window dimensions.

  • window_strides (Union[int, Sequence[int]]) – A sequence containing the window strides. (default: 1)

  • padding (Union[str, int, Sequence[Tuple[int, int]]]) – Either the string ‘SAME’ (padding with zeros evenly), the string ‘VALID’ (no (default: 'VALID') padding), or a sequence of n (low, high) integer pairs that give the padding to apply before and after each spatial dimension.

  • base_dilation (Union[int, Sequence[int]]) – A sequence containing the base dilation values. (default: 1)

  • window_dilation (Union[int, Sequence[int]]) – A sequence containing the window dilation values. (default: 1)

Return type:

Container

Returns:

ret – The result of the pooling-like operation.

Examples

>>> x = ivy.Container(
...     a=ivy.array([[1, 2, 3, 4],
...                  [5, 6, 7, 8],
...                  [9, 10, 11, 12]]),
...     b=ivy.array([[13, 14, 15, 16],
...                  [17, 18, 19, 20],
...                  [21, 22, 23, 24]])
... )
>>> x.reduce_window(0, ivy.sum, (2, 2))
{
    a: ivy.array([[21 25 29]
                  [33 37 41]
                  [45 49 53]]),
    b: ivy.array([[63 67 71]
                  [75 79 83]
                  [87 91 95]])
}
scaled_dot_product_attention(k, v, scale, /, *, mask=None, key_chains=None, to_apply=True, prune_unapplied=False, map_sequences=False, out=None)[source]#

ivy.Container instance method variant of ivy.scaled_dot_product_attention. This method simply wraps the function, and so the docstring for ivy.scaled_dot_product_attention also applies to this method with minimal changes.

Parameters:

  • self (Container) – The queries input container. The shape of queries input array leaves should be in [batch_shape,num_queries,feat_dim]. The queries input array leaves should have the same size as keys and values.

  • k (Union[Array, NativeArray, Container]) – The keys input array container. The shape of keys input array leaves should be in [batch_shape,num_keys,feat_dim]. The keys input array leaves should have the same size as queries and values.

  • v (Union[Array, NativeArray, Container]) – The values input array container. The shape of values input array leaves should be in [batch_shape,num_keys,feat_dim]. The values input array leaves should have the same size as queries and keys.

  • scale (float) – The scale float value. The scale float value is used to scale the query-key pairs before softmax.

  • mask (Optional[Union[Array, NativeArray, Container]]) – The mask input array/container. The mask to apply to the query-key values. (default: None) Default is None. The shape of mask input array leaves should be in [batch_shape,num_queries,num_keys].

  • key_chains (Optional[Union[List[str], Dict[str, str]]]) – The key-chains to apply or not apply the method to. Default is None. (default: None)

  • to_apply (bool) – If True, the method will be applied to key_chains, otherwise key_chains (default: True) will be skipped. Default is True.

  • prune_unapplied (bool) – Whether to prune key_chains for which the function was not applied. (default: False) Default is False.

  • map_sequences (bool) – Whether to also map method to sequences (lists, tuples). (default: False) Default is False.

  • out (Optional[Container]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Container

Returns:

ret – The output container following applications of scaled dot-product attention. The output array is the weighted sum produced by the attention score and value. The shape of output array is [batch_shape,num_queries,feat_dim] .

Examples

With ivy.Container input:

>>> q = ivy.Container(a=ivy.array([[[0.2, 1.], [2.7, 3.], [4.4, 5.6]]]),
...                   b=ivy.array([[[1.2, 1.], [2.2, 3.], [4.4, 5.6]]]))
>>> k = ivy.Container(a=ivy.array([[[4.2, 1.], [2.2, 3.3],[4.4, 5.6]]]),
...                   b=ivy.array([[[3.2, 1.], [2.2, 3.6], [4.0, 5.6]]]))
>>> v = ivy.Container(a=ivy.array([[[5.2, 1.], [2.1, 3.],[4.4, 5.6]]]),
...                   b=ivy.array([[[0.2, 1.], [2.2, 3.],[4.4, 5.6]]]))
>>> mask =
... ivy.Container(a=ivy.array([[[1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0],
...                             [1.0, 1.0,1.0]]]),
...               b=ivy.array([[[1.0, 1.0, 1.0],
...                             [1.0, 1.0, 1.0],
...                             [1.0, 1.0,1.0]]]))
>>> result = q.scaled_dot_product_attention(k,
                                            v,
                                            1,
                                            mask=mask)
>>> print(result)
{
    a: ivy.array([[[4.27, 5.4],
                [4.4, 5.6],
                [4.4, 5.6]]]),
    b: ivy.array([[[4.35, 5.54],
                [4.4, 5.6],
                [4.4, 5.6]]])
}