Layers#

class ivy.data_classes.array.layers._ArrayWithLayers[source]#

Bases: ABC

_abc_impl = <_abc_data object>#

conv1d(filters, strides, padding, /, *, data_format='NWC', dilations=1, out=None)[source]#

ivy.Array instance method variant of ivy.conv1d. This method simply wraps the function, and so the docstring for ivy.conv1d also applies to this method with minimal changes.

Parameters:

self (Array) – Input image [batch_size,w,d_in] or [batch_size,d_in,w].
filters (Union[Array, NativeArray]) – Convolution filters [fw,d_in,d_out].
strides (Union[int, Tuple[int]]) – The stride of the sliding window for each dimension of input.
padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str) – “NWC” or “NCW”. Defaults to “NWC”. (default: 'NWC')
dilations (Union[int, Tuple[int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)
out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that (default: None) the inputs broadcast to.

Return type:

Array

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.array([[[1., 2.], [3., 4.], [6., 7.], [9., 11.]]])  # NWC
>>> filters = ivy.array([[[0., 1.], [1., 1.]]])  # WIO (I == C)
>>> result = x.conv1d(filters, (1,), 'VALID')
>>> print(result)
ivy.array([[[ 2.,  3.],
...         [ 4.,  7.],
...         [ 7., 13.],
...         [11., 20.]]])

conv1d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NWC', dilations=1, out=None)[source]#

ivy.Array instance method variant of ivy.conv1d_transpose. This method simply wraps the function, and so the docstring for ivy.conv1d_transpose also applies to this method with minimal changes.

Parameters:

self (Array) – Input image [batch_size,w,d_in] or [batch_size,d_in,w].
filters (Union[Array, NativeArray]) – Convolution filters [fw,d_in,d_out].
strides (Union[int, Tuple[int]]) – The stride of the sliding window for each dimension of input.
padding (str) – either the string ‘SAME’ (padding with zeros evenly), the string ‘VALID’ (no padding), or a sequence of n (low, high) integer pairs that give the padding to apply before and after each spatial dimension.
output_shape (Optional[Union[Shape, NativeShape]]) – Shape of the output (Default value = None) (default: None)
data_format (str) – The ordering of the dimensions in the input, one of “NWC” or “NCW”. “NWC” (default: 'NWC') corresponds to input with shape (batch_size, width, channels), while “NCW” corresponds to input with shape (batch_size, channels, width).
dilations (Union[int, Tuple[int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)
out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that (default: None) the inputs broadcast to.

Return type:

Array

Returns:

ret – The result of the transpose convolution operation.

Examples

>>> x = ivy.array([[[1., 2.], [3., 4.], [6., 7.], [9., 11.]]])  # NWC
>>> filters = ivy.array([[[0., 1.], [1., 1.]]])  # WIO (I == C)
>>> result = x.conv1d_transpose(filters, (1,), 'VALID')
>>> print(result)
ivy.array([[[ 2.,  3.],
...         [ 4.,  7.],
...         [ 7., 13.],
...         [11., 20.]]])

conv2d(filters, strides, padding, /, *, data_format='NHWC', dilations=1, out=None)[source]#

ivy.Array instance method variant of ivy.conv2d. This method simply wraps the function, and so the docstring for ivy.conv2d also applies to this method with minimal changes.

Parameters:

self (Array) – Input image [batch_size,h,w,d_in] or [batch_size,d_in,h,w].
filters (Union[Array, NativeArray]) – Convolution filters [fh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int]]) – The stride of the sliding window for each dimension of input.
padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str) – “NHWC” or “NCHW”. Defaults to “NHWC”. (default: 'NHWC')
dilations (Union[int, Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)
out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that (default: None) the inputs broadcast to.

Return type:

Array

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.array([[[[1.], [2.0],[3.]],
...                 [[1.], [2.0],[3.]],
...                 [[1.], [2.0],[3.]]]]) #NHWC
>>> filters = ivy.array([[[[0.]], [[1.]], [[0.]]],
...                      [[[0.]], [[1.]], [[0.]]],
...                      [[[0.]], [[1.]], [[0.]]]]) #HWIO
>>> result = x.conv2d(filters, 1, 'SAME', data_format='NHWC',
...    dilations= 1)
>>> print(result)
ivy.array([[
          [[2.],[4.],[6.]],
          [[3.],[6.],[9.]],
          [[2.],[4.],[6.]]
          ]])

conv2d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NHWC', dilations=1, out=None)[source]#

ivy.Array instance method variant of ivy.conv2d_transpose. This method simply wraps the function, and so the docstring for ivy.conv2d_transpose also applies to this method with minimal changes.

Parameters:

self (Array) – Input image [batch_size,h,w,d_in] or [batch_size,d_in,h,w].
filters (Union[Array, NativeArray]) – Convolution filters [fh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int]]) – The stride of the sliding window for each dimension of input.
padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
output_shape (Optional[Union[Shape, NativeShape]]) – Shape of the output (Default value = None) (default: None)
data_format (str) – The ordering of the dimensions in the input, one of “NHWC” or “NCHW”. “NHWC” (default: 'NHWC') corresponds to inputs with shape (batch_size, height, width, channels), while “NCHW” corresponds to input with shape (batch_size, channels, height, width). Default is "NHWC".
dilations (Union[int, Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)
out (Optional[Array]) – Optional output array, for writing the result to. It must have a shape that (default: None) the inputs broadcast to.

Return type:

Array

Returns:

ret – The result of the transpose convolution operation.

Examples

>>> x = ivy.random_normal(mean=0, std=1, shape=[1, 28, 28, 3])
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 6])
>>> y = x.conv2d_transpose(filters, 2, 'SAME')
>>> print(y.shape)
(1, 56, 56, 6)

conv3d(filters, strides, padding, /, *, data_format='NDHWC', dilations=1, out=None)[source]#

ivy.Array instance method variant of ivy.conv3d. This method simply wraps the function, and so the docstring for ivy.conv3d also applies to this method with minimal changes.

Parameters:

x – Input volume [batch_size,d,h,w,d_in].
filters (Union[Array, NativeArray]) – Convolution filters [fd,fh,fw,d_in,d_out].
strides (Union[int, Tuple[int, int, int]]) – The stride of the sliding window for each dimension of input.
padding (str) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str) – “NDHWC” or “NCDHW”. Defaults to “NDHWC”. (default: 'NDHWC')
dilations (Union[int, Tuple[int, int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)
out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that (default: None) the inputs broadcast to.

Return type:

Array

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.ones((1, 3, 3, 3, 1)).astype(ivy.float32)

>>> filters = ivy.ones((1, 3, 3, 1, 1)).astype(ivy.float32)

>>> result = x.conv3d(filters, 2, 'SAME')
>>> print(result)
ivy.array([[[[[4.],[4.]],[[4.],[4.]]],[[[4.],[4.]],[[4.],[4.]]]]])

conv3d_transpose(filters, strides, padding, /, *, output_shape=None, data_format='NDHWC', dilations=1, out=None)[source]#

ivy.Array instance method variant of ivy.conv3d_transpose. This method simply wraps the function, and so the docstring for ivy.conv3d_transpose also applies to this method with minimal changes.

Parameters:

self (Array) – Input volume [batch_size,d,h,w,d_in] or [batch_size,d_in,d,h,w].
filters (Union[Array, NativeArray]) – Convolution filters [fd,fh,fw,d_in,d_out].
strides (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int]]) – The stride of the sliding window for each dimension of input.
padding (Union[str, List[int]]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
output_shape (Optional[Union[Shape, NativeShape]]) – Shape of the output (Default value = None) (default: None)
data_format (str) –
The ordering of the dimensions in the input, one of “NDHWC” or (default: 'NDHWC') “NCDHW”. “NDHWC” corresponds to inputs with shape (batch_size,

depth, height, width, channels), while “NCDHW” corresponds to input with shape (batch_size, channels, depth, height, width).
dilations (Union[int, Tuple[int], Tuple[int, int], Tuple[int, int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)
out (Optional[Array]) – optional output array, for writing the result to. It must have a (default: None) shape that the inputs broadcast to.

Return type:

Array

Returns:

ret – The result of the transpose convolution operation.

Examples

>>> x = ivy.random_normal(mean=0, std=1, shape=[1, 3, 28, 28, 3])
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3, 3, 6])
>>> y = x.conv3d_transpose(filters, 2, 'SAME')
>>> print(y.shape)
(1, 6, 56, 56, 6)

depthwise_conv2d(filters, strides, padding, /, *, data_format='NHWC', dilations=1, out=None)[source]#

ivy.Array instance method variant of ivy.depthwise_conv2d. This method simply wraps the function, and so the docstring for ivy.depthwise_conv2d also applies to this method with minimal changes.

Parameters:

self (Array) – Input image [batch_size,h,w,d].
filters (Union[Array, NativeArray]) – Convolution filters [fh,fw,d_in]. (d_in must be the same as d from self)
strides (Union[int, Tuple[int], Tuple[int, int]]) – The stride of the sliding window for each dimension of input.
padding (Union[str, List[int]]) – “SAME” or “VALID” indicating the algorithm, or list indicating the per-dimension paddings.
data_format (str) – “NHWC” or “NCHW”. Defaults to “NHWC”. (default: 'NHWC')
dilations (Union[int, Tuple[int], Tuple[int, int]]) – The dilation factor for each dimension of input. (Default value = 1) (default: 1)
out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that (default: None) the inputs broadcast to.

Return type:

Array

Returns:

ret – The result of the convolution operation.

Examples

>>> x = ivy.randint(0, 255, shape=(1, 128, 128, 3)).astype(ivy.float32) / 255.0
>>> filters = ivy.random_normal(mean=0, std=1, shape=[3, 3, 3])
>>> y = x.depthwise_conv2d(filters, 2, 'SAME')
>>> print(y.shape)
(1, 64, 64, 3)

dropout(prob, /, *, scale=True, dtype=None, training=True, seed=None, noise_shape=None, out=None)[source]#

ivy.Array instance method variant of ivy.dropout. This method simply wraps the function, and so the docstring for ivy.droput also applies to this method with minimal changes.

Parameters:

self (Array) – The input array x to perform dropout on.
prob (float) – The probability of zeroing out each array element, float between 0 and 1.
scale (bool) – Whether to scale the output by 1/(1-prob), default is True. (default: True)
dtype (Optional[Union[Dtype, NativeDtype]]) – output array data type. If dtype is None, the output array data type (default: None) must be inferred from x. Default: None.
training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)
seed (Optional[int]) – Set a default seed for random number generating (for (default: None) reproducibility).Default is None.
noise_shape (Optional[Sequence[int]]) – a sequence representing the shape of the binary dropout mask that will be (default: None) multiplied with the input.
out (Optional[Array]) – optional output array, for writing the result to. It must have (default: None) a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret – Result array of the output after dropout is performed.

Examples

With ivy.Array instances:

>>> x = ivy.array([[1., 2., 3.],
...                [4., 5., 6.],
...                [7., 8., 9.],
...                [10., 11., 12.]])
>>> y = x.dropout(0.3)
>>> print(y)
ivy.array([[ 1.42857146,  2.85714293,  4.28571415],
           [ 5.71428585,  7.14285755,  8.5714283 ],
           [ 0.        , 11.4285717 , 12.8571434 ],
           [14.2857151 ,  0.        ,  0.        ]])

>>> x = ivy.array([[1., 2., 3.],
...                [4., 5., 6.],
...                [7., 8., 9.],
...                [10., 11., 12.]])
>>> y = x.dropout(0.3, scale=False)
>>> print(y)
ivy.array([[ 1.,  2., 3.],
           [ 4.,  5., 0.],
           [ 7.,  0., 9.],
           [10., 11., 0.]])

dropout1d(prob, /, *, training=True, data_format='NWC', out=None)[source]#

ivy.Array instance method variant of ivy.dropout1d. This method simply wraps the function, and so the docstring for ivy.droput1d also applies to this method with minimal changes.

Parameters:

self (Array) – The input array x to perform dropout on.
prob (float) – The probability of zeroing out each array element, float between 0 and 1.
training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)
data_format (str) – “NWC” or “NCW”. Default is "NCW". (default: 'NWC')
out (Optional[Array]) – optional output array, for writing the result to. It must have (default: None) a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret – Result array of the output after dropout is performed.

Examples

>>> x = ivy.array([1, 1, 1]).reshape([1, 1, 3])
>>> y = x.dropout1d(0.5)
>>> print(y)
ivy.array([[[2., 0, 2.]]])

dropout3d(prob, /, *, training=True, data_format='NDHWC', out=None)[source]#

ivy.Array instance method variant of ivy.dropout3d. This method simply wraps the function, and so the docstring for ivy.droput3d also applies to this method with minimal changes.

Parameters:

self (Array) – The input array x to perform dropout on.
prob (float) – The probability of zeroing out each array element, float between 0 and 1.
training (bool) – Turn on dropout if training, turn off otherwise. Default is True. (default: True)
data_format (str) – “NDHWC” or “NCDHW”. Default is "NDHWC". (default: 'NDHWC')
out (Optional[Array]) – optional output array, for writing the result to. It must have (default: None) a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret – Result array of the output after dropout is performed.

linear(weight, /, *, bias=None, out=None)[source]#

ivy.Array instance method variant of ivy.linear. This method simply wraps the function, and so the docstring for ivy.linear also applies to this method with minimal changes.

Parameters:

self (Array) – The input array to compute linear transformation on. [outer_batch_shape,inner_batch_shape,in_features]
weight (Union[Array, NativeArray]) – The weight matrix. [outer_batch_shape,out_features,in_features]
bias (Optional[Union[Array, NativeArray]]) – The bias vector, default is None. [outer_batch_shape,out_features] (default: None)
out (Optional[Array]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Array

Returns:

ret – Result array of the linear transformation. [outer_batch_shape,inner_batch_shape,out_features]

Examples

>>> x = ivy.array([[1.1, 2.2, 3.3],                            [4.4, 5.5, 6.6],                            [7.7, 8.8, 9.9]])
>>> w = ivy.array([[1., 2., 3.],                            [4., 5., 6.],                            [7., 8., 9.]])
>>> b = ivy.array([1., 0., -1.])
>>> y = x.linear(w, bias=b)
>>> print(y)
ivy.array([[ 16.4,  35.2,  54. ],
           [ 36.2,  84.7, 133. ],
           [ 56. , 134. , 212. ]])

lstm_update(init_h, init_c, kernel, recurrent_kernel, /, *, bias=None, recurrent_bias=None)[source]#

ivy.Array instance method variant of ivy.lstm_update. This method simply wraps the function, and so the docstring for ivy.lstm_update also applies to this method with minimal changes.

Parameters:

init_h (Union[Array, NativeArray]) – initial state tensor for the cell output [batch_shape, out].
init_c (Union[Array, NativeArray]) – initial state tensor for the cell hidden state [batch_shape, out].
kernel (Union[Array, NativeArray]) – weights for cell kernel [in, 4 x out].
recurrent_kernel (Union[Array, NativeArray]) – weights for cell recurrent kernel [out, 4 x out].
bias (Optional[Union[Array, NativeArray]]) – bias for cell kernel [4 x out]. (Default value = None) (default: None)
recurrent_bias (Optional[Union[Array, NativeArray]]) – bias for cell recurrent kernel [4 x out]. (Default value = None) (default: None)

Return type:

Tuple[Array, Array]

Returns:

ret – hidden state for all timesteps [batch_shape,t,out] and cell state for last timestep [batch_shape,out]

Examples

>>> x = ivy.randint(0, 20, shape=(6, 20, 3))
>>> h_i = ivy.random_normal(shape=(6, 5))
>>> c_i = ivy.random_normal(shape=(6, 5))
>>> kernel = ivy.random_normal(shape=(3, 4 * 5))
>>> rc = ivy.random_normal(shape=(5, 4 * 5))
>>> result = x.lstm_update(h_i, c_i, kernel, rc)

>>> result[0].shape
(6, 20, 5)
>>> result[1].shape
(6, 5)

multi_head_attention(scale, num_heads, /, *, context=None, mask=None, to_q_fn=None, to_kv_fn=None, to_out_fn=None, to_q_v=None, to_kv_v=None, to_out_v=None, out=None)[source]#

Return type:: Array

scaled_dot_product_attention(k, v, scale, /, *, mask=None, out=None)[source]#

ivy.Array instance method variant of ivy.scaled_dot_product_attention. This method simply wraps the function, and so the docstring for ivy.scaled_dot_product_attention also applies to this method with minimal changes.

Parameters:

self (Array) – The queries input array. The shape of queries input array should be in [batch_shape,num_queries,feat_dim]. The queries input array should have the same size as keys and values.
k (Union[Array, NativeArray]) – The keys input array. The shape of keys input array should be in [batch_shape,num_keys,feat_dim]. The keys input array should have the same size as queries and values.
v (Union[Array, NativeArray]) – The values input array. The shape of values input should be in [batch_shape,num_keys,feat_dim]. The values input array should have the same size as queries and keys.
scale (float) – The scale float value. The scale float value is used to scale the query-key pairs before softmax.
mask (Optional[Union[Array, NativeArray]]) – The mask input array. The mask to apply to the query-key values. (default: None) Default is None. The shape of mask input should be in [batch_shape,num_queries,num_keys].
out (Optional[Array]) – optional output array, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Return type:

Array

Returns:

ret – The output following application of scaled dot-product attention. The output array is the weighted sum produced by the attention score and value. The shape of output array is [batch_shape,num_queries,feat_dim] .

Examples

With ivy.Array input:

>>> q = ivy.array([[[0.2, 1.], [2.2, 3.], [4.4, 5.6]]])
>>> k = ivy.array([[[0.6, 1.5], [2.4, 3.3], [4.2, 5.1]]])
>>> v = ivy.array([[[0.4, 1.3], [2.2, 3.1], [4.3, 5.3]]])
>>> mask = ivy.array([[[0.0, 0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0, 0.0]]])
>>> result = q.scaled_dot_product_attention(k, v, 1, mask=mask)
>>> print(result)
ivy.array([[[2.3, 3.23],[2.3, 3.23],[2.3, 3.23]]])

>>> q = ivy.array([[[0.2, 1.], [2.2, 3.], [4.4, 5.6]]])
>>> k = ivy.array([[[0.6, 1.5], [2.4, 3.3], [4.2, 5.1]]])
>>> v = ivy.array([[[0.4, 1.3], [2.2, 3.1], [4.3, 5.3]]])
>>> mask = ivy.array([[[0.0, 0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0, 0.0]]])
>>> out = ivy.zeros(shape=(1, 3, 2))
>>> q.scaled_dot_product_attention(k, v, 1, mask=mask, out=out)
>>> print(out)
ivy.array([[[2.3, 3.23],[2.3, 3.23],[2.3, 3.23]]])