# Losses#

ivy.huber_loss(true, pred, /, *, delta=1.0, reduction='mean', out=None)[source]#

Compute the Huber loss (smooth L1 loss) between true and predicted values.

Parameters:
• true (array_like) – The true (ground truth) values.

• pred (array_like) – The predicted values by the model.

• delta (float, optional) – The threshold parameter that determines the point where the loss transitions fro -m squared error to absolute error. Default is 1.0.

• reduction (str, optional) – The type of reduction to apply to the loss. Possible values are “mean” (default) and “sum”.

• out (array_like, optional) – Optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret (array_like) – The Huber loss between the true and predicted values.

Examples

>>> true = ivy.array([2, 4, 7, 1])
>>> pred = ivy.array([2.5, 3.5, 8, 0.8])
>>> huber_loss(true, pred, delta=1.0)
ivy.array([0.125, 0.125, 0.5  , 0.125])

>>> huber_loss(true, pred, delta=2.0)
ivy.array([0.125, 0.125, 0.5  , 0.2  ])

>>> huber_loss(true, pred, delta=0.5)
ivy.array([0.25 , 0.25 , 0.   , 0.125])

ivy.kl_div(input, target, /, *, reduction='mean', log_target=False, out=None)[source]#

Compute the Kullback-Leibler divergence loss between two input tensors (conventionally, probability distributions).

Parameters:
• input (array_like) – Tensor of arbitrary shape in log-probabilities

• target (array_like) – Tensor of the same shape as input. See log_target for the target’s interpretation

• reduction ({'mean', 'sum', 'batchmean', 'none'}, optional) – Type of reduction to apply to the output. Default is ‘mean’.

• log_target (bool) – A flag indicating whether target is passed in the log space. It is recommended to pass certain distributions (like softmax) in the log space to avoid numerical issues caused by explicit log. Default: False

Return type:

Array

Returns:

ret (array) – The Kullback-Leibler divergence loss between the two input tensors.

Examples

>>> input = ivy.array([[0.2, 0.8], [0.5, 0.5]])
>>> target = ivy.array([[0.6, 0.4], [0.3, 0.7]])
>>> ivy.kl_div(input, target)
ivy.array(-0.555969)

>>> input = ivy.array([[0.2, 0.8], [0.5, 0.5]])
>>> target = ivy.array([[0.6, 0.4], [0.3, 0.7]])
>>> ivy.kl_div(input, target, reduction='sum')
ivy.array(-2.223876)

>>> input = ivy.array([[0.2, 0.8], [0.5, 0.5]])
>>> target = ivy.array([[0.6, 0.4], [0.3, 0.7]])
>>> ivy.kl_div(input, target, reduction='batchmean')
ivy.array(-1.111938)

>>> input = ivy.array([0.2, 0.8], [0.5, 0.5])
>>> target = ivy.array([0.6, 0.4], [0.3, 0.7])
>>> ivy.kl_div(input, target, reduction='none')
ivy.array([[-0.42649534, -0.68651628],
[-0.51119184, -0.59967244]])

ivy.l1_loss(input, target, /, *, reduction='mean', out=None)[source]#

Compute L1 loss (Mean Absolute Error - MAE) between targeticted and input values.

Parameters:
• input (Union[ivy.Array, ivy.NativeArray]) – Input array containing input values.

• target (Union[ivy.Array, ivy.NativeArray]) – Input array containing targeted values.

• reduction (str, optional) – Reduction method for the output loss. Options: “none” (no reduction), “mean” (mean of losses), “sum” (sum of losses). Default: “mean”.

• out (Optional[ivy.Array], optional) – Optional output array for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Array

Returns:

ivy.Array – The L1 loss (MAE) between the given input and targeticted values.

Examples

>>> x = ivy.array([1.0, 2.0, 3.0])
>>> y = ivy.array([0.5, 2.5, 2.0])
>>> print(ivy.l1_loss(x, y))
ivy.array(0.6)
>>> a = ivy.array([[1.0, 2.0], [3.0, 4.0]])
>>> b = ivy.array([[0.5, 1.5], [2.5, 3.5]])
>>> print(ivy.l1_loss(a, b))
ivy.array(0.5)

ivy.log_poisson_loss(true, pred, /, *, compute_full_loss=False, axis=-1, reduction='none', out=None)[source]#

Compute the log-likelihood loss between the prediction and the target under the assumption that the target has a Poisson distribution. Caveat: By default, this is not the exact loss, but the loss minus a constant term [log(z!)]. That has no effect for optimization, but does not play well with relative loss comparisons. To compute an approximation of the log factorial term, specify compute_full_loss=True to enable Stirling’s Approximation.

Parameters:
• true (Union[Array, NativeArray]) – input array containing true labels.

• pred (Union[Array, NativeArray]) – input array containing Predicted labels.

• compute_full_loss (bool, default: False) – whether to compute the full loss. If false, a constant term is dropped in favor of more efficient optimization. Default: False.

• axis (int, default: -1) – the axis along which to compute the log-likelihood loss. If axis is -1, the log-likelihood loss will be computed along the last dimension. Default: -1.

• reduction (str, default: 'none') – 'none': No reduction will be applied to the output. 'mean': The output will be averaged. 'sum': The output will be summed. Default: 'none'.

• out (Optional[Array], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret – The binary log-likelihood loss between the given distributions.

Examples

>>> x = ivy.array([0, 0, 1, 0])
>>> y = ivy.array([0.25, 0.25, 0.25, 0.25])
>>> print(ivy.log_poisson_loss(x, y))
ivy.array([1.28402555, 1.28402555, 1.03402555, 1.28402555])

>>> z = ivy.array([0.1, 0.1, 0.7, 0.1])
>>> print(ivy.log_poisson_loss(x, z, reduction='mean'))
ivy.array(1.1573164)

ivy.poisson_nll_loss(input, target, *, log_input=True, full=False, eps=1e-08, reduction='mean')[source]#

Compute the Poisson Negative Log Likelihood Loss.

This function calculates the negative log likelihood loss between the input and targetunder the assumption that the target follows a Poisson distribution. By default, the loss is not the exact loss, but the loss minus a constant term [log(z!)]. This omission does not affect optimization but can be significant for relative loss comparisons. The Stirling’s Approximation is used to approximate the log factorial term when full is set to True.

Parameters:
• input (Union[Array, NativeArray]) – Expectation of the underlying Poisson distribution.

• target (Union[Array, NativeArray]) – Random sample from the Poisson distribution described by the input.

• log_input (bool, default: True) – If True, the loss is computed as $$exp(input) - target * input$$. If False, the loss is computed as $$input - target * log(input + eps)$$. Default is True.

• full (bool, default: False) – Whether to compute the full loss, i.e., to add the Stirling approximation term $$target * log(target) - target + 0.5 * log(2 * pi * target)$$. Default is False.

• eps (float, default: 1e-08) – Small value to prevent evaluation of log(0) when log_input is False. Default is 1e-8.

• reduction (str, default: 'mean') – Specifies the reduction applied to the output. Options are ‘none’, ‘mean’, or ‘sum’. ‘none’: no reduction will be applied. ‘mean’: the output will be averaged. ‘sum’: the output will be summed. Default is ‘mean’.

Return type:

Array

Returns:

ret – An array of the same shape as input representing the Poisson Negative Log Likelihood Loss.

Raises:

ValueError – If the input and target tensors do not have the same shape.

Examples

>>> input_tensor = ivy.array([1, 2, 3, 4], dtype=ivy.float64)
>>> target_tensor = ivy.array([2, 2, 2, 2], dtype=ivy.float64)
>>> loss = poisson_nll_loss(input_tensor, target_tensor, log_input=False)
>>> print(loss)
ivy.array(0.91097307)

ivy.smooth_l1_loss(input, target, /, *, beta=1.0, reduction='mean', out=None)[source]#

Compute the smooth L1 loss between two input tensors.

Parameters:
• input (array_like) – First input tensor.

• target (array_like) – Second input tensor.

• beta (float, optional) – The smooth parameter. Default is 1.0.

• reduction (str, optional) – Specifies the type of reduction to apply to the output. Should be one of ‘none’, ‘sum’, or ‘mean’. Default is ‘mean’.

• out (array, optional) – Optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret (array) – The smooth_l1_loss between the two input tensors.

Examples

>>> input = ivy.array([1.0, 2.0, 3.0])
>>> target = ivy.array([2.5, 1.8, 3.2])
>>> ivy.smooth_l1_loss(x, y, beta=1.0)
ivy.array(0.3467)
>>> input = ivy.array([1.0, 2.0, 3.0])
>>> target = ivy.array([6.0, 2.0, 3.0])
>>> ivy.smooth_l1_loss(x, y, beta=1.0)
ivy.array(1.5)
>>> input = ivy.array([2.0, 3.0, 5.0, 7.0])
>>> target = ivy.array([2.5, 3.5, 5.5, 6.5])
>>> loss = ivy.smooth_l1_loss(input, target, beta=1.5, reduction='sum')
ivy.array(0.5)
>>> input = ivy.array([0.8, 1.2, 2.5, 3.7])
>>> target = ivy.array([0.9, 1.0, 2.3, 3.6])
>>> loss = ivy.smooth_l1_loss(input, target, beta=0.5, reduction='none')
ivy.array([0.0133, 0.0250, 0.0056, 0.0025])
>>> input = ivy.array([2.0, 3.0, 5.0, 7.0])
>>> target = ivy.array([2.5, 3.5, 5.5, 6.5])
>>> loss = ivy.smooth_l1_loss(input, target, beta=0.2, reduction='mean')
ivy.array(0.025)


With ivy.NativeArray input:

>>> x = ivy.native_array([1.5, 2.2, 3.7])
>>> y = ivy.native_array([2.1, 1.9, 3.5])
>>> print(ivy.smooth_l1_loss(x, y, beta=0.5))
ivy.array(0.0675)


With ivy.Container input:

>>> x = ivy.Container(a=ivy.array([1.0, 2.0, 3.0]))
>>> y = ivy.Container(a=ivy.array([2.5, 1.8, 3.2]))
>>> print(ivy.smooth_l1_loss(x, y, beta=1.0))
{
a: ivy.array(0.3467)
}


With a mix of ivy.Array and ivy.NativeArray inputs:

>>> x = ivy.array([1.0, 2.0, 3.0])
>>> y = ivy.native_array([6.0, 2.0, 3.0])
>>> print(ivy.smooth_l1_loss(x, y, beta=0.5))
ivy.array(1.5)


With a mix of ivy.Array and ivy.Container inputs:

>>> x = ivy.array([1.0, 2.0, 3.0])
>>> y = ivy.Container(a=ivy.array([6.0, 2.0, 3.0]))
>>> print(ivy.smooth_l1_loss(x, y, beta=1.0))
{
a: ivy.array(1.5)
}


Instance Method Examples

With ivy.Array input:

>>> x = ivy.array([1.0, 2.0, 3.0])
>>> y = ivy.array([2.5, 1.8, 3.2])
>>> print(x.smooth_l1_loss(y, beta=1.0))
ivy.array(0.3467)


With ivy.Container input:

>>> x = ivy.Container(a=ivy.array([1.0, 2.0, 3.0]))
>>> y = ivy.Container(a=ivy.array([2.5, 1.8, 3.2]))
>>> print(x.smooth_l1_loss(y, beta=1.0))
{
a: ivy.array(0.3467)
}

ivy.soft_margin_loss(input, target, /, *, reduction='mean', out=None)[source]#

Compute the soft-margin hinge loss between predicted scores and true binary labels.

Parameters:
• input (array_like) – True binary labels, of shape (batch_size,).

• target (array_like) – Predicted scores, of shape (batch_size,).

• reduction ({'mean', 'sum', 'none'}, optional) – Type of reduction to apply to the output. Default is ‘mean’.

• out (array_like, optional) – Optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret (array) – The soft-margin hinge loss between the predicted scores and true binary labels.

Examples

>>> input = ivy.array([1, 0, 1, 0])
>>> target = ivy.array([0.8, 0.2, -0.6, 1.5])
>>> ivy.soft_margin_loss(input, target)
ivy.array(0.6987)

>>> input = ivy.array([1, 1, 0, 0])
>>> target = ivy.array([0.8, 0.7, 0.2, 0.1])
>>> ivy.soft_margin_loss(input, target, reduction='sum')
ivy.array(2.1606)

>>> input = ivy.array([1, 1, 0, 0])
>>> target = ivy.array([0.8, 0.7, 0.2, 0.1])
>>> ivy.soft_margin_loss(input, target, reduction='none')
ivy.array([0.3711, 0.4032, 0.6931, 0.6931])