lars_update#

ivy.lars_update(w, dcdw, lr, /, *, decay_lambda=0, stop_gradients=True, out=None)[source]#

Update weights ws of some function, given the derivatives of some cost c with respect to ws, [dc/dw for w in ws], by applying Layerwise Adaptive Rate Scaling (LARS) method.

Parameters:

  • w (Union[Array, NativeArray]) – Weights of the function to be updated.

  • dcdw (Union[Array, NativeArray]) – Derivates of the cost c with respect to the weights ws, [dc/dw for w in ws].

  • lr (Union[float, Array, NativeArray]) – Learning rate, the rate at which the weights should be updated relative to the gradient.

  • decay_lambda (float) – The factor used for weight decay. Default is zero. (default: 0)

  • stop_gradients (bool) – Whether to stop the gradients of the variables after each gradient step. (default: True) Default is True.

  • out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that the (default: None) inputs broadcast to.

Return type:

Array

Returns:

ret – The new function weights ws_new, following the LARS updates.

Array.lars_update(self, dcdw, lr, /, *, decay_lambda=0, stop_gradients=True, out=None)#

ivy.Array instance method variant of ivy.lars_update. This method simply wraps the function, and so the docstring for ivy.lars_update also applies to this method with minimal changes.

Parameters:

  • self (Array) – Weights of the function to be updated.

  • dcdw (Union[Array, NativeArray]) – Derivates of the cost c with respect to the weights ws, [dc/dw for w in ws].

  • lr (Union[float, Array, NativeArray]) – Learning rate, the rate at which the weights should be updated relative to the gradient.

  • decay_lambda (float) – The factor used for weight decay. Default is zero. (default: 0)

  • stop_gradients (bool) – Whether to stop the gradients of the variables after each gradient step. (default: True) Default is True.

  • out (Optional[Array]) – optional output array, for writing the result to. It must have a shape that (default: None) the inputs broadcast to.

Return type:

Array

Returns:

ret – The new function weights ws_new, following the LARS updates.

Container.lars_update(self, dcdw, lr, /, *, decay_lambda=0, stop_gradients=True, out=None)#

Update weights ws of some function, given the derivatives of some cost c with respect to ws, [dc/dw for w in ws], by applying Layerwise Adaptive Rate Scaling (LARS) method.

Parameters:

  • self (Container) – Weights of the function to be updated.

  • dcdw (Union[Array, NativeArray, Container]) – Derivates of the cost c with respect to the weights ws, [dc/dw for w in ws].

  • lr (Union[float, Array, NativeArray, Container]) – Learning rate, the rate at which the weights should be updated relative to the gradient.

  • decay_lambda (float) – The factor used for weight decay. Default is zero. (default: 0)

  • stop_gradients (bool) – Whether to stop the gradients of the variables after each gradient step. (default: True) Default is True.

  • out (Optional[Container]) – optional output container, for writing the result to. It must have a shape (default: None) that the inputs broadcast to.

Returns:

ret – The new function weights ws_new, following the LARS updates.