lars_update#
- ivy.lars_update(w, dcdw, lr, /, *, decay_lambda=0, stop_gradients=True, out=None)[source]#
Update weights ws of some function, given the derivatives of some cost c with respect to ws, [dc/dw for w in ws], by applying Layerwise Adaptive Rate Scaling (LARS) method.
- Parameters:
w (
Union
[Array
,NativeArray
]) – Weights of the function to be updated.dcdw (
Union
[Array
,NativeArray
]) – Derivates of the cost c with respect to the weights ws, [dc/dw for w in ws].lr (
Union
[float
,Array
,NativeArray
]) – Learning rate, the rate at which the weights should be updated relative to the gradient.decay_lambda (
float
) – The factor used for weight decay. Default is zero. (default:0
)stop_gradients (
bool
) – Whether to stop the gradients of the variables after each gradient step. (default:True
) Default isTrue
.out (
Optional
[Array
]) – optional output array, for writing the result to. It must have a shape that the (default:None
) inputs broadcast to.
- Return type:
- Returns:
ret – The new function weights ws_new, following the LARS updates.
- Array.lars_update(self, dcdw, lr, /, *, decay_lambda=0, stop_gradients=True, out=None)#
ivy.Array instance method variant of ivy.lars_update. This method simply wraps the function, and so the docstring for ivy.lars_update also applies to this method with minimal changes.
- Parameters:
self (
Array
) – Weights of the function to be updated.dcdw (
Union
[Array
,NativeArray
]) – Derivates of the cost c with respect to the weights ws, [dc/dw for w in ws].lr (
Union
[float
,Array
,NativeArray
]) – Learning rate, the rate at which the weights should be updated relative to the gradient.decay_lambda (
float
) – The factor used for weight decay. Default is zero. (default:0
)stop_gradients (
bool
) – Whether to stop the gradients of the variables after each gradient step. (default:True
) Default isTrue
.out (
Optional
[Array
]) – optional output array, for writing the result to. It must have a shape that (default:None
) the inputs broadcast to.
- Return type:
Array
- Returns:
ret – The new function weights ws_new, following the LARS updates.
- Container.lars_update(self, dcdw, lr, /, *, decay_lambda=0, stop_gradients=True, out=None)#
Update weights ws of some function, given the derivatives of some cost c with respect to ws, [dc/dw for w in ws], by applying Layerwise Adaptive Rate Scaling (LARS) method.
- Parameters:
self (
Container
) – Weights of the function to be updated.dcdw (
Union
[Array
,NativeArray
,Container
]) – Derivates of the cost c with respect to the weights ws, [dc/dw for w in ws].lr (
Union
[float
,Array
,NativeArray
,Container
]) – Learning rate, the rate at which the weights should be updated relative to the gradient.decay_lambda (
float
) – The factor used for weight decay. Default is zero. (default:0
)stop_gradients (
bool
) – Whether to stop the gradients of the variables after each gradient step. (default:True
) Default isTrue
.out (
Optional
[Container
]) – optional output container, for writing the result to. It must have a shape (default:None
) that the inputs broadcast to.
- Returns:
ret – The new function weights ws_new, following the LARS updates.