Ctrl+K

Backend Setting#

The backend framework can either be set by calling ivy.set_backend(backend_name) or it can inferred from the arguments. For the latter, a global variable implicit_backend is located in the file which is initialized as numpy, and is always used to infer the backend in cases where: (a) no backend has been set using the set_backend function and (b) the backend cannot be inferred from the inputs. If the framework can be inferred from the inputs, then this is always used, and the implicit_backend is overwritten with the framework inferred. numpy will always be the default backend unless it is explicitly set or is inferred. When calling this function for setting the backend, the following steps are performed:

  1. store a global copy of the original ivy.__dict__ to ivy_original_dict, if this is not already stored.

  2. import the backend module, for example ivy.functional.backends.torch, if the backend has been passed in as a string. All functions in this unmodified backend module are primary functions, because only primary functions are stored in ivy.functional.backends.backend_name. This backend module does not include any compositional functions.

  3. loop through the original ivy_original_dict (which has all functions, including compositional), and (a) add the primary function from the backend if it exists, (b) else add the compositional function from ivy_original_dict.

  4. wrap the functions where necessary, extending them with shared repeated functionality and writing the function to ivy.__dict__. Wrapping is used in order to avoid excessive code duplication in every backend function implementation. This is explained in more detail in the next section: Function Wrapping.

It’s helpful to look at an example:

x = ivy.array([[2., 3.]])
ivy.current_backend()
<module 'ivy.functional.backends.numpy' from '/opt/project/ivy/functional/backends/numpy/__init__.py'>

y = ivy.multiply(torch.Tensor([3.]), torch.Tensor([4.]))
ivy.current_backend()
<module 'ivy.functional.backends.torch' from '/opt/project/ivy/functional/backends/torch/__init__.py'>

ivy.set_backend('jax')
z = ivy.matmul(jax.numpy.array([[2.,3.]]), jax.numpy.array([[5.],[6.]]))
ivy.current_backend()
<module 'ivy.functional.backends.jax' from '/opt/project/ivy/functional/backends/jax/__init__.py'>
ivy.previous_backend()
ivy.current_backend()
<module 'ivy.functional.backends.torch' from '/opt/project/ivy/functional/backends/torch/__init__.py'>

In the last example above, the moment any backend is set, it will be used over the implicit_backend. However when the current backend is set to the previous using the ivy.previous_backend(), the implicit_backend will be used as a fallback, which will assume the backend from the last run. While the implicit_backend functionality gives more freedom to the user, the recommended way of doing things would be to set the backend explicitly. In addition, all the previously set backends can be cleared by calling ivy.unset_backend().

Dynamic Backend Setting#

Working with different backends in Ivy can be challenging, especially when you need to switch between backends frequently. To make this easier, users can make use of the dynamic backend attribute of ivy.Array and ivy.Container classes which allow you to automatically convert ivy arrays to the new backend whenever the backend is changed. Essentially, when the user calls ivy.set_backend(<backend>, dynamic=True), the following steps are performed:

  1. First, all live objects in the current project scope are found and then filtered to only include ivy.Array/ivy.Container objects.

  2. Then, these objects are iterated through and converted to the target backend using DLPack or numpy as an intermediary.

By default, the dynamic backend attribute is set to True when you create an ivy array (e.g., x = ivy.array([1,2,3])), but the attribute is mutable and can be changed after the ivy array is created (e.g., x.dynamic_backend= True). Here’s an example to illustrate how this works in practice:

ivy.set_backend('torch')
x = ivy.array([1,2,3])
y = ivy.array([1,2,3])
y.dynamic_backend=False
x.dynamic_backend=True
x.data # torch tensor
y.data # torch.tensor

ivy.set_backend('jax')
x.data # will be a jax array
y.data # will still be a torch tensor since dynamic_backend=False

Setting the attribute to True converts the array to the current backend even if the backend was set with dynamic=False. In addition to setting the dynamic backend attribute for individual ivy arrays, you can also set or unset the dynamic backend feature globally for all such instances using ivy.set_dynamic_backend and ivy.unset_dynamic_backend respectively.

Another useful feature of the dynamic backend is the ivy.dynamic_backend_as context manager. This allows you to write code like this:

with ivy.dynamic_backend_as(True):
  a = ivy.array([0., 1.])
  b = ivy.array([2., 3.])

with ivy.dynamic_backend_as(False):
  c = ivy.array([4., 5.])
  d = ivy.array([6., 7.])

This makes it easy to define different sections of your project with different settings, without having to explicitly call ivy.set_<something> and ivy.unset_<something> etc.

Backend and Frontend Version Support#

Each time a new ivy backend is set, the backend_handler modifies the ivy.__dict__ to support the multiple versions of functions that are not forward compatible. For example, torch.ones_like() in the latest stable version 1.12 has many new arguments dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format compared to the same function at version 0.3.1. None of these new arguments will cause any forward compatibility issues: they weren’t used in old code, and they can now just be used in new code if desired. However, the removal of the out argument does break forward compatibility. Old torch code will raise an Argument Not Found error if being run with new torch versions. However, such forward-breaking changes are in the vast minority.

We currently use a naming convention for such functions and name them as fn_name_v_1p12_and_above which means that this particular implementation of the function is valid for versions 1.12 and above. Similarly, fn_name_v_1p01_to_1p1 means that the function is valid for versions between 1.01 and 1.1 both inclusive. Each time a backend is set, we go through the backend.__dict__ and for all functions for which multiple versions are detected, we simply import and assign the original fn_name to the version specific one. We do so by detecting the version of the backend framework installed on the user’s end.

We follow the same workflow for providing version support to the frontend functions. Again the version is inferred by importing the corresponding framework on the user’s system. If the user’s system doesn’t have the backend framework installed, we default to the latest version.

Round Up

This should have hopefully given you a good feel for how the backend framework is set.

If you have any questions, please feel free to reach out on discord in the backend setting thread!

Video

On this page