Video Tutorial
Write Ivy code#
Get familiar with Ivy’s basic concepts and start writing framework-agnostic code.
Contents#
Installing Ivy#
⚠️ If you are running this notebook in Colab, you will have to install Ivy and some dependencies manually. You can do so by running the cell below ⬇️
If you want to run the notebook locally but don’t have Ivy installed just yet, you can check out the Get Started section of the docs.
[ ]:
!pip install ivy
In this introduction we’ll go over the basics of using Ivy to write your own framework-indepent, future-proof code!
If you want to delve deeper into the theory behind the contents of this notebook you can check out the Design and the Deep Dive sections of the documentation!
Importing Ivy#
First of all, let’s import Ivy!
[2]:
import ivy
Ivy Backend Handler#
Ivy, when used as a ML framework, is esentially an abstraction layer that supports multiple frameworks as the backend. This means that any code written in Ivy can be executed in any of the supported frameworks, with their framework-specific data structures, functions, optimizations, quirks and perks, all managed by Ivy under the hood.
To change the backend, we can simply call ivy.set_backend with the appropiate framework passed as a string. This is the simplest way to interact with the Backend Handler submodule, which keeps track of the current backend and links Ivy’s objects and functions with the appropriate framework-specific ones.
For example:
[4]:
ivy.set_backend("tensorflow")
Data Structures#
The basic data structure in Ivy is the ivy.Array. This is an abstraction of the array classes of the supported frameworks. Likewise, we also have ivy.NativeArray, which is an alias for the array class of the selected backend.
Lastly, there is another structure, the ivy.Container, which is a subclass of dict optimized for recursive operations, you can learn more about it here!
Let’s create an array using ivy.array(). In a similar fashion, we can use ivy.native_array() to create a torch.Tensor, as the backend is now torch.
[10]:
ivy.set_backend("torch")
x = ivy.array([1, 2, 3])
print(type(x))
x = ivy.native_array([1, 2, 3])
print(type(x))
<class 'ivy.data_classes.array.array.Array'>
<class 'torch.Tensor'>
Ivy Functional API#
Ivy does not implement its own low-level (C++/CUDA) backend for its functions. Instead, it wraps the functional API of existing frameworks, unifying their fundamental functions under a common signature. For example, let’s take a look at ivy.matmul():
[11]:
ivy.set_backend("jax")
x1, x2 = ivy.array([[1], [2], [3]]), ivy.array([[1, 2, 3]])
output = ivy.matmul(x1, x2)
print(type(output.to_native()))
ivy.set_backend("tensorflow")
x1, x2 = ivy.array([[1], [2], [3]]), ivy.array([[1, 2, 3]])
output = ivy.matmul(x1, x2)
print(type(output.to_native()))
ivy.set_backend("torch")
x1, x2 = ivy.array([[1], [2], [3]]), ivy.array([[1, 2, 3]])
output = ivy.matmul(x1, x2)
print(type(output.to_native()))
<class 'jaxlib.xla_extension.ArrayImpl'>
<class 'tensorflow.python.framework.ops.EagerTensor'>
<class 'torch.Tensor'>
The output arrays above are ivy.Array instances, which is why we need to call the to_native() method to retrieve the underlying, native array.
However, if you want the functions to return the native arrays directly, you can disable the array_mode of Ivy using ivy.set_array_mode().
[12]:
ivy.set_array_mode(False)
ivy.set_backend("jax")
x1, x2 = ivy.native_array([[1], [2], [3]]), ivy.native_array([[1, 2, 3]])
output = ivy.matmul(x1, x2)
print(type(output))
ivy.set_backend("tensorflow")
x1, x2 = ivy.native_array([[1], [2], [3]]), ivy.native_array([[1, 2, 3]])
output = ivy.matmul(x1, x2)
print(type(output))
ivy.set_backend("torch")
x1, x2 = ivy.native_array([[1], [2], [3]]), ivy.native_array([[1, 2, 3]])
output = ivy.matmul(x1, x2)
print(type(output))
ivy.set_array_mode(True)
<class 'jaxlib.xla_extension.ArrayImpl'>
<class 'tensorflow.python.framework.ops.EagerTensor'>
<class 'torch.Tensor'>
Keeping this in mind, you can build any function you want as a composition of Ivy functions. When executed, this function will ultimately call the current backend functions from its functional API.
[13]:
def sigmoid(z):
return ivy.divide(1, (1 + ivy.exp(-z)))
Ivy Stateful API#
Alongside the Functional API, Ivy also has a stateful API, which builds on its functional API and the ivy.Container class to provide high-level classes such as optimizers, network layers, or trainable modules.
The most important stateful class within Ivy is ivy.Module, which can be used to create trainable layers and entire networks. Given the importance of this class, we will explore it further in the Write a model using Ivy tutorial!
Round Up#
Congratulations! There is much more to come, but you now have a basic understanding of Ivy and how it can be used to write framework-independent, future-proof code! Now that you have a good foundation, let’s keep exploring Ivy’s tools and their powerful features! 🚀