Skip to main content
Ctrl+K
Logo image

Overview

  • Background
    • ML Explosion
    • Why Unify?
    • Standardization
  • Design
    • Building Blocks
    • Ivy as a Transpiler
    • Ivy as a Framework
      • Ivy Container
      • Ivy Stateful API
      • Ivy Array
  • Related Work
    • API Standards
    • Wrapper Frameworks
    • Frameworks
    • Graph Tracers
    • Exchange Formats
    • Compiler Infrastructure
    • Multi-Vendor Compiler Frameworks
    • Vendor-Specific APIs
    • Vendor-Specific Compilers
    • ML-Unifying Companies
    • What does Ivy Add?
  • Extensions
    • Applied Libraries
  • Contributing
    • Setting Up
    • The Basics
    • Building the Docs
    • Deep Dive
    • Open Tasks
    • Applied Libraries
    • Helpful Resources
  • Deep Dive
    • Navigating the Code
    • Function Types
    • Superset Behaviour
    • Backend Setting
    • Arrays
    • Containers
    • Data Types
    • Devices
    • Inplace Updates
    • Function Wrapping
    • Formatting
    • Function Arguments
    • Docstrings
    • Docstring Examples
    • Array API Tests
    • Ivy Tests
    • Ivy Frontends
    • Ivy Frontend Tests
    • Exception Handling
    • Continuous Integration
    • Gradients
    • Building the Docs Pipeline
  • Examples
  • FAQ
  • Glossary

Compiling and Transpiling

  • Graph Compiler
  • Transpiler

API Reference

  • Functions
    • Activations
      • deserialize
      • gelu
      • get
      • leaky_relu
      • log_softmax
      • mish
      • relu
      • sigmoid
      • softmax
      • softplus
    • Constants
    • Control flow ops
      • for_loop
      • if_else
      • while_loop
    • Creation
      • arange
      • array
      • asarray
      • asarray_handle_nestable
      • asarray_infer_device
      • asarray_inputs_to_native_shapes
      • asarray_to_native_arrays_and_back
      • copy_array
      • empty
      • empty_like
      • eye
      • from_dlpack
      • frombuffer
      • full
      • full_like
      • linspace
      • logspace
      • meshgrid
      • native_array
      • one_hot
      • ones
      • ones_like
      • tril
      • triu
      • triu_indices
      • zeros
      • zeros_like
    • Data type
      • as_ivy_dtype
      • as_native_dtype
      • astype
      • broadcast_arrays
      • broadcast_to
      • can_cast
      • check_float
      • closest_valid_dtype
      • default_complex_dtype
      • default_dtype
      • default_float_dtype
      • default_int_dtype
      • default_uint_dtype
      • dtype
      • dtype_bits
      • finfo
      • function_supported_dtypes
      • function_unsupported_dtypes
      • iinfo
      • infer_default_dtype
      • invalid_dtype
      • is_bool_dtype
      • is_complex_dtype
      • is_float_dtype
      • is_int_dtype
      • is_native_dtype
      • is_uint_dtype
      • promote_types
      • promote_types_of_inputs
      • result_type
      • set_default_complex_dtype
      • set_default_dtype
      • set_default_float_dtype
      • set_default_int_dtype
      • set_default_uint_dtype
      • type_promote_arrays
      • unset_default_complex_dtype
      • unset_default_dtype
      • unset_default_float_dtype
      • unset_default_int_dtype
      • unset_default_uint_dtype
      • valid_dtype
    • Device
      • as_ivy_dev
      • as_native_dev
      • clear_cached_mem_on_dev
      • default_device
      • dev
      • dev_util
      • function_supported_devices
      • function_unsupported_devices
      • get_all_ivy_arrays_on_dev
      • gpu_is_available
      • num_cpu_cores
      • num_gpus
      • num_ivy_arrays_on_dev
      • percent_used_mem_on_dev
      • print_all_ivy_arrays_on_dev
      • set_default_device
      • set_split_factor
      • split_factor
      • split_func_call
      • to_device
      • total_mem_on_dev
      • tpu_is_available
      • unset_default_device
      • used_mem_on_dev
    • Elementwise
      • abs
      • acos
      • acosh
      • add
      • asin
      • asinh
      • atan
      • atan2
      • atanh
      • bitwise_and
      • bitwise_invert
      • bitwise_left_shift
      • bitwise_or
      • bitwise_right_shift
      • bitwise_xor
      • ceil
      • cos
      • cosh
      • deg2rad
      • divide
      • equal
      • erf
      • exp
      • exp2
      • expm1
      • floor
      • floor_divide
      • fmin
      • fmod
      • greater
      • greater_equal
      • isfinite
      • isinf
      • isnan
      • isreal
      • lcm
      • less
      • less_equal
      • log
      • log10
      • log1p
      • log2
      • logaddexp
      • logical_and
      • logical_not
      • logical_or
      • logical_xor
      • maximum
      • minimum
      • multiply
      • negative
      • not_equal
      • positive
      • pow
      • rad2deg
      • reciprocal
      • remainder
      • round
      • sign
      • sin
      • sinh
      • sqrt
      • square
      • subtract
      • tan
      • tanh
      • trapz
      • trunc
      • trunc_divide
    • Experimental
      • Activations
        • logit
        • logsigmoid
        • prelu
        • relu6
        • selu
        • silu
        • thresholded_relu
      • Constants
      • Creation
        • eye_like
        • hamming_window
        • hann_window
        • indices
        • kaiser_bessel_derived_window
        • kaiser_window
        • ndenumerate
        • ndindex
        • tril_indices
        • vorbis_window
      • Data type
      • Device
      • Elementwise
        • allclose
        • angle
        • binarizer
        • conj
        • copysign
        • count_nonzero
        • diff
        • fix
        • float_power
        • fmax
        • frexp
        • gcd
        • gradient
        • hypot
        • imag
        • isclose
        • ldexp
        • lerp
        • logaddexp2
        • nan_to_num
        • nansum
        • nextafter
        • real
        • signbit
        • sinc
        • xlogy
        • zeta
      • General
      • Gradients
        • bind_custom_gradient_function
      • Layers
        • adaptive_avg_pool1d
        • adaptive_avg_pool2d
        • avg_pool1d
        • avg_pool2d
        • avg_pool3d
        • dct
        • dft
        • dropout1d
        • dropout3d
        • embedding
        • fft
        • ifft
        • interp
        • interpolate
        • max_pool1d
        • max_pool2d
        • max_pool3d
      • Linear algebra
        • adjoint
        • cond
        • cov
        • diagflat
        • eig
        • eigh_tridiagonal
        • eigvals
        • kron
        • matrix_exp
        • multi_dot
      • Losses
      • Manipulation
        • as_strided
        • associative_scan
        • atleast_1d
        • atleast_2d
        • atleast_3d
        • broadcast_shapes
        • concat_from_sequence
        • dsplit
        • dstack
        • expand
        • flatten
        • fliplr
        • flipud
        • heaviside
        • hsplit
        • hstack
        • i0
        • moveaxis
        • pad
        • rot90
        • take_along_axis
        • top_k
        • unique_consecutive
        • vsplit
        • vstack
      • Meta
      • Nest
      • Norms
        • batch_norm
        • instance_norm
        • l1_normalize
        • l2_normalize
        • lp_normalize
      • Random
        • bernoulli
        • beta
        • dirichlet
        • gamma
        • poisson
      • Searching
        • unravel_index
      • Set
      • Sorting
        • invert_permutation
        • lexsort
      • Sparse array
        • is_ivy_sparse_array
        • is_native_sparse_array
        • native_sparse_array
        • native_sparse_array_to_indices_values_and_shape
      • Statistical
        • bincount
        • corrcoef
        • histogram
        • igamma
        • median
        • nanmean
        • nanmedian
        • quantile
      • Utility
    • General
      • all_equal
      • arg_info
      • arg_names
      • array_equal
      • assert_supports_inplace
      • cache_fn
      • clip_matrix_norm
      • clip_vector_norm
      • container_types
      • current_backend_str
      • default
      • einops_rearrange
      • einops_reduce
      • einops_repeat
      • exists
      • fourier_encode
      • function_supported_devices_and_dtypes
      • function_unsupported_devices_and_dtypes
      • gather
      • gather_nd
      • get_all_arrays_in_memory
      • get_array_mode
      • get_exception_trace_mode
      • get_item
      • get_min_base
      • get_min_denominator
      • get_nestable_mode
      • get_num_dims
      • get_queue_timeout
      • get_referrers_recursive
      • get_show_func_wrapper_trace_mode
      • get_tmp_dir
      • has_nans
      • inplace_arrays_supported
      • inplace_decrement
      • inplace_increment
      • inplace_update
      • inplace_variables_supported
      • is_array
      • is_ivy_array
      • is_ivy_container
      • is_native_array
      • isin
      • isscalar
      • itemsize
      • match_kwargs
      • multiprocessing
      • num_arrays_in_memory
      • print_all_arrays_in_memory
      • scatter_flat
      • scatter_nd
      • set_array_mode
      • set_exception_trace_mode
      • set_min_base
      • set_min_denominator
      • set_nestable_mode
      • set_queue_timeout
      • set_shape_array_mode
      • set_show_func_wrapper_trace_mode
      • set_tmp_dir
      • shape
      • shape_array_mode
      • stable_divide
      • stable_pow
      • strides
      • supports_inplace_updates
      • to_ivy_shape
      • to_list
      • to_native_shape
      • to_numpy
      • to_scalar
      • try_else_none
      • unset_array_mode
      • unset_exception_trace_mode
      • unset_nestable_mode
      • unset_queue_timeout
      • unset_shape_array_mode
      • unset_show_func_wrapper_trace_mode
      • value_is_nan
      • vmap
    • Gradients
      • adam_step
      • adam_update
      • execute_with_gradients
      • grad
      • gradient_descent_update
      • jac
      • lamb_update
      • lars_update
      • optimizer_update
      • stop_gradient
      • value_and_grad
    • Layers
      • conv
      • conv1d
      • conv1d_transpose
      • conv2d
      • conv2d_transpose
      • conv3d
      • conv3d_transpose
      • conv_general_dilated
      • conv_general_transpose
      • depthwise_conv2d
      • dropout
      • linear
      • lstm_update
      • multi_head_attention
      • scaled_dot_product_attention
    • Linear algebra
      • cholesky
      • cross
      • det
      • diag
      • diagonal
      • eig
      • eigh
      • eigvalsh
      • inner
      • inv
      • lu_factor
      • matmul
      • matrix_norm
      • matrix_power
      • matrix_rank
      • matrix_transpose
      • outer
      • pinv
      • qr
      • slogdet
      • solve
      • svd
      • svdvals
      • tensordot
      • tensorsolve
      • trace
      • vander
      • vecdot
      • vector_norm
      • vector_to_skew_symmetric_matrix
    • Losses
      • binary_cross_entropy
      • cross_entropy
      • sparse_cross_entropy
    • Manipulation
      • clip
      • concat
      • constant_pad
      • expand_dims
      • flip
      • permute_dims
      • repeat
      • reshape
      • roll
      • split
      • squeeze
      • stack
      • swapaxes
      • tile
      • unstack
      • zero_pad
    • Meta
      • fomaml_step
      • maml_step
      • reptile_step
    • Nest
      • all_nested_indices
      • copy_nest
      • duplicate_array_index_chains
      • index_nest
      • insert_into_nest_at_index
      • insert_into_nest_at_indices
      • map
      • map_nest_at_index
      • map_nest_at_indices
      • multi_index_nest
      • nested_any
      • nested_argwhere
      • nested_map
      • nested_multi_map
      • prune_empty
      • prune_nest_at_index
      • prune_nest_at_indices
      • set_nest_at_index
      • set_nest_at_indices
    • Norms
      • layer_norm
    • Random
      • multinomial
      • randint
      • random_normal
      • random_uniform
      • seed
      • shuffle
    • Searching
      • argmax
      • argmin
      • argwhere
      • nonzero
      • where
    • Set
      • unique_all
      • unique_counts
      • unique_inverse
      • unique_values
    • Sorting
      • argsort
      • msort
      • searchsorted
      • sort
    • Statistical
      • cummax
      • cummin
      • cumprod
      • cumsum
      • einsum
      • max
      • mean
      • min
      • prod
      • std
      • sum
      • var
    • Utility
      • all
      • any
  • Data classes
    • Array
      • Activations
      • Conversions
      • Creation
      • Data type
      • Device
      • Elementwise
      • Experimental
      • General
      • Gradients
      • Image
      • Layers
      • Linear algebra
      • Losses
      • Manipulation
      • Norms
      • Random
      • Searching
      • Set
      • Sorting
      • Statistical
      • Utility
      • Wrapping
    • Container
      • Activations
      • Base
      • Conversions
      • Creation
      • Data type
      • Device
      • Elementwise
      • Experimental
      • General
      • Gradients
      • Image
      • Layers
      • Linear algebra
      • Losses
      • Manipulation
      • Norms
      • Random
      • Searching
      • Set
      • Sorting
      • Statistical
      • Utility
      • Wrapping
  • Framework classes
    • Activations
    • Converters
    • Helpers
    • Initializers
    • Layers
    • Module
    • Norms
    • Optimizers
    • Sequential
  • Nested array
    • Nested array
  • Utils
    • Assertions
    • Backend
      • Ast helpers
      • Handler
      • Sub backend handler
    • Dynamic import
    • Exceptions
    • Inspection
    • Profiler
    • Verbosity
  • Testing
    • Assertions
    • Available frameworks
    • Function testing
    • Globals
    • Hypothesis helpers
      • Array helpers
      • Dtype helpers
      • General helpers
      • Number helpers
    • Structs
    • Test parameter flags
    • Testing helpers

Docs

  • Ivy
  • Ivy mech
  • Ivy vision
  • Ivy robot
  • Ivy gym
  • Ivy memory
  • Ivy builder
  • Ivy models
  • Ivy ecosystem
  • Related Work

Related Work#

In this section, we explain how Ivy compares to many other very important and related pieces of work, which also address fragmentation but at other areas within the ML stack.

Firstly, we need to look at the overall ML stack, and understand how the high level frameworks relate to the low level components.

In order to conceptualize this rather complex hierarchy, we have broken the ML stack into 9 groups, which are: API Standards, Wrapper Frameworks, Frameworks, Graph Tracers, Exchange Formats, Compiler Infrastructure, Multi-Vendor Compiler Frameworks, Vendor-Specific APIs and Vendor-Specific Compilers, going from high level to low level respectively.

https://github.com/unifyai/unifyai.github.io/blob/master/img/externally_linked/related_work/ml_stack.png?raw=true

Each of these groups within the ML stack has it’s own sub-section, linked below, within which we discuss various related projects which operate at that particular level within the stack.

We then compare Ivy to some other ML-unifying companies which are working on very important problems and are helping to unify the lower levels of the ML stack. We see these efforts as being very complimentary to Ivy’s vision for high level unification.

Finally, we discuss how Ivy compares to each of these important works at all levels within the ML stack.

(a) API Standards 🀝🏽
Standardized APIs which similar libraries should adhere to

(b) Wrapper Frameworks 🎁
Frameworks which wrap other ML frameworks

(c) Frameworks πŸ”’
Standalone ML Frameworks

(d) Graph Tracers πŸ•ΈοΈ
Extracting acyclic directed computation graphs from code

(e) Exchange Formats πŸ’±
File formats to exchange neural networks between frameworks

(f) Compiler Infrastructure πŸ”ŸοΈπŸ—οΈ
Infrastructure and standards to simplify the lives of compiler designers

(g) Multi-Vendor Compiler Frameworks πŸ–₯οΈπŸ’»πŸ”Ÿ
Executing ML code on a variety of hardware targets

(h) Vendor-Specific APIs πŸ’»πŸ”’
Interfacing with specific hardware in an intuitive manner

(i) Vendor-Specific Compilers πŸ’»πŸ”Ÿ
Compiling code to specific hardware

(j) ML-Unifying Companies πŸ“ˆ
Companies working towards unification in ML

(k) What does Ivy Add? 🟒
How does Ivy fit into all of this?

previous

Ivy Array

next

API Standards

Β© Copyright 2020-2023, Ivy Team.

Created using Sphinx 6.2.0.

Built with the PyData Sphinx Theme 0.13.3.