Skip to main content
Ctrl+K
Logo image
Github Docs Discord
Github Docs Discord
  • Home

The Basics

  • Get Started
  • Quickstart

Demos

  • Learn the basics
    • Write Ivy code
    • Unify code
    • Compile code
    • Transpile code
    • Lazy vs Eager
    • How to use decorators
    • Transpile any library
    • Transpile any model
  • Guides
    • Transpiling a PyTorch model to build on top
    • Transpiling a Tensorflow model to build on top
  • Examples and Demos
    • Using Ivy ResNet
    • Accelerating PyTorch models with JAX
    • Accelerating MMPreTrain models with JAX
    • Image Segmentation with Ivy UNet
    • Ivy AlexNet demo

Background

  • Motivation
    • ML Explosion
    • Why Unify?
    • Standardization
  • Related Work
    • API Standards
    • Wrapper Frameworks
    • Frameworks
    • Graph Tracers
    • Exchange Formats
    • Compiler Infrastructure
    • Multi-Vendor Compiler Frameworks
    • Vendor-Specific APIs
    • Vendor-Specific Compilers
    • ML-Unifying Companies
    • What does Ivy Add?

Contributors

  • Design
    • Building Blocks
    • Ivy as a Transpiler
    • Ivy as a Framework
      • Ivy Container
      • Ivy Stateful API
      • Ivy Array
  • Contributing
    • Setting Up
    • The Basics
    • Building the Docs
    • Deep Dive
    • Open Tasks
    • Helpful Resources
    • Error Handling
  • Deep Dive
    • Navigating the Code
    • Function Types
    • Superset Behaviour
    • Backend Setting
    • Arrays
    • Containers
    • Data Types
    • Devices
    • Inplace Updates
    • Function Wrapping
    • Formatting
    • Ivy-Lint: Ivy’s Custom Code Formatters
    • Function Arguments
    • Docstrings
    • Docstring Examples
    • Array API Tests
    • Ivy Tests
    • Ivy Frontends
    • Ivy Frontend Tests
    • Exception Handling
    • Continuous Integration
    • Gradients
    • Operating Modes
    • Building the Docs Pipeline
  • Glossary
  • FAQ

API Reference

  • One liners
    • ivy.compile()
    • ivy.transpile()
    • ivy.unify()
  • Functions
    • Activations
      • gelu
      • hardswish
      • leaky_relu
      • log_softmax
      • mish
      • relu
      • sigmoid
      • softmax
      • softplus
      • softsign
    • Constants
    • Control flow ops
      • cmp_is
      • cmp_isnot
      • for_loop
      • if_else
      • try_except
      • while_loop
    • Creation
      • arange
      • array
      • asarray
      • copy_array
      • empty
      • empty_like
      • eye
      • from_dlpack
      • frombuffer
      • full
      • full_like
      • linspace
      • logspace
      • meshgrid
      • native_array
      • one_hot
      • ones
      • ones_like
      • to_dlpack
      • tril
      • triu
      • triu_indices
      • zeros
      • zeros_like
    • Data type
      • as_ivy_dtype
      • as_native_dtype
      • astype
      • broadcast_arrays
      • broadcast_to
      • can_cast
      • check_float
      • closest_valid_dtype
      • default_complex_dtype
      • default_dtype
      • default_float_dtype
      • default_int_dtype
      • default_uint_dtype
      • dtype
      • dtype_bits
      • finfo
      • function_supported_dtypes
      • function_unsupported_dtypes
      • iinfo
      • infer_default_dtype
      • invalid_dtype
      • is_bool_dtype
      • is_complex_dtype
      • is_float_dtype
      • is_hashable_dtype
      • is_int_dtype
      • is_native_dtype
      • is_uint_dtype
      • promote_types
      • promote_types_of_inputs
      • result_type
      • set_default_complex_dtype
      • set_default_dtype
      • set_default_float_dtype
      • set_default_int_dtype
      • set_default_uint_dtype
      • type_promote_arrays
      • unset_default_complex_dtype
      • unset_default_dtype
      • unset_default_float_dtype
      • unset_default_int_dtype
      • unset_default_uint_dtype
      • valid_dtype
    • Device
      • as_ivy_dev
      • as_native_dev
      • clear_cached_mem_on_dev
      • default_device
      • dev
      • dev_util
      • function_supported_devices
      • function_unsupported_devices
      • get_all_ivy_arrays_on_dev
      • gpu_is_available
      • handle_soft_device_variable
      • num_cpu_cores
      • num_gpus
      • num_ivy_arrays_on_dev
      • percent_used_mem_on_dev
      • print_all_ivy_arrays_on_dev
      • set_default_device
      • set_soft_device_mode
      • set_split_factor
      • split_factor
      • split_func_call
      • to_device
      • total_mem_on_dev
      • tpu_is_available
      • unset_default_device
      • unset_soft_device_mode
      • used_mem_on_dev
    • Elementwise
      • abs
      • acos
      • acosh
      • add
      • angle
      • asin
      • asinh
      • atan
      • atan2
      • atanh
      • bitwise_and
      • bitwise_invert
      • bitwise_left_shift
      • bitwise_or
      • bitwise_right_shift
      • bitwise_xor
      • ceil
      • cos
      • cosh
      • deg2rad
      • divide
      • equal
      • erf
      • exp
      • exp2
      • expm1
      • floor
      • floor_divide
      • fmin
      • fmod
      • gcd
      • greater
      • greater_equal
      • imag
      • isfinite
      • isinf
      • isnan
      • isreal
      • lcm
      • less
      • less_equal
      • log
      • log10
      • log1p
      • log2
      • logaddexp
      • logaddexp2
      • logical_and
      • logical_not
      • logical_or
      • logical_xor
      • maximum
      • minimum
      • multiply
      • nan_to_num
      • negative
      • not_equal
      • positive
      • pow
      • rad2deg
      • real
      • reciprocal
      • remainder
      • round
      • sign
      • sin
      • sinh
      • sqrt
      • square
      • subtract
      • tan
      • tanh
      • trapz
      • trunc
      • trunc_divide
    • Experimental
      • Activations
        • elu
        • hardtanh
        • logit
        • logsigmoid
        • prelu
        • relu6
        • selu
        • silu
        • thresholded_relu
      • Constants
      • Creation
        • blackman_window
        • eye_like
        • hamming_window
        • hann_window
        • indices
        • kaiser_bessel_derived_window
        • kaiser_window
        • mel_weight_matrix
        • ndenumerate
        • ndindex
        • random_cp
        • random_parafac2
        • random_tr
        • random_tt
        • random_tucker
        • tril_indices
        • trilu
        • unsorted_segment_min
        • unsorted_segment_sum
        • vorbis_window
      • Data type
      • Device
      • Elementwise
        • allclose
        • binarizer
        • conj
        • copysign
        • count_nonzero
        • diff
        • digamma
        • erfc
        • fix
        • float_power
        • fmax
        • frexp
        • gradient
        • hypot
        • isclose
        • ldexp
        • lerp
        • lgamma
        • modf
        • nansum
        • nextafter
        • signbit
        • sinc
        • sparsify_tensor
        • xlogy
        • zeta
      • General
        • reduce
      • Gradients
        • bind_custom_gradient_function
      • Layers
        • adaptive_avg_pool1d
        • adaptive_avg_pool2d
        • adaptive_max_pool2d
        • area_interpolate
        • avg_pool1d
        • avg_pool2d
        • avg_pool3d
        • dct
        • dft
        • dropout1d
        • dropout2d
        • dropout3d
        • embedding
        • fft
        • fft2
        • generate_einsum_equation
        • get_interpolate_kernel
        • idct
        • ifft
        • ifftn
        • interp
        • interpolate
        • max_pool1d
        • max_pool2d
        • max_pool3d
        • max_unpool1d
        • nearest_interpolate
        • pool
        • reduce_window
        • rfftn
        • sliding_window
        • stft
      • Linear algebra
        • adjoint
        • batched_outer
        • cond
        • diagflat
        • dot
        • eig
        • eigh_tridiagonal
        • eigvals
        • general_inner_product
        • higher_order_moment
        • initialize_tucker
        • khatri_rao
        • kron
        • kronecker
        • make_svd_non_negative
        • matrix_exp
        • mode_dot
        • multi_dot
        • multi_mode_dot
        • partial_tucker
        • svd_flip
        • truncated_svd
        • tt_matrix_to_tensor
        • tucker
      • Losses
        • huber_loss
        • kl_div
        • l1_loss
        • log_poisson_loss
        • poisson_nll_loss
        • smooth_l1_loss
        • soft_margin_loss
      • Manipulation
        • as_strided
        • associative_scan
        • atleast_1d
        • atleast_2d
        • atleast_3d
        • broadcast_shapes
        • choose
        • column_stack
        • concat_from_sequence
        • dsplit
        • dstack
        • expand
        • fill_diagonal
        • flatten
        • fliplr
        • flipud
        • fold
        • heaviside
        • hsplit
        • hstack
        • i0
        • matricize
        • moveaxis
        • pad
        • partial_fold
        • partial_tensor_to_vec
        • partial_unfold
        • partial_vec_to_tensor
        • put_along_axis
        • rot90
        • soft_thresholding
        • take_along_axis
        • top_k
        • unfold
        • unique_consecutive
        • vsplit
        • vstack
      • Meta
      • Nest
      • Norms
        • batch_norm
        • group_norm
        • instance_norm
        • l1_normalize
        • l2_normalize
        • lp_normalize
      • Random
        • bernoulli
        • beta
        • dirichlet
        • gamma
        • poisson
      • Searching
        • unravel_index
      • Set
      • Sorting
        • invert_permutation
        • lexsort
      • Sparse array
        • is_ivy_sparse_array
        • is_native_sparse_array
        • native_sparse_array
        • native_sparse_array_to_indices_values_and_shape
      • Statistical
        • bincount
        • corrcoef
        • cov
        • cummax
        • cummin
        • histogram
        • igamma
        • median
        • nanmean
        • nanmedian
        • nanprod
        • quantile
      • Utility
        • optional_get_element
    • General
      • all_equal
      • arg_info
      • arg_names
      • array_equal
      • assert_supports_inplace
      • cache_fn
      • clip_matrix_norm
      • clip_vector_norm
      • container_types
      • current_backend_str
      • default
      • einops_rearrange
      • einops_reduce
      • einops_repeat
      • exists
      • fourier_encode
      • function_supported_devices_and_dtypes
      • function_unsupported_devices_and_dtypes
      • gather
      • gather_nd
      • get_all_arrays_in_memory
      • get_item
      • get_num_dims
      • get_referrers_recursive
      • has_nans
      • inplace_arrays_supported
      • inplace_decrement
      • inplace_increment
      • inplace_update
      • inplace_variables_supported
      • is_array
      • is_ivy_array
      • is_ivy_container
      • is_ivy_nested_array
      • is_native_array
      • isin
      • isscalar
      • itemsize
      • match_kwargs
      • multiprocessing
      • num_arrays_in_memory
      • print_all_arrays_in_memory
      • scatter_flat
      • scatter_nd
      • set_array_mode
      • set_exception_trace_mode
      • set_inplace_mode
      • set_item
      • set_min_base
      • set_min_denominator
      • set_nestable_mode
      • set_precise_mode
      • set_queue_timeout
      • set_shape_array_mode
      • set_show_func_wrapper_trace_mode
      • set_tmp_dir
      • shape
      • stable_divide
      • stable_pow
      • strides
      • supports_inplace_updates
      • to_ivy_shape
      • to_list
      • to_native_shape
      • to_numpy
      • to_scalar
      • try_else_none
      • unset_array_mode
      • unset_exception_trace_mode
      • unset_inplace_mode
      • unset_min_base
      • unset_min_denominator
      • unset_nestable_mode
      • unset_precise_mode
      • unset_queue_timeout
      • unset_shape_array_mode
      • unset_show_func_wrapper_trace_mode
      • unset_tmp_dir
      • value_is_nan
      • vmap
    • Gradients
      • adam_step
      • adam_update
      • execute_with_gradients
      • grad
      • gradient_descent_update
      • jac
      • lamb_update
      • lars_update
      • optimizer_update
      • stop_gradient
      • value_and_grad
    • Layers
      • conv
      • conv1d
      • conv1d_transpose
      • conv2d
      • conv2d_transpose
      • conv3d
      • conv3d_transpose
      • conv_general_dilated
      • conv_general_transpose
      • depthwise_conv2d
      • dropout
      • linear
      • lstm_update
      • multi_head_attention
      • nms
      • roi_align
      • scaled_dot_product_attention
    • Linear algebra
      • cholesky
      • cross
      • det
      • diag
      • diagonal
      • eig
      • eigh
      • eigvalsh
      • inner
      • inv
      • lu_factor
      • matmul
      • matrix_norm
      • matrix_power
      • matrix_rank
      • matrix_transpose
      • outer
      • pinv
      • qr
      • slogdet
      • solve
      • svd
      • svdvals
      • tensordot
      • tensorsolve
      • trace
      • vander
      • vecdot
      • vector_norm
      • vector_to_skew_symmetric_matrix
    • Losses
      • binary_cross_entropy
      • cross_entropy
      • sparse_cross_entropy
    • Manipulation
      • clip
      • concat
      • constant_pad
      • expand_dims
      • flip
      • permute_dims
      • repeat
      • reshape
      • roll
      • split
      • squeeze
      • stack
      • swapaxes
      • tile
      • unstack
      • zero_pad
    • Meta
      • fomaml_step
      • maml_step
      • reptile_step
    • Nest
      • all_nested_indices
      • copy_nest
      • duplicate_array_index_chains
      • index_nest
      • insert_into_nest_at_index
      • insert_into_nest_at_indices
      • map
      • map_nest_at_index
      • map_nest_at_indices
      • multi_index_nest
      • nested_any
      • nested_argwhere
      • nested_map
      • nested_multi_map
      • prune_empty
      • prune_nest_at_index
      • prune_nest_at_indices
      • set_nest_at_index
      • set_nest_at_indices
    • Norms
      • layer_norm
    • Random
      • multinomial
      • randint
      • random_normal
      • random_uniform
      • seed
      • shuffle
    • Searching
      • argmax
      • argmin
      • argwhere
      • nonzero
      • where
    • Set
      • unique_all
      • unique_counts
      • unique_inverse
      • unique_values
    • Sorting
      • argsort
      • msort
      • searchsorted
      • sort
    • Statistical
      • cumprod
      • cumsum
      • einsum
      • max
      • mean
      • min
      • prod
      • std
      • sum
      • var
    • Utility
      • all
      • any
      • load
      • save
  • Data classes
    • Array
      • Activations
      • Conversions
      • Creation
      • Data type
      • Device
      • Elementwise
      • Experimental
      • General
      • Gradients
      • Image
      • Layers
      • Linear algebra
      • Losses
      • Manipulation
      • Norms
      • Random
      • Searching
      • Set
      • Sorting
      • Statistical
      • Utility
      • Wrapping
    • Container
      • Activations
      • Base
      • Conversions
      • Creation
      • Data type
      • Device
      • Elementwise
      • Experimental
      • General
      • Gradients
      • Image
      • Layers
      • Linear algebra
      • Losses
      • Manipulation
      • Norms
      • Random
      • Searching
      • Set
      • Sorting
      • Statistical
      • Utility
      • Wrapping
    • Factorized tensor
      • Base
      • Cp tensor
      • Parafac2 tensor
      • Tr tensor
      • Tt tensor
      • Tucker tensor
    • Nested array
      • Base
      • Elementwise
  • Framework classes
    • Activations
    • Converters
    • Helpers
    • Initializers
    • Layers
    • Losses
    • Module
    • Norms
    • Optimizers
    • Sequential
  • Utils
    • Assertions
    • Backend
      • Ast helpers
      • Handler
      • Sub backend handler
    • Binaries
    • Dynamic import
    • Einsum parser
    • Exceptions
    • Inspection
    • Logging
    • Profiler
    • Verbosity
  • Testing
    • Assertions
    • Available frameworks
    • Function testing
    • Globals
    • Hypothesis helpers
      • Array helpers
      • Dtype helpers
      • General helpers
      • Number helpers
    • Multiprocessing
    • Pipeline helper
    • Structs
    • Test parameter flags
    • Testing helpers
  • Related Work

Related Work#

In this section, we explain how Ivy compares to many other very important and related pieces of work, which also address fragmentation but at other areas within the ML stack.

Firstly, we need to look at the overall ML stack, and understand how the high level frameworks relate to the low level components.

In order to conceptualize this rather complex hierarchy, we have broken the ML stack into 9 groups, which are: RWorks API Standards, RWorks Wrapper Frameworks, RWorks Frameworks, RWorks Graph Tracers, RWorks Exchange Formats, RWorks Compiler Infrastructure, RWorks Multi-Vendor Compiler Frameworks, RWorks Vendor-Specific APIs and RWorks Vendor-Specific Compilers, going from high level to low level respectively.

https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/related_work/ml_stack.png?raw=true

Each of these groups within the ML stack has it’s own sub-section, linked below, within which we discuss various related projects which operate at that particular level within the stack.

We then compare Ivy to some other ML-unifying companies which are working on very important problems and are helping to unify the lower levels of the ML stack. We see these efforts as being very complimentary to Ivy’s vision for high level unification.

Finally, we discuss how Ivy compares to each of these important works at all levels within the ML stack.

(a) RWorks API Standards 🀝🏽
Standardized APIs which similar libraries should adhere to

(b) RWorks Wrapper Frameworks 🎁
Frameworks which wrap other ML frameworks

(c) RWorks Frameworks πŸ”’
Standalone ML Frameworks

(d) RWorks Graph Tracers πŸ•ΈοΈ
Extracting acyclic directed computation graphs from code

(e) RWorks Exchange Formats πŸ’±
File formats to exchange neural networks between frameworks

(f) RWorks Compiler Infrastructure πŸ”ŸοΈπŸ—οΈ
Infrastructure and standards to simplify the lives of compiler designers

(g) RWorks Multi-Vendor Compiler Frameworks πŸ–₯οΈπŸ’»πŸ”Ÿ
Executing ML code on a variety of hardware targets

(h) RWorks Vendor-Specific APIs πŸ’»πŸ”’
Interfacing with specific hardware in an intuitive manner

(i) RWorks Vendor-Specific Compilers πŸ’»πŸ”Ÿ
Compiling code to specific hardware

(j) RWorks ML-Unifying Companies πŸ“ˆ
Companies working towards unification in ML

(k) RWorks What does Ivy Add? 🟒
How does Ivy fit into all of this?

previous

Standardization

next

API Standards

Β© Copyright 2020-2023, Ivy Team.

Created using Sphinx 7.2.5.

Built with the PyData Sphinx Theme 0.13.3.