Function testing#

ivy_tests.test_ivy.helpers.function_testing.args_to_container(array_args)[source]#
ivy_tests.test_ivy.helpers.function_testing.args_to_frontend(*args, frontend_array_fn=None, include_derived=None, **kwargs)[source]#
ivy_tests.test_ivy.helpers.function_testing.arrays_to_frontend(frontend_array_fn=None)[source]#
ivy_tests.test_ivy.helpers.function_testing.as_lists(*args)[source]#

Change the elements in args to be of type list.

ivy_tests.test_ivy.helpers.function_testing.compiled_if_required(fn, test_compile=False, args=None, kwargs=None)[source]#
ivy_tests.test_ivy.helpers.function_testing.convtrue(argument)[source]#

Convert NativeClass in argument to true framework counter part.

ivy_tests.test_ivy.helpers.function_testing.create_args_kwargs(*, args_np, arg_np_vals, args_idxs, kwargs_np, kwarg_np_vals, kwargs_idxs, input_dtypes, test_flags, on_device)[source]#

Create arguments and keyword-arguments for the function to test.

Parameters:

  • args_np – A dictionary of arguments in Numpy.

  • kwargs_np – A dictionary of keyword-arguments in Numpy.

  • input_dtypes – data-types of the input arguments and keyword-arguments.

Returns:

Backend specific arguments, keyword-arguments

ivy_tests.test_ivy.helpers.function_testing.empty_func(*args, **kwargs)[source]#
ivy_tests.test_ivy.helpers.function_testing.flatten(*, ret)[source]#

Return a flattened numpy version of the arrays in ret.

ivy_tests.test_ivy.helpers.function_testing.flatten_and_to_np(*, ret)[source]#
ivy_tests.test_ivy.helpers.function_testing.flatten_frontend(*, ret, frontend_array_fn=None)[source]#

Return a flattened numpy version of the frontend arrays in ret.

ivy_tests.test_ivy.helpers.function_testing.flatten_frontend_to_np(*, ret, frontend_array_fn=None)[source]#
ivy_tests.test_ivy.helpers.function_testing.flatten_fw_and_to_np(*, ret, fw)[source]#

Return a flattened numpy version of the arrays in ret for a given framework.

ivy_tests.test_ivy.helpers.function_testing.get_frontend_ret(frontend_fn, *args, as_ivy_arrays=True, **kwargs)[source]#
ivy_tests.test_ivy.helpers.function_testing.get_ret_and_flattened_np_array(fn, *args, test_compile=False, **kwargs)[source]#

Run func with args and kwargs.

Return the result along with its flattened version.

ivy_tests.test_ivy.helpers.function_testing.gradient_incompatible_function(*, fn)[source]#
ivy_tests.test_ivy.helpers.function_testing.gradient_test(*, fn, all_as_kwargs_np, args_np, kwargs_np, input_dtypes, test_flags, test_compile=False, rtol_=None, atol_=1e-06, xs_grad_idxs=None, ret_grad_idxs=None, ground_truth_backend, on_device)[source]#
ivy_tests.test_ivy.helpers.function_testing.gradient_unsupported_dtypes(*, fn)[source]#
ivy_tests.test_ivy.helpers.function_testing.kwargs_to_args_n_kwargs(*, num_positional_args, kwargs)[source]#

Split the kwargs into args and kwargs.

The first num_positional_args ported to args.

ivy_tests.test_ivy.helpers.function_testing.test_frontend_function(*, input_dtypes, test_flags, on_device='cpu', frontend, fn_tree, rtol=None, atol=1e-06, test_values=True, **all_as_kwargs_np)[source]#

Test a frontend function for the current backend by comparing the result with the function in the associated framework.

Parameters:

  • input_dtypes (Union[Dtype, List[Dtype]]) – data types of the input arguments in order.

  • all_aliases – a list of strings containing all aliases for that function in the current frontend with their full namespaces.

  • frontend (str) – current frontend (framework).

  • fn_tree (str) – Path to function in frontend framework namespace.

  • rtol (Optional[float]) – relative tolerance value. (default: None)

  • atol (float) – absolute tolerance value. (default: 1e-06)

  • test_values (bool) – if True, test for the correctness of the resulting values. (default: True)

  • all_as_kwargs_np – input arguments to the function as keyword arguments.

Returns:

  • ret – optional, return value from the function

  • ret_np – optional, return value from the Numpy function

ivy_tests.test_ivy.helpers.function_testing.test_frontend_method(*, init_input_dtypes=None, method_input_dtypes, init_flags, method_flags, init_all_as_kwargs_np=None, method_all_as_kwargs_np, frontend, frontend_method_data, on_device, rtol_=None, atol_=1e-06, test_values=True)[source]#

Test a class-method that consumes (or returns) arrays for the current backend by comparing the result with numpy.

Parameters:

  • init_input_dtypes (Optional[Union[Dtype, List[Dtype]]]) – data types of the input arguments to the constructor in order. (default: None)

  • init_as_variable_flags – dictates whether the corresponding input argument passed to the constructor should be treated as an ivy.Variable.

  • init_num_positional_args – number of input arguments that must be passed as positional arguments to the constructor.

  • init_native_array_flags – dictates whether the corresponding input argument passed to the constructor should be treated as a native array.

  • init_all_as_kwargs_np (Optional[dict]) – input arguments to the constructor as keyword arguments. (default: None)

  • method_input_dtypes (Union[Dtype, List[Dtype]]) – data types of the input arguments to the method in order.

  • method_all_as_kwargs_np (dict) – input arguments to the method as keyword arguments.

  • frontend (str) – current frontend (framework).

  • rtol – relative tolerance value.

  • atol – absolute tolerance value.

  • test_values (Union[bool, str]) – can be a bool or a string to indicate whether correctness of values should be (default: True) tested. If the value is with_v, shapes are tested but not values.

Returns:

  • ret – optional, return value from the function

  • ret_gt – optional, return value from the Ground Truth function

ivy_tests.test_ivy.helpers.function_testing.test_function(*, input_dtypes, test_flags, fw, fn_name, rtol_=None, atol_=1e-06, test_values=True, xs_grad_idxs=None, ret_grad_idxs=None, ground_truth_backend, on_device, return_flat_np_arrays=False, **all_as_kwargs_np)[source]#

Test a function that consumes (or returns) arrays for the current backend by comparing the result with numpy.

Parameters:

  • input_dtypes (Union[Dtype, List[Dtype]]) – data types of the input arguments in order.

  • test_flags (FunctionTestFlags) – FunctionTestFlags object that stores all testing flags, including: num_positional_args, with_out, instance_method, as_variable, native_arrays, container, gradient

  • fw (str) – current backend (framework).

  • fn_name (str) – name of the function to test.

  • rtol – relative tolerance value.

  • atol – absolute tolerance value.

  • test_values (bool) – if True, test for the correctness of the resulting values. (default: True)

  • xs_grad_idxs – Indices of the input arrays to compute gradients with respect to. If None, gradients are returned with respect to all input arrays. (Default value = None)

  • ret_grad_idxs – Indices of the returned arrays for which to return computed gradients. If None, gradients are returned for all returned arrays. (Default value = None)

  • ground_truth_backend (str) – Ground Truth Backend to compare the result-values.

  • on_device (str) – The device on which to create arrays

  • return_flat_np_arrays (bool) – If test_values is False, this flag dictates whether the original returns are (default: False) returned, or whether the flattened numpy arrays are returned.

  • all_as_kwargs_np – input arguments to the function as keyword arguments.

Returns:

  • ret – optional, return value from the function

  • ret_gt – optional, return value from the Ground Truth function

Examples

>>> input_dtypes = 'float64'
>>> as_variable_flags = False
>>> with_out = False
>>> num_positional_args = 0
>>> native_array_flags = False
>>> container_flags = False
>>> instance_method = False
>>> test_flags = FunctionTestFlags(num_positional_args, with_out,
    instance_method,
    as_variable,
    native_arrays,
    container_flags,
    none)
>>> fw = "torch"
>>> fn_name = "abs"
>>> x = np.array([-1])
>>> test_function(input_dtypes, test_flags, fw, fn_name, x=x)

>>> input_dtypes = ['float64', 'float32']
>>> as_variable_flags = [False, True]
>>> with_out = False
>>> num_positional_args = 1
>>> native_array_flags = [True, False]
>>> container_flags = [False, False]
>>> instance_method = False
>>> test_flags = FunctionTestFlags(num_positional_args, with_out,
    instance_method,
    as_variable,
    native_arrays,
    container_flags,
    none)
>>> fw = "numpy"
>>> fn_name = "add"
>>> x1 = np.array([1, 3, 4])
>>> x2 = np.array([-3, 15, 24])
>>> test_function(input_dtypes, test_flags, fw, fn_name, x1=x1, x2=x2)
ivy_tests.test_ivy.helpers.function_testing.test_method(*, init_input_dtypes=None, method_input_dtypes=None, init_all_as_kwargs_np=None, method_all_as_kwargs_np=None, init_flags, method_flags, class_name, method_name='__call__', init_with_v=False, method_with_v=False, rtol_=None, atol_=1e-06, test_values=True, test_gradients=False, xs_grad_idxs=None, ret_grad_idxs=None, test_compile=False, ground_truth_backend, on_device, return_flat_np_arrays=False)[source]#

Test a class-method that consumes (or returns) arrays for the current backend by comparing the result with numpy.

Parameters:

  • init_input_dtypes (Optional[List[Dtype]]) – data types of the input arguments to the constructor in order. (default: None)

  • init_as_variable_flags – dictates whether the corresponding input argument passed to the constructor should be treated as an ivy.Array.

  • init_num_positional_args – number of input arguments that must be passed as positional arguments to the constructor.

  • init_native_array_flags – dictates whether the corresponding input argument passed to the constructor should be treated as a native array.

  • init_all_as_kwargs_np (Optional[dict]) – input arguments to the constructor as keyword arguments. (default: None)

  • method_input_dtypes (Optional[List[Dtype]]) – data types of the input arguments to the method in order. (default: None)

  • method_as_variable_flags – dictates whether the corresponding input argument passed to the method should be treated as an ivy.Array.

  • method_num_positional_args – number of input arguments that must be passed as positional arguments to the method.

  • method_native_array_flags – dictates whether the corresponding input argument passed to the method should be treated as a native array.

  • method_container_flags – dictates whether the corresponding input argument passed to the method should be treated as an ivy Container.

  • method_all_as_kwargs_np (Optional[dict]) – input arguments to the method as keyword arguments. (default: None)

  • class_name (str) – name of the class to test.

  • method_name (str) – name of tthe method to test. (default: '__call__')

  • init_with_v (bool) – if the class being tested is an ivy.Module, then setting this flag as True will (default: False) call the constructor with the variables v passed explicitly.

  • method_with_v (bool) – if the class being tested is an ivy.Module, then setting this flag as True will (default: False) call the method with the variables v passed explicitly.

  • rtol – relative tolerance value.

  • atol – absolute tolerance value.

  • test_values (Union[bool, str]) – can be a bool or a string to indicate whether correctness of values should be (default: True) tested. If the value is with_v, shapes are tested but not values.

  • test_gradients (bool) – if True, test for the correctness of gradients. (default: False)

  • xs_grad_idxs – Indices of the input arrays to compute gradients with respect to. If None, gradients are returned with respect to all input arrays. (Default value = None)

  • ret_grad_idxs – Indices of the returned arrays for which to return computed gradients. If None, gradients are returned for all returned arrays. (Default value = None)

  • test_compile (bool) – If True, test for the correctness of compilation. (default: False)

  • ground_truth_backend (str) – Ground Truth Backend to compare the result-values.

  • device – The device on which to create arrays.

  • return_flat_np_arrays (bool) – If test_values is False, this flag dictates whether the original returns are (default: False) returned, or whether the flattened numpy arrays are returned.

Returns:

  • ret – optional, return value from the function

  • ret_gt – optional, return value from the Ground Truth function

ivy_tests.test_ivy.helpers.function_testing.var_fn(x, *, dtype=None, device=None)[source]#

Return x as a variable wrapping an Ivy Array with given dtype and device.

ivy_tests.test_ivy.helpers.function_testing.wrap_frontend_function_args(argument)[source]#

Wrap frontend function arguments to return native arrays.