Function testing#
- ivy_tests.test_ivy.helpers.function_testing.args_to_frontend(*args, frontend_array_fn=None, include_derived=None, **kwargs)[source]#
- ivy_tests.test_ivy.helpers.function_testing.as_lists(*args)[source]#
Change the elements in args to be of type list.
- ivy_tests.test_ivy.helpers.function_testing.compiled_if_required(fn, test_compile=False, args=None, kwargs=None)[source]#
- ivy_tests.test_ivy.helpers.function_testing.convtrue(argument)[source]#
Convert NativeClass in argument to true framework counter part.
- ivy_tests.test_ivy.helpers.function_testing.create_args_kwargs(*, args_np, arg_np_vals, args_idxs, kwargs_np, kwarg_np_vals, kwargs_idxs, input_dtypes, test_flags, on_device)[source]#
Create arguments and keyword-arguments for the function to test.
- Parameters:
args_np – A dictionary of arguments in Numpy.
kwargs_np – A dictionary of keyword-arguments in Numpy.
input_dtypes – data-types of the input arguments and keyword-arguments.
- Returns:
Backend specific arguments, keyword-arguments
- ivy_tests.test_ivy.helpers.function_testing.flatten(*, ret)[source]#
Return a flattened numpy version of the arrays in ret.
- ivy_tests.test_ivy.helpers.function_testing.flatten_frontend(*, ret, frontend_array_fn=None)[source]#
Return a flattened numpy version of the frontend arrays in ret.
- ivy_tests.test_ivy.helpers.function_testing.flatten_frontend_to_np(*, ret, frontend_array_fn=None)[source]#
- ivy_tests.test_ivy.helpers.function_testing.flatten_fw_and_to_np(*, ret, fw)[source]#
Return a flattened numpy version of the arrays in ret for a given framework.
- ivy_tests.test_ivy.helpers.function_testing.get_frontend_ret(frontend_fn, *args, as_ivy_arrays=True, **kwargs)[source]#
- ivy_tests.test_ivy.helpers.function_testing.get_ret_and_flattened_np_array(fn, *args, test_compile=False, **kwargs)[source]#
Run func with args and kwargs.
Return the result along with its flattened version.
- ivy_tests.test_ivy.helpers.function_testing.gradient_test(*, fn, all_as_kwargs_np, args_np, kwargs_np, input_dtypes, test_flags, test_compile=False, rtol_=None, atol_=1e-06, xs_grad_idxs=None, ret_grad_idxs=None, ground_truth_backend, on_device)[source]#
- ivy_tests.test_ivy.helpers.function_testing.kwargs_to_args_n_kwargs(*, num_positional_args, kwargs)[source]#
Split the kwargs into args and kwargs.
The first num_positional_args ported to args.
- ivy_tests.test_ivy.helpers.function_testing.test_frontend_function(*, input_dtypes, test_flags, on_device='cpu', frontend, fn_tree, rtol=None, atol=1e-06, test_values=True, **all_as_kwargs_np)[source]#
Test a frontend function for the current backend by comparing the result with the function in the associated framework.
- Parameters:
input_dtypes (
Union
[Dtype
,List
[Dtype
]]) – data types of the input arguments in order.all_aliases – a list of strings containing all aliases for that function in the current frontend with their full namespaces.
frontend (
str
) – current frontend (framework).fn_tree (
str
) – Path to function in frontend framework namespace.rtol (
Optional
[float
]) – relative tolerance value. (default:None
)atol (
float
) – absolute tolerance value. (default:1e-06
)test_values (
bool
) – if True, test for the correctness of the resulting values. (default:True
)all_as_kwargs_np – input arguments to the function as keyword arguments.
- Returns:
ret – optional, return value from the function
ret_np – optional, return value from the Numpy function
- ivy_tests.test_ivy.helpers.function_testing.test_frontend_method(*, init_input_dtypes=None, method_input_dtypes, init_flags, method_flags, init_all_as_kwargs_np=None, method_all_as_kwargs_np, frontend, frontend_method_data, on_device, rtol_=None, atol_=1e-06, test_values=True)[source]#
Test a class-method that consumes (or returns) arrays for the current backend by comparing the result with numpy.
- Parameters:
init_input_dtypes (
Optional
[Union
[Dtype
,List
[Dtype
]]]) – data types of the input arguments to the constructor in order. (default:None
)init_as_variable_flags – dictates whether the corresponding input argument passed to the constructor should be treated as an ivy.Variable.
init_num_positional_args – number of input arguments that must be passed as positional arguments to the constructor.
init_native_array_flags – dictates whether the corresponding input argument passed to the constructor should be treated as a native array.
init_all_as_kwargs_np (
Optional
[dict
]) – input arguments to the constructor as keyword arguments. (default:None
)method_input_dtypes (
Union
[Dtype
,List
[Dtype
]]) – data types of the input arguments to the method in order.method_all_as_kwargs_np (
dict
) – input arguments to the method as keyword arguments.frontend (
str
) – current frontend (framework).rtol – relative tolerance value.
atol – absolute tolerance value.
test_values (
Union
[bool
,str
]) – can be a bool or a string to indicate whether correctness of values should be (default:True
) tested. If the value is with_v, shapes are tested but not values.
- Returns:
ret – optional, return value from the function
ret_gt – optional, return value from the Ground Truth function
- ivy_tests.test_ivy.helpers.function_testing.test_function(*, input_dtypes, test_flags, fw, fn_name, rtol_=None, atol_=1e-06, test_values=True, xs_grad_idxs=None, ret_grad_idxs=None, ground_truth_backend, on_device, return_flat_np_arrays=False, **all_as_kwargs_np)[source]#
Test a function that consumes (or returns) arrays for the current backend by comparing the result with numpy.
- Parameters:
input_dtypes (
Union
[Dtype
,List
[Dtype
]]) – data types of the input arguments in order.test_flags (
FunctionTestFlags
) – FunctionTestFlags object that stores all testing flags, including: num_positional_args, with_out, instance_method, as_variable, native_arrays, container, gradientfw (
str
) – current backend (framework).fn_name (
str
) – name of the function to test.rtol – relative tolerance value.
atol – absolute tolerance value.
test_values (
bool
) – if True, test for the correctness of the resulting values. (default:True
)xs_grad_idxs – Indices of the input arrays to compute gradients with respect to. If None, gradients are returned with respect to all input arrays. (Default value = None)
ret_grad_idxs – Indices of the returned arrays for which to return computed gradients. If None, gradients are returned for all returned arrays. (Default value = None)
ground_truth_backend (
str
) – Ground Truth Backend to compare the result-values.on_device (
str
) – The device on which to create arraysreturn_flat_np_arrays (
bool
) – If test_values is False, this flag dictates whether the original returns are (default:False
) returned, or whether the flattened numpy arrays are returned.all_as_kwargs_np – input arguments to the function as keyword arguments.
- Returns:
ret – optional, return value from the function
ret_gt – optional, return value from the Ground Truth function
Examples
>>> input_dtypes = 'float64' >>> as_variable_flags = False >>> with_out = False >>> num_positional_args = 0 >>> native_array_flags = False >>> container_flags = False >>> instance_method = False >>> test_flags = FunctionTestFlags(num_positional_args, with_out, instance_method, as_variable, native_arrays, container_flags, none) >>> fw = "torch" >>> fn_name = "abs" >>> x = np.array([-1]) >>> test_function(input_dtypes, test_flags, fw, fn_name, x=x)
>>> input_dtypes = ['float64', 'float32'] >>> as_variable_flags = [False, True] >>> with_out = False >>> num_positional_args = 1 >>> native_array_flags = [True, False] >>> container_flags = [False, False] >>> instance_method = False >>> test_flags = FunctionTestFlags(num_positional_args, with_out, instance_method, as_variable, native_arrays, container_flags, none) >>> fw = "numpy" >>> fn_name = "add" >>> x1 = np.array([1, 3, 4]) >>> x2 = np.array([-3, 15, 24]) >>> test_function(input_dtypes, test_flags, fw, fn_name, x1=x1, x2=x2)
- ivy_tests.test_ivy.helpers.function_testing.test_method(*, init_input_dtypes=None, method_input_dtypes=None, init_all_as_kwargs_np=None, method_all_as_kwargs_np=None, init_flags, method_flags, class_name, method_name='__call__', init_with_v=False, method_with_v=False, rtol_=None, atol_=1e-06, test_values=True, test_gradients=False, xs_grad_idxs=None, ret_grad_idxs=None, test_compile=False, ground_truth_backend, on_device, return_flat_np_arrays=False)[source]#
Test a class-method that consumes (or returns) arrays for the current backend by comparing the result with numpy.
- Parameters:
init_input_dtypes (
Optional
[List
[Dtype
]]) – data types of the input arguments to the constructor in order. (default:None
)init_as_variable_flags – dictates whether the corresponding input argument passed to the constructor should be treated as an ivy.Array.
init_num_positional_args – number of input arguments that must be passed as positional arguments to the constructor.
init_native_array_flags – dictates whether the corresponding input argument passed to the constructor should be treated as a native array.
init_all_as_kwargs_np (
Optional
[dict
]) – input arguments to the constructor as keyword arguments. (default:None
)method_input_dtypes (
Optional
[List
[Dtype
]]) – data types of the input arguments to the method in order. (default:None
)method_as_variable_flags – dictates whether the corresponding input argument passed to the method should be treated as an ivy.Array.
method_num_positional_args – number of input arguments that must be passed as positional arguments to the method.
method_native_array_flags – dictates whether the corresponding input argument passed to the method should be treated as a native array.
method_container_flags – dictates whether the corresponding input argument passed to the method should be treated as an ivy Container.
method_all_as_kwargs_np (
Optional
[dict
]) – input arguments to the method as keyword arguments. (default:None
)class_name (
str
) – name of the class to test.method_name (
str
) – name of tthe method to test. (default:'__call__'
)init_with_v (
bool
) – if the class being tested is an ivy.Module, then setting this flag as True will (default:False
) call the constructor with the variables v passed explicitly.method_with_v (
bool
) – if the class being tested is an ivy.Module, then setting this flag as True will (default:False
) call the method with the variables v passed explicitly.rtol – relative tolerance value.
atol – absolute tolerance value.
test_values (
Union
[bool
,str
]) – can be a bool or a string to indicate whether correctness of values should be (default:True
) tested. If the value is with_v, shapes are tested but not values.test_gradients (
bool
) – if True, test for the correctness of gradients. (default:False
)xs_grad_idxs – Indices of the input arrays to compute gradients with respect to. If None, gradients are returned with respect to all input arrays. (Default value = None)
ret_grad_idxs – Indices of the returned arrays for which to return computed gradients. If None, gradients are returned for all returned arrays. (Default value = None)
test_compile (
bool
) – If True, test for the correctness of compilation. (default:False
)ground_truth_backend (
str
) – Ground Truth Backend to compare the result-values.device – The device on which to create arrays.
return_flat_np_arrays (
bool
) – If test_values is False, this flag dictates whether the original returns are (default:False
) returned, or whether the flattened numpy arrays are returned.
- Returns:
ret – optional, return value from the function
ret_gt – optional, return value from the Ground Truth function