MLX Array API compatibility: upstream status, compatibility

@ev-br

MLX Array API compatibility overview

This issue tracks the compatibility of MLX with the Python Array API
standard and coordinates potential changes across:

MLX
array-api-compat
array-api-tests
the Array API specification

Scope

Array API version:
MLX version/commit:
array-api-tests commit:
array-api-compat branch/commit:
Platform: macos

Compatibility tracking:

Constants, Datatypes and Array Attributes MLX compatibility: Constants, Datatypes and Array Attributes #451
Creation Methods MLX compatibility: Creation functions #452
Manipulation Methods MLX compatibility: Manipulation functions #462
Statistical functions MLX compatibility: Statistical functions #463
Operators and elementwise functions: MLX compatibility: operators and elementwise functions #465
Indexing functions: MLX compatibility: indexing functions #466

The desired endgame

Upstream has a CI job which runs array-api-tests with a list of known failures.

The test suite needs to run with ARRAY_API_TESTS_SKIP_DTYPES=float64 environment variable, and --disable-data-dependent-shapes. For example,

ARRAY_API_TESTS_SKIP_DTYPES=float64 ARRAY_API_TESTS_MODULE=mlx.core pytest array_api_tests/test_searching_functions.py -v --disable-data-dependent-shapes --max-examples=1000

Process

Use this issue an umrella issue, open individual issues for groups of tests from the test suite, edit the links into this top post into the list above. (If you cannot edit it, ping @ev-br)

Problems/failures with individual functions, as uncovered by the test suite: discuss in the sub-issues. Keep this one for general blockers.

Known blockers

missing __array_namespace_info__ : would be best to submit a PR upstream
missing .device attribute of the array object : nothing to do about it

@ev-br

Adding more info to this tracker:

array-api-compat: Atleast __array_namespace_info__() is required as per standard which is not available in MLX. This has to be the first step addition to the compat library.
array-api-tests: Changes needed (which are also needed for us to be able to at least run the test suite with MLX) are mentioned in this comment: None dtype comparison not supported in MLX array-api-tests#447 (comment), and the PR: None dtype comparison not supported in MLX array-api-tests#447 itself needs to be addressed.

With that said, running the test suite right now (skipping array_namespace_info and adding rest of the changes) looks something like this :

484 failed, 953 passed, 7 skipped, 64 warnings

As suggested by @ev-br, trying to find a breadth-first summary of the failures (and this is from AI):

Root error / signature	Approx. affected tests	Fix count
`TypeError: float() argument must be a string or a real number, not 'complex'`	100+	One major root cause
Missing functions (`AssertionError: ... is not defined` / `module has no attribute`)	~60	Mostly adding wrappers/implementations
Signature mismatches (`Argument 'axis' missing`, wrong parameter names, etc.)	~70	One API compliance sweep
`TypeError: incompatible function arguments`	~15	Similar wrapper/signature fixes
Missing array methods/attributes (`__index__`, `__complex__`, `device`, `mT`, `to_device`)	~10	Small API additions
Missing info namespace (`__array_namespace_info__`)	~6	One implementation
Missing metadata (`iinfo.bits`, etc.)	~10	One implementation
`Hypothesis found 2 distinct failures`	~8	Needs investigation individually
Numerical assertion failures	~10	Independent algorithm bugs
RuntimeError loading Metal kernels	~40	One backend/kernel issue

Most of the test failures are because MLX has no __complex__ method on its array type, so every attempt to pull a Python complex scalar out of a complex-dtype MLX array falls back to trying __float__ instead (that's standard CPython behavior for the complex() constructor, not a bug in the test) and blows up.

Some of these failures, I believe, are also a fix in the array-api-tests itself. I tried changing complex(x) -> complex(float(xp.real(x)), float(xp.imag(x))) in some places and now we are left with 415 failures (all of the complex failures were not removed. It might make sense for MLX to xfail these but not sure yet!)

I also found a seg fault in this process (discovered via hypothesis):

import mlx.core as mlx

x = mlx.zeros(())
mlx.expand_dims(x, axis=(-3, -2))

Gives:

[1]    87881 segmentation fault

Reporting this to mlx. The test suite is already uncovering bugs in mlx!

I'll continue to keep adding more info to this report.

Edit: Just saw ml-explore/mlx#3984, might help solve the complex issue mentioned above.

Hi, I think it would be better to instead of running all the tests together make sub-issues for different categories for easier tracking :)

@aaishwarymishra

Hi, I think it would be better to instead of running all the tests together make sub-issues for different categories for easier tracking

Hi, sorry, I don't understand. The above table has some categories. I haven't really worked on any failures yet, just pointed some out. The fix that I suggested was just to unblock the test suite and the ones that were bugs in MLX itself.

All the complex failures are gone now after the upstream PR merge in MLX! That means all the

float() argument must be a string or a real number, not 'complex'

errors are gone. Thanks for the effort @aaishwarymishra! Which means now we have

289 failed, 1146 passed

Hypothesis found another bug in MLX.

import mlx.core as mx
a = mx.zeros((0, 0))
r = mx.linalg.cholesky(a)
mx.eval(r)   # process aborts here

Output:

** On entry to  POTRF, parameter number  4 had an illegal value
libc++abi: terminating due to uncaught exception of type std::runtime_error: [Cholesky::eval_cpu] Cholesky decomposition failed with error code -4
[1]    31154 abort

Reporting it too.

@ev-br Should we add __array_namespace_info__() in compat? Currently, I run the tests by:

if not hasattr(xp, "__array_namespace_info__"):
    pytest.skip(
        "__array_namespace_info__ is not defined in the array module",
        allow_module_level=True,
    )

@aaishwarymishra

Thanks for pointing out failure categories!
My suggestion is to work through them in passes: you did the first pass, with categories, great. Next pass is to list individual failures per group of functions, in a style similar to #452, preferably with reproducible examples that the test suite emits. Then we can quickly triage them. And then for those which are easy-fix in MLX, send PRs to MLX. Observing @aaishwarymishra 's recent MLX PRs, it looks like they are receptive to PRs and are ready to merge them quickly. Opening issues upstream not sure, especially for small items like empty(0) or cholesky((0,0)).

Should we add __array_namespace_info__() in compat?

If it helps working through unrelated items, yes why not. We won't merge MLX compat PRs not yet, so you could work off a -compat branch with __array_namespace_info__() if that makes it easier.

list individual failures per group of functions

Makes sense. Thanks for the suggestion. Here's a tracker for all failing functions along with their failures. I'll add more to this table.

Function(s)	Failure	Minimal reproducer
`__getitem__`	Boolean indexing unsupported
`__setitem__`	Indexing semantics incompatible
`__index__`	Missing method	`mx.asarray(1).__index__()`
`__pos__`	Missing unary positive	`+mx.asarray(1)`
`to_device`	Missing method
`device`	Missing attribute	`mx.asarray(1).device`
`mT`	Missing attribute	`mx.ones((2,2)).mT`
`__array_namespace__`	Missing `api_version` parameter
`__array_namespace_info__`	Missing implementation	`mx.__array_namespace_info__()`
`__dlpack__`	Missing `stream` parameter
`arange`	Signature mismatch
`asarray`	Copy/dtype semantics
`empty`	Signature mismatch
`empty_like`	Missing `dtype`
`eye`	Missing `k` parameter
`full`	Signature mismatch
`full_like`	Missing `fill_value`
`linspace`	Missing `num`
`meshgrid`	Returns `list` instead of `tuple`	`type(mx.meshgrid(mx.arange(3)))`
`ones`	Signature mismatch
`ones_like`	Missing `dtype`
`zeros`	Signature mismatch
`zeros_like`	Missing `dtype`
`astype`	Signature mismatch
`broadcast_arrays`	Returns `list` instead of `tuple`
`broadcast_shapes`	Empty input handling	`mx.broadcast_shapes()`
`can_cast`	Incorrect dtype behavior
`finfo`	Constructor/signature issue
`iinfo`	Missing `bits` attribute	`mx.iinfo(mx.int32).bits`
`result_type`	Scalar handling differs
`from_dlpack`	Signature mismatch
`fft`, `ifft`, `fftn`, `ifftn`, `rfft`, `irfft`, `rfftn`, `irfftn`	Missing signature parameters
`hfft`, `ihfft`	Missing implementation	`mx.fft.hfft(...)`
`fftfreq`, `rfftfreq`	Missing `d` parameter / behavior
`fftshift`, `ifftshift`	Missing `axes` parameter
`take_along_axis`	Shape/signature mismatch
`expand_dims`	Invalid axis handling / segfault	`mx.expand_dims(mx.zeros(()), axis=(-3, -2))`
`moveaxis`	Missing `destination`
`repeat`	Behavioral failures
`reshape`	Signature mismatch
`unstack`	Incorrect behavior
`broadcast_to`	Signature mismatch
`concat`	Missing `axis`
`diff`	Missing `axis`
`flip`	Missing `axis`
`permute_dims`	Signature mismatch
`roll`	Signature mismatch
`squeeze`	Signature mismatch
`stack`	Missing `axis`
`cholesky`	Missing `upper`; aborts on empty `(0,0)`	`mx.linalg.cholesky(mx.zeros((0,0)))`
`cross`	Missing `axis`
`det`	Complex inputs unsupported
`diagonal`	Missing implementation	`mx.linalg.diagonal(...)`
`eig`	Wrong return type
`eigh`	Behavioral failures
`eigvalsh`	Incorrect handling
`inv`	Complex inputs unsupported
`matmul`	Integer dtype behavior differs
`matrix_norm`	Missing implementation	`mx.linalg.matrix_norm(mx.eye(3))`
`matrix_power`	Missing implementation	`mx.linalg.matrix_power(mx.eye(2), 2)`
`matrix_rank`	Missing implementation	`mx.linalg.matrix_rank(mx.eye(3))`
`matrix_transpose`	Missing implementation / wrong shape
`outer`	Missing implementation	`mx.linalg.outer(...)`
`pinv`	Missing `rtol` / behavioral failures
`qr`	Missing `mode` / behavioral failures
`slogdet`	Behavioral failures
`solve`	Shape/indexing failures
`svd`	Missing `full_matrices` / behavioral failures
`svdvals`	Missing implementation	`mx.linalg.svdvals(...)`
`tensordot`	Missing implementation	`mx.linalg.tensordot(...)`
`trace`	Missing implementation	`mx.linalg.trace(...)`
`vecdot`	Missing implementation	`mx.linalg.vecdot(...)`
`vector_norm`	Missing implementation	`mx.linalg.vector_norm(...)`
`argmax`	Wrong output dtype
`argmin`	Wrong output dtype
`count_nonzero`	Missing `axis`
`nonzero`	Missing implementation	`mx.nonzero(...)`
`searchsorted`	Missing implementation	`mx.searchsorted(...)`
`argsort`	Missing `axis`; behavioral failures
`sort`	Missing `axis`; behavioral failures
`isin`	Missing implementation	`mx.isin(...)`
`unique_all`	Missing implementation
`unique_counts`	Missing implementation
`unique_inverse`	Missing implementation
`unique_values`	Missing implementation
`all`, `any`, `max`, `mean`, `min`, `prod`, `std`, `sum`, `var`, `cumulative_sum`, `cumulative_prod`	Missing `axis` parameter and/or behavioral failures
`acos`, `acosh`, `asin`, `asinh`, `cos`, `sin`, `tanh`	Incorrect numerical/special-case behavior
`atan2`	Signature mismatch
`bitwise_left_shift`, `bitwise_right_shift`, `bitwise_xor`	Incorrect results
`clip`	Signature mismatch
`copysign`	Missing implementation	`mx.copysign(...)`
`expm1`	Incorrect complex special-case behavior
`floor_divide`	Incorrect results
`hypot`	Missing implementation	`mx.hypot(...)`
`nextafter`	Missing implementation	`mx.nextafter(...)`
`remainder`	Incorrect signed-zero behavior
`sign`	Incorrect NaN handling
`signbit`	Missing implementation	`mx.signbit(...)`

@aaishwarymishra

Thanks, this is very helpful.

Would be great to continue filling in missing MWEs.
Some of these items, esp creation functions, are being worked on in #452

Quick triage:

leave out things with data-dependent shapes (nonzero, unique_all etc)
leave out device-related issues
special cases for numerical functions (infs nans and such): we maintain lists of xfails for these for essentially all wrapped libraries, so can leave to the end
several items look uncontroversial and free from backwards compatible considerations (__pos__, __index__, mT, hypot, empty inputs). Are you up to sending upstream PRs for these and related omissions? Would be best to coordinate with @aaishwarymishra
simple signature tweaks, like renaming/permuting arguments can be done in the compat layer
missing axis arguments: I actually wonder what it takes to implement it in MLX.
some of the listed "signature mismatch" I'm not sure about. Take broadcast_to as an example. What is the mismatch? The MLX signature looks to be a proper superset of the Array API signature, modulo tuple vs sequence annotation (which can be ignored IMO):

broadcast_to(x: array,          /, shape: Tuple[int, ...]) → array
broadcast_to(a: scalar | array, /, shape: Sequence[int], *, stream: None | Stream | Device = None) → array

Re missing axis argument in reductions: the docs list it, at least for some of them, eg. https://ml-explore.github.io/mlx/build/html/python/_autosummary/mlx.core.any.html#mlx.core.any

Is this version-dependent, or are there some behaviour differences that the test suite catches?

This has to be a Python version-specific issue. I ran the tests till now on Python 3.14 (inside a conda environment), and it failed for function signatures even though they were present in the nanobind C-extension function (nb::sig): https://github.com/ml-explore/mlx/blob/8056817bd1202477b6bed2b1f871da5960d64b28/python/src/ops.cpp#L1086

This kind of function signature is handled here:
https://github.com/data-apis/array-api-tests/blob/4b42cafee2e6a119e51a7178215aaba63e4663e9/array_api_tests/test_signatures.py#L265

And they are expected to raise a ValueError.

Here's the catch: on my local machines, Python 3.12.2 (from python.org), it passes. Some chance is, inspect.signature handling has been relaxed? I'm not sure, and I need to investigate it a bit more. Here's the CPython section that might be handling this: https://github.com/python/cpython/blob/115400b5090f14233b72b255e483b4485c868100/Lib/inspect.py#L2436

Should I raise an issue in CPython?

Also, with a workaround, I'm able to fix this by adding a condition in the test suite, and we are down to just 240 failures! All the axis not found and a few other function signature complaints are just inconsistencies.

The workaround (kuddos to Claude Code, I haven't tested it too well):

def _is_uninformative_signature(sig: Signature) -> bool:
    # Some Python versions (e.g. 3.14) don't raise ValueError for callables
    # backed by extension modules (e.g. nanobind, pybind11) that lack a
    # parseable text signature. Instead inspect.signature() falls back to a
    # generic "(*args, **kwargs)" signature which carries no real parameter
    # information, so we treat it the same as an uninspectable signature.
    params = list(sig.parameters.values())
    return len(params) == 2 and [p.kind for p in params] == [
        Parameter.VAR_POSITIONAL,
        Parameter.VAR_KEYWORD,
    ]

Should I raise an issue in CPython?

Maybe. At the moment though, there seem to be too many moving parts: array-api-tests, mlx, nanobind and inspect.signature. It's really not yet clear where the problem is, not to me at least.
It'd be best to move the two first items out of the equation: cook up a dummy nanobind-wrapped module and see if the inspect problem reproduces in the same python version dependent way. If it does, there's still a question whether it's in inspect or in nanobind, but at least we know it's upstream not us.

I don't think it's a nanobind issue. I created a minimal example and ran it on different versions of nanobind and Python in CI. See.

The results are consistent with nanobind v2.0.0 (2 years old), v2.13.0 (what mlx currently pins) and the latest.

It fails for these Python versions starting from 3.11.0 (The failure here means not raising the ValueError that our test suite expects):

3.11.9
3.11.13
3.12.3
3.12.10
3.13.0
3.13.9
3.14.0

Passes for:

3.11.0
3.11.8
3.12.0
3.12.2

Let's take a step back, sorry I should have asked this earlier. What is the expected behavior and what is the actual behavior we are seeing? Is it that some signatures were not parseable with inspect.signature in older versions of python/nanobind, and are parseable with newer versions? If so, that's an improvement. Or is it the other way around, some signatures were parseable on old versions and newer versions only give a generic <Signature (*args, **kwargs)>?

Next, what is the extent of the problem: is it throwing off some tests (test_signatures, what else?) or only the generation of the compatibility overview table, #450 (comment) ?

If the latter, maybe we could use the __nb_signature__ attribute instead of inspect?

I think it would be cleaner to make sub issue for each section for better tracking so we don't pollute the main issue and this can be be used as the parent tracker.

@ev-br

Is that table above really the status? I thought MLX was much closer to completeness; I just checked a couple of common functions: moveaxis does exist, and argsort and stack do have an axis keyword.

Also agreed with @ev-br's about no device= keyword) - that's a single issue, it just touches many functions.

The table above, I think, is slightly off indeed. Quick check; moveaxis complaint about incompatible signature just hides an edge case with empty inputs:

In [1]: import mlx.core as mx

In [2]: mx.moveaxis(mx.asarray([], dtype=bool), (), ())
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[2], line 1
----> 1 mx.moveaxis(mx.asarray([], dtype=bool), (), ())

TypeError: asarray(): incompatible function arguments. The following argument types are supported:
    1. asarray(a: Union[scalar, array, Sequence, DLPackCompatible], dtype: Optional[Dtype] = None, *, copy: Optional[bool] = None) -> array

Invoked with types: list, kwargs = { dtype: type }

Whether there are other problems is a bit hard to tell because this edge case chokes the test suite. One other thing that gets in the way of the test suite is the lack of __complex__, fixed in ml-explore/mlx#3984.

So I suppose it'd be best to re-generate the table with the built-from-source mlx.

What is the expected behavior and what is the actual behavior we are seeing?

Here's the thing: In older CPython versions, the inspect module for custom callables simply raised a ValueError. This is also what our test suite expects, and once a ValueError is raised, it goes to _test_uninspectable_func(stub.__name__, func, stub_sig).

The newer versions of CPython, instead of raising a ValueError, it gives a generic signature as you mentioned <Signature (*args, **kwargs)> this case is not handled by array-api-tests and simply gives
Argument 'X' missing from signature. A simple condition like the one mentioned above should be enough to fix this. Let me put before after here for easy comparison:

Before:

    try:
        sig = signature(func)
    except ValueError:
        try:
            _test_uninspectable_func(stub.__name__, func, stub_sig)
        except Exception as e:
            raise e from None  # suppress parent exception for cleaner pytest output
    else:
        _test_inspectable_func(sig, stub_sig)

After:

    try:
        sig = signature(func)
    except ValueError:
        sig = None
    if sig is not None:
        params = list(sig.parameters.values())
        if [p.kind for p in params] == [
            Parameter.VAR_POSITIONAL,
            Parameter.VAR_KEYWORD,
        ]:
            sig = None
    if sig is None:
        try:
            _test_uninspectable_func(stub.__name__, func, stub_sig)
        except Exception as e:
            raise e from None  # suppress parent exception for cleaner pytest output
    else:
        _test_inspectable_func(sig, stub_sig)

(If signature contains *args, **kwargs, return sig = None and again follow the same patter as before)

The pass and fail for different versions in CI should mostly be because of how CPython shipped this feature; all the latest ones have it anyway. This fix works for all the versions. (I think this is the PR in CPython that adds this feature.)

Edit: My stupid laptop hung midway, continuing from above

With that said, I do need to update the table since there have been more fixes to the upstream MLX and some fixes in my local array-api-tests module. So the table above is quite inaccurate.

@ev-br

With all error logs now and the triage recommendations from @ev-br (mentioned above), this is the newly generated table.

Function(s)	Failure	Minimal reproducer
`__getitem__`	Boolean indexing unsupported
`__setitem__`	Indexing semantics / masking failures
`__index__`	Missing method	`mx.asarray(1).__index__()`
`__pos__`	Missing unary positive	`+mx.asarray(1)`
~~`to_device`~~	Missing method	Device-related; leave until later
~~`device`~~	Missing attribute	Device-related; leave until later
`mT`	Missing attribute	`mx.ones((2,2)).mT`
`__array_namespace_info__`	Missing implementation	`mx.__array_namespace_info__()`
`arange`	Signature mismatch	Compat-layer argument adaptation
`asarray`	Copy semantics differ	`copy=False` semantics incompatible
`empty_like`	Signature mismatch	Compat-layer argument adaptation
`full`	Incorrect fill behavior
`full_like`	Incorrect fill behavior	Compat-layer argument adaptation may help
`linspace`	Signature mismatch	Compat-layer argument adaptation
`meshgrid`	Returns `list` instead of `tuple`	`type(mx.meshgrid(mx.arange(3)))`
`ones_like`	Signature mismatch	Compat-layer argument adaptation
`zeros_like`	Signature mismatch	Compat-layer argument adaptation
`astype`	Signature mismatch	Compat-layer argument adaptation
`broadcast_arrays`	Returns `list` instead of `tuple`
`broadcast_shapes`	Empty-input behavior differs	`mx.broadcast_shapes()`
`can_cast`	Incorrect dtype behavior
`finfo`	Constructor/signature issue
`iinfo`	Missing `bits` attribute	`mx.iinfo(mx.int32).bits`
`result_type`	Scalar handling differs
`from_dlpack`	Signature mismatch	Compat-layer argument adaptation
`rfft`, `rfftn`	Missing complex dtype support
`hfft`, `ihfft`	Missing implementation	`mx.fft.hfft(...)`
`fftfreq`, `rfftfreq`	Behavioral failures
`take_along_axis`	Shape/signature mismatch
`expand_dims`	Invalid negative-axis handling	`mx.expand_dims(mx.zeros(()), axis=(-3, -2))`
`moveaxis`	Signature mismatch	Compat-layer argument adaptation
`repeat`	Behavioral failures
`reshape`	Signature mismatch	Compat-layer argument adaptation
`unstack`	Incorrect behavior
`permute_dims`	Signature mismatch	Compat-layer argument adaptation
`cross`	Missing/incorrect `axis` handling	Compat-layer argument adaptation
`cholesky`	Complex inputs unsupported
`det`	Complex inputs unsupported
`diagonal`	Missing implementation	`mx.linalg.diagonal(...)`
`eig`	Wrong return type (not namedtuple)
`eigvals`	Missing complex dtype support
`eigh`	Behavioral failures
`eigvalsh`	Behavioral failures
`inv`	Complex inputs unsupported
`linalg.matmul`	Missing implementation	`mx.linalg.matmul(...)`
`matmul`	Integer dtype behavior differs
`matrix_norm`	Missing implementation	`mx.linalg.matrix_norm(...)`
`matrix_power`	Missing implementation	`mx.linalg.matrix_power(...)`
`matrix_rank`	Missing implementation	`mx.linalg.matrix_rank(...)`
`matrix_transpose`	Missing implementation / wrong shape
`outer`	Missing implementation	`mx.linalg.outer(...)`
`pinv`	Signature mismatch	Compat-layer argument adaptation
`qr`	Signature mismatch / behavioral failures
`slogdet`	Behavioral failures
`solve`	Shape/indexing failures
`svd`	Signature mismatch / behavioral failures
`svdvals`	Missing implementation	`mx.linalg.svdvals(...)`
`tensordot`	Missing implementation / integer dtype failures	`mx.linalg.tensordot(...)`
`trace`	Missing implementation	`mx.linalg.trace(...)`
`vecdot`	Missing implementation / behavioral failures	`mx.linalg.vecdot(...)`
`vector_norm`	Missing implementation	`mx.linalg.vector_norm(...)`
`argmax`	Wrong output dtype (`uint32`)
`argmin`	Wrong output dtype (`uint32`)
~~`nonzero`~~	Missing implementation	Data-dependent shape API; leave until later
`searchsorted`	Missing implementation	`mx.searchsorted(...)`
`argsort`	Signature mismatch / behavioral failures
`sort`	Signature mismatch / behavioral failures
~~`isin`~~	Missing implementation	Data-dependent shape API; leave until later
~~`unique_all`~~	Missing implementation	Data-dependent shape API; leave until later
~~`unique_counts`~~	Missing implementation	Data-dependent shape API; leave until later
~~`unique_inverse`~~	Missing implementation	Data-dependent shape API; leave until later
~~`unique_values`~~	Missing implementation	Data-dependent shape API; leave until later
`cumulative_sum`	Signature mismatch / behavioral failures	Compat-layer argument adaptation may help
`cumulative_prod`	Signature mismatch / behavioral failures	Compat-layer argument adaptation may help
`prod`	Signature mismatch / behavioral failures	Compat-layer argument adaptation may help
`sum`	Signature mismatch / behavioral failures	Compat-layer argument adaptation may help
`std`	Signature mismatch	Compat-layer argument adaptation
`var`	Signature mismatch	Compat-layer argument adaptation
~~`acos`~~	Incorrect numerical/special-case behavior	Covered by existing xfail lists for wrapped libraries
~~`acosh`~~	Incorrect numerical/special-case behavior	Covered by existing xfail lists for wrapped libraries
~~`asin`~~	Incorrect numerical/special-case behavior	Covered by existing xfail lists for wrapped libraries
~~`asinh`~~	Incorrect numerical/special-case behavior	Covered by existing xfail lists for wrapped libraries
~~`cos`~~	Incorrect numerical/special-case behavior	Covered by existing xfail lists for wrapped libraries
~~`sin`~~	Incorrect numerical/special-case behavior	Covered by existing xfail lists for wrapped libraries
~~`tanh`~~	Incorrect numerical/special-case behavior	Covered by existing xfail lists for wrapped libraries
`atan2`	Signature mismatch	Compat-layer argument adaptation
`bitwise_left_shift`	Incorrect results
`bitwise_right_shift`	Incorrect results
`bitwise_xor`	Incorrect results
`clip`	Signature mismatch	Compat-layer argument adaptation
`copysign`	Missing implementation	`mx.copysign(...)`
~~`expm1`~~	Incorrect numerical/special-case behavior	Covered by existing xfail lists for wrapped libraries
`floor_divide`	Incorrect results
`hypot`	Missing implementation	`mx.hypot(...)`
`nextafter`	Missing implementation	`mx.nextafter(...)`
~~`remainder`~~	Incorrect signed-zero behavior	Covered by existing xfail lists for wrapped libraries
~~`sign`~~	Incorrect NaN/signed-zero handling	Covered by existing xfail lists for wrapped libraries
`signbit`	Missing implementation	`mx.signbit(...)`

Still need to check for edge cases

Here's the thing: In older CPython versions, the inspect module for custom callables simply raised a ValueError. This is also what our test suite expects, and once a ValueError is raised

Okay, so this is something specific to test_signatures. I'm still not 100% sure what is the user visible effect here (where "user" is the person running the test suite), but anyhow let's take it to a issue or even a PR to array-api-tests.

let's take it to a issue

Sure. Let's brainstorm a little more on this in a separate issue thread.

With some manual effort and comparing function signatures from the docs, I have these findings. I'll check for exact edge cases later.
Next goal is to make a similar list for extensions.

These functions with missing parameters cause 27 failures:

arrange: (edge case?)
diff: missing prepend, append
linespace: missing endpoint
astype: missing copy
take_along_axis: (edge case probably)
pinv: no rtol
moveaxis: (edge case)
reshape: no copy
clip: (edge case maybe)
arctan2: (may be an edge case)
cumsum: include_initial missing
cumprod: include_initial missing
prod: missing dtype missingg
std: correction parameter
sum: dtype
var: correction parameter
permute_dims: (edge case?)
argsort: descending, stable missing
sort: descending, stable missing
qr: mode missing
svd: full_matrices parameter (mlx has compute_uv instead — different semantics)

I edited the top post with some details.

@ev-br

@ev-br
Small question. How is it decided what goes in the compat layer and what in the upstream library? For example the __array_namespace_info__, some libraries have it some have it in compat layer. A few parameters like device would be a bit more complicated, so it makes sense (at least temporarily) to have them in compat (like CuPy).

Ideally, the compat layer is ephemeral and goes away at some point. And MLX seems receptive to PRs, so---again ideally---the endgame for MLX is similar to JAX today: there is no compatibility layer, everything lives in MLX itself, and those things where it deviates from the standard (JAX: __setitem__, data-dependent shapes, MLX: devices, data-dependent shapes) are clearly marked / skipped upstream/

Makes sense.

Once ml-explore/mlx#4334 lands, the test suite will run without local patching. Once that's possible, one next step is to send a PR to MLX with a CI setup and an xfail file. Basically, revive ml-explore/mlx#3526 and add --disable-data-dependent-shapes and add an xfail file (cf https://github.com/data-apis/array-api-strict/blob/main/.github/workflows/array-api-tests.yml#L62).

The xfails file will have ~250 entries, of which ~100 are "low-prio" special cases. The remaining ones are also repeated: a missing function causes at least two failures; the missing __index__ causes a half dozen etc.

With that, we could ask MLX maintainers if they prefer issues for individual failures, or PRs, or something else entirely.

Since array-api-tests now supports multiple xfail files, it would make sense to have them for different builds. Some build-dependent failures were discovered here; there might be a few more.

I'm not sure if we have found any platform-dependent failures yet.

Navigation Menu

Repository navigation

MLX Array API compatibility: upstream status, compatibility #450

MLX Array API compatibility overview

Scope

Compatibility tracking:

The desired endgame

Process

Known blockers

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

MLX Array API compatibility: upstream status, compatibility #450

Description

MLX Array API compatibility overview

Scope

Compatibility tracking:

The desired endgame

Process

Known blockers

Activity

prady0t commented on Aug 5, 2026

aaishwarymishra commented on Aug 5, 2026

prady0t commented on Aug 6, 2026

ev-br commented on Aug 6, 2026

ev-br commented on Aug 6, 2026

prady0t commented on Aug 6, 2026

ev-br commented on Aug 6, 2026

ev-br commented on Aug 7, 2026

prady0t commented on Aug 7, 2026

ev-br commented on Aug 8, 2026

prady0t commented on Aug 8, 2026

ev-br commented on Aug 9, 2026

aaishwarymishra commented on Aug 10, 2026

rgommers commented on Aug 10, 2026

ev-br commented on Aug 10, 2026

prady0t commented on Aug 10, 2026

prady0t commented on Aug 10, 2026

prady0t commented on Aug 10, 2026

ev-br commented on Aug 11, 2026

prady0t commented on Aug 11, 2026

prady0t commented on Aug 11, 2026

ev-br commented on Aug 13, 2026

prady0t commented on Aug 14, 2026

ev-br commented on Aug 14, 2026

prady0t commented on Aug 14, 2026

ev-br commented on Aug 19, 2026

prady0t commented on Aug 19, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions