Naive question

[MordicusEtCubitus]

Could the new dask backend using cupy implements by itself the cupy-xarray machinery ?

%%time
import dask.array as da
import cupy as cp
import dask
cp.cuda.Device(0).use()
with dask.config.set({"array.backend": "cupy"}):
    x = da.random.normal(10,1, (100_000, 100_000), chunks=(10_000, 2000))
    y = x.mean(axis=0)[::100]
    z = y.compute()
    print(z.shape, type(z))

Or is it too naive and xarray is using numpy directly at many places in the code ?

Gaël,

[keewis]

the issue is elsewhere, see pydata/xarray#7721: cupy refuses to interact with 0D numpy arrays, so in order to use the array API we have to cast python scalars to cupy arrays (the array API does not allow passing python or numpy scalars, these need to become 0D arrays). xarray has a special-case for wrapped cupy arrays, but that fails to detect cupy wrapped by dask.

To fix this, we'd have to add another special-case for cupy+dask, which is not great. I've been thinking about this for a while, and I think something like the mechanism described in nested-duck-arrays could help with avoiding the accumulation of too many of these special-cases.

Edit: looks like I completely misunderstood the question

[dcherian]

Thanks @keewis

@MordicusEtCubitus I think this discussion belongs elsewhere. There is no xarray in your example! :)

[MordicusEtCubitus]

No, keewis properly replied. Indeed, I should have add an xarray code below with dask.config.set({"array.backend": "cupy"})