Using pipeline API in Mobile Devices

[LuSrodri]

Question

How can I do the pipeline running in mobile devices?

Like here:
pipeline('background-removal', 'briaai/RMBG-1.4', { device: "webgpu" })

Or it depends from the model avaliable?

I don't find documentations about pipeline API options, like 'device' and others params...

[xenova]

Hi there 👋 The models you are able to use greatly depends on the target devices you want to run on, as many phones either:

1. Limit the amount of memory that you can consume (e.g., on WASM), or
2. Don't yet support WebGPU

Models like https://huggingface.co/Xenova/modnet should be able to run quite well on mobile, but you can try others from this list: https://huggingface.co/models?library=transformers.js&other=background-removal&sort=trending

[saikumaru]

@xenova What would be a good way to filter text-generation models for running on mobile?

        generator = await pipeline("text-generation", modelName, {
          dtype: "fp16",
          device: "wasm"
        });