[Web] [Feature Request] Ability to abort

[sroussey]

Describe the issue

I am working on a PR for huggingface/transformers.js that adds the ability to abort an operation. Right now, it works for model downloads, but it does not work for generation that happens via ORT.

To reproduce

On the JS side of things, an AbortController as used by fetch() would be great.

Urgency

No response

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.20.1 and 1.21.0-dev.20250206-d981b153d3

Execution Provider

'wasm'/'cpu' (WebAssembly CPU), 'webgpu' (WebGPU)

[sroussey]

For reference:

huggingface/transformers.js#1193

huggingface/transformers.js#1182

[fs-eire]

In my understanding, the only thing abortable is the fetch that happens in ort.InferenceSession.create(). No other operation is really abortable, including the inference session initialization and run.

If you are running a transformer model, each iteration (or, token generation) is a call to OrtRun. It's reasonable to abort between the iterations (which is discussed in huggingface/transformers.js#1182), but there is no way to "abort" inside the OrtRun function call.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[snnn]

This issue has been automatically closed because it has been marked as 'stale' for more than 30 days without activity. If you believe this is still an issue, please feel free to reopen it.