[executorch][muse] Accept images in CUDA DFlash batching by Gasoonjia · Pull Request #24066 · pytorch/executorch

Gasoonjia · 2026-10-21T18:57:47Z

Stack from ghstack (oldest at bottom):

The CUDA DFlash executor accepts prepared image inputs on admission; image
steps run the target from embeddings, then hidden taps and drafting proceed as
for tokens.

Differential Revision: D125987295

[ghstack-poisoned]

pytorch-bot · 2026-10-21T18:57:52Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24066

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Pending, 1 Unrelated Failure, 1 Unclassified Failure

As of commit d95c9d5 with merge base 5e8d11b ():

NEW FAILURE - The following job has failed:

pull / android / run-emulator (gh)
##[error]The process '/usr/local/lib/android/sdk/platform-tools/adb' failed with exit code 224

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

Test CUDA Builds / test-muse-glimmer-cuda-e2e-kquant-17gb-dflash-image-batching / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
##[error]received SIGINT

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / test-arm-backend-no-driver (test_pytest_ops_tosa) / linux-job (gh) (trunk failure)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-10-21T18:58:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[ghstack-poisoned]

Pull Request resolved: #24066 The CUDA DFlash executor accepts prepared image inputs on admission; image steps run the target from embeddings, then hidden taps and drafting proceed as for tokens. ghstack-source-id: 465398729 @exported-using-ghexport Differential Revision: [D125987295](https://our.internmc.facebook.com/intern/diff/D125987295/)

[ghstack-poisoned]

Pull Request resolved: #24066 The CUDA DFlash executor accepts prepared image inputs on admission; image steps run the target from embeddings, then hidden taps and drafting proceed as for tokens. ghstack-source-id: 465632846 @exported-using-ghexport Differential Revision: [D125987295](https://our.internmc.facebook.com/intern/diff/D125987295/)

…ckends Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0) (oldest at bottom): * #24076 * #24107 * #24075 * #24074 * #24073 * #24072 * #24066 * #24065 * #24064 * #24063 * __->__ #24062 Move the MLX executor's EngineEmbeddings (vision encoder + embed_text + MuseGlimmerMaterializer) into runtime/model_embeddings so the CUDA batching executors can materialize prepared image inputs the same way. MLX behavior is unchanged; it now holds a MuseGlimmerModelEmbeddings. Differential Revision: D125987290 Pull Request resolved: #24062

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0) (oldest at bottom): * #24076 * #24107 * #24075 * #24074 * #24073 * #24072 * #24066 * #24065 * #24064 * __->__ #24063 * #24062 Steps that carry a prepared (image) input run forward_from_embeddings with the materialized rows; token-only steps keep the static forward_{N} methods and their CUDA graphs. SessionTable packs prepared rows as placeholders so the cache still plans their positions. Differential Revision: D125987291 Pull Request resolved: #24063

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0) (oldest at bottom): * #24076 * #24107 * #24075 * #24074 * #24073 * #24072 * #24066 * #24065 * __->__ #24064 * #24063 * #24062 export_cuda_batching optionally adds vision_encoder, embed_text and a dynamic forward_from_embeddings over the same off-graph KV contract, and writes pos_embed.bin next to the program. Differential Revision: D125987292 Pull Request resolved: #24064

Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0) (oldest at bottom): * #24076 * #24107 * #24075 * #24074 * #24073 * #24072 * #24066 * __->__ #24065 * #24064 * #24063 * #24062 The CUDA backend factory builds the image preparation spec from the shared embeddings engine, and serve_batching passes --pos-embed-path through, so image requests are served on regular CUDA batching. Differential Revision: D125987293 Pull Request resolved: #24065

@Gasoonjia

This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #24066 by @Gasoonjia ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/Gasoonjia/89/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/Gasoonjia/89/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/Gasoonjia/88/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/Gasoonjia/89/orig Differential Revision: [D125987295](https://our.internmc.facebook.com/intern/diff/D125987295/) @diff-train-skip-merge Co-authored-by: gasoonjia <gasoonjia@icloud.com>

Update

aa563d7

[ghstack-poisoned]

Gasoonjia temporarily deployed to cadence October 21, 2026 18:58 — with GitHub Actions Inactive

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 21, 2026

meta-codesync Bot added the meta-exported label Oct 21, 2026

Gasoonjia temporarily deployed to cadence October 21, 2026 19:16 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence October 21, 2026 19:18 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence October 21, 2026 19:19 — with GitHub Actions Inactive

metascroy approved these changes Oct 21, 2026

View reviewed changes

Update

cce090b

[ghstack-poisoned]

Gasoonjia mentioned this pull request Oct 23, 2026

[executorch][muse-glimmer] Bound the CUDA prefill logits selector by the sampler rows #24107

Merged

Gasoonjia temporarily deployed to cadence October 23, 2026 08:51 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence October 23, 2026 09:10 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence October 23, 2026 09:14 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence November 5, 2026 03:08 — with GitHub Actions Inactive

Gasoonjia added the ciflow/cuda label Nov 5, 2026

Gasoonjia temporarily deployed to cadence November 5, 2026 03:24 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence November 5, 2026 03:28 — with GitHub Actions Inactive

pytorch-bot Bot deployed to upload-benchmark-results November 5, 2026 04:26 Active

Update

d95c9d5

[ghstack-poisoned]

Gasoonjia temporarily deployed to cadence November 5, 2026 08:18 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence November 5, 2026 08:32 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence November 5, 2026 08:33 — with GitHub Actions Inactive

Gasoonjia temporarily deployed to cadence November 5, 2026 08:35 — with GitHub Actions Inactive

meta-codesync Bot merged commit 83baecf into gh/Gasoonjia/89/base Nov 5, 2026

meta-codesync Bot deleted the gh/Gasoonjia/89/head branch November 5, 2026 09:05

meta-codesync Bot temporarily deployed to cherry-pick-bot November 5, 2026 09:05 Inactive

pytorchbot mentioned this pull request Nov 5, 2026

[executorch][muse] Accept images in CUDA DFlash batching #24656

Merged

pytorch-bot Bot deployed to upload-benchmark-results November 5, 2026 09:48 Active

Gasoonjia deployed to upload-benchmark-results November 5, 2026 10:03 — with GitHub Actions Active

Navigation Menu

Repository navigation

[executorch][muse] Accept images in CUDA DFlash batching - #24066

[executorch][muse] Accept images in CUDA DFlash batching#24066
meta-codesync[bot] merged 4 commits into
gh/Gasoonjia/89/basefrom
gh/Gasoonjia/89/head

Gasoonjia commented Oct 21, 2026 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Oct 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Gasoonjia commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24066

❌ 1 New Failure, 1 Pending, 1 Unrelated Failure, 1 Unclassified Failure

Uh oh!

github-actions Bot commented Oct 21, 2026

This PR needs a release notes: label

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Gasoonjia commented Oct 21, 2026 •

edited

Loading

pytorch-bot Bot commented Oct 21, 2026 •

edited

Loading

This PR needs a `release notes:` label