[rascani]

A new CI job builds the component the way a project using the published
component gets it. It generates and packs the component, adds the archive to a
file:// component mirror (.ci/scripts/esp_idf_mirror.py writes the layout the
registry serves, and refuses conditional dependencies, which it cannot
evaluate the way the registry does), and builds idf_component_consumer, which
resolves pytorch/executorch from that mirror at the version just generated and
esp-nn from the registry. The builds use a fresh ESP-IDF Python environment
that the job checks cannot import executorch, with EXECUTORCH_PYTHON unset, so
they take the kernel bundles made beforehand. They build a copy of the project,
which keeps the component manager's state out of the source tree, and the
source, model, kernel bundle and build paths have a space. On ESP-IDF 6.1 the
job builds for ESP32 (run on ESP-IDF's QEMU), ESP32-S3, -C3, -C5, -C6 and -P4
(run on esp-emu); on 6.0.1, a second job, for ESP32-S3 and -C6. It also lints
the component's manifest, and checks that the lock file records the mirror's
archive.

Every build runs the smoke model. The ESP32-S3 and -C6 builds also run the
MLPerf Tiny streaming wake word model, quantized with the Cadence quantizer
and exported by the new idf_component_smoke/export_wakeword_model.py, a test
fixture with random weights, as a bundled program of 40 consecutive frames,
all 40 of which must match the quantized PyTorch model,
and the quantized mul on the esp_nn:: op library; the ESP32-S3 build also runs
the ESP-NN delegate program, so the delegate and op library are built from the
packed archive against the esp-nn the component manager resolves. The kernel
bundles come from the archive's own gen_kernels.py, the copy consumers run. To run a
stateful model, the smoke app now runs all of a bundled program's test cases
in order on one loaded method, and split_bundled_program.py can repeat them so
single-case models still check a second execute. idf_component_consumer reuses
the smoke app's main component, so the two cannot drift apart.

setup.sh gains --idf-version and, unless --skip-emulator, installs QEMU when
esp32 is a target. The workflow now also runs for changes to what the component
vendors (runtime, the portable, quantized and prim op kernels, schema, the
buffer data loader and flat_tensor schema, codegen, the Cadence AOT and generic
kernels, the flatbuffers submodule and build_variables.bzl) and to version.txt.

Authored with Claude Code.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24709

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 622e9d6 with merge base 2d8cd35 ():

NEW FAILURES - The following jobs have failed:

• pull / android / run-emulator (gh)
  adb: device offline
• Test QNN Windows MSVC build / build-qnn-windows-arm64 (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[JakeStevens]

@claude review effort=max

[claude]

Claude finished @JakeStevens's task in 37m 4s —— View job

---

PR Review: #24709

Nothing blocks this PR, and both items the Nov 6 review requested changes for are fixed. The one new gap is that CI makes the kernel bundles with the unpacked tree's gen_kernels.py instead of the copy in the packed archive. On @JakeStevens's question, I'd keep export_wakeword_model.py as a test fixture and move it under idf_component_smoke/.

Detailed review

Summary

The PR packs the generated component and serves it from a file:// mirror. It then builds idf_component_consumer from that mirror with ESP-IDF's own Python and runs it on every emulated target: six on v6.1, two on v6.0.1. The S3 and C6 builds also run a stateful wake word model frame by frame.

Since the Nov 6 review:

• Fixed: the paths: filter, QEMU now gated on --skip-emulator, delegate and op-library coverage, the no-executorch check, spaces in asset paths, and the mirror refusing conditional dependencies.
• New: one testing gap, and where the wake word exporter sits.
• Unchanged: several minor points, listed below.
• CI: no Test ESP32 Emulators run exists for 96b707b, so the new jobs haven't run on this head.

Code Quality

• @JakeStevens's question: keep export_wakeword_model.py, but as a test fixture.
  • It is the only model with state in the ESP-IDF tests. The S3 and C6 consumer builds check for wakeword: PASS, 40/40 cases (test_espressif_idf_consumer.sh:187,193), and the multi-case loop in idf_component_smoke/main/main.c:105-135 exists for it.
  • It uses random weights and only checks kernels and state frame by frame, so it shouldn't overlap with an application exporter like the one in muse gadgets. I can't see that PR from here, though.
  • The problem is where it sits. It is next to export_smoke_model.py, which README:276 presents as a user step, so it reads as the component's way to export a wake word model. README:32-34 says wake word pipelines stay with the application.
  • Move it to idf_component_smoke/ next to export_op_library_model.py, and update test_espressif_idf_consumer.sh:66. Fix this →
• These Nov 6 points are unchanged and have no reply. The fixes are in that review; close any you meant to keep.
  • export_wakeword_model.py:40-43: reset_history matches buffer names instead of using StreamingWakeWord.HISTORY_NAMES.
  • test_espressif_idf_consumer.sh:151-169: if QEMU exits early, the poll still waits the full 120 s, and esp-emu output isn't tee'd as it is at test_espressif_idf_component.sh:269.
  • test_espressif_idf_consumer.sh:179-180: consumer and five variables are exported only so that setsid bash -c can use them.
  • test_espressif_idf_consumer.sh:89: the version comes from sed instead of build_idf_component.sh --version.
  • idf_component_consumer/CMakeLists.txt:23-25: the plain set(SDKCONFIG_DEFAULTS ...) shadows -DSDKCONFIG_DEFAULTS.

Testing

• Minor. .ci/scripts/test_espressif_idf_consumer.sh:97-109 makes the kernel bundles with ${component_dir}/tools/gen_kernels.py, the unpacked tree, and never with the copy in the archive.
  • Consumers run the archive's copy. project_include.cmake:120,149 prints that path, and the fallback when no bundle is given runs it (:173).
  • gen_kernels.py reads its own component's idf_component.yml, VERSION, tools/kernel_libraries.json, the library YAMLs and tools/codegen (gen_kernels.py:88-141,256).
  • Trigger: compote component pack drops one of those files, through a default exclude or a later files: rule. This job still passes, because the unpacked tree stamps the same version and commit into the bundle. Every user's bundle generation fails, and covering that is the job's purpose.
  • Fix: extract ${archive} and point gen_kernels at its tools/gen_kernels.py. Fix this →
• These Nov 6 points are also unchanged and have no reply:
  • test_espressif_idf_consumer.sh:75: the wake word's --atol 1e-6 --rtol 0 still has no stated reason.
  • Question on export_wakeword_model.py:77-80: would a non-zero initial history catch a runtime that ignores a mutable buffer's initial state?
  • Question on test_espressif_idf_component.sh:79-89: can wakeword_ops now come from wakeword.pte?

Specific Comments

• examples/espressif/README.md:268-271 and setup.sh:33-34 (low confidence): these say each ESP-IDF release needs its own ESPRESSIF_TOOLS_DIR. But both releases install their Python requirements, each under its own constraints, into the same active environment (setup.sh:54-63,94). If 6.0 and 6.1 pin different versions, sourcing the other release's setup_path.sh may fail ESP-IDF's dependency check. Either say a release also needs its own Python environment, or confirm that one environment works for both.

Recommendation

Approve. Nothing above blocks. I'd still fix where the kernel bundles come from in this PR, since it's the one place the job doesn't test what consumers get. Merge once Test ESP32 Emulators has passed on this head.