Stamp ESP-IDF component versions from git and add registry metadata

The component's version was version.txt plus a hand-bumped packaging
revision (1.6.0~3), which nothing tied to what was being packaged. The
generator now derives it from git and refuses stamps the checkout cannot
back: a plain X.Y.Z needs HEAD at the vX.Y.Z tag, X.Y.Z-rc.N needs the
vX.Y.Z-rcN tag, and an X.Y.Z~N respin needs HEAD after vX.Y.Z on
release/X.Y with no changes since the tag outside examples/espressif/,
.ci/, .github/ and docs/. Those three also refuse a checkout with changes
to what the component ships: to tracked files, or untracked files where
the generator globs sources, the flatbuffers submodule's headers
included. Builds from any other commit pass --allow-dev-stamp and get
X.Y.Z-dev.<UTC commit date>, which is never published. An ExecuTorch
tree that is not the top of its own git checkout gets no stamp, so a
copy inside another repository is never stamped from that repository's
history. examples/espressif/test_version_stamp.sh checks these guards in
a scratch repository, and CI runs it first.

idf_component.yml gains the registry's repository, documentation (at the
packaged commit), issues, discussion and tags fields, and its license now
lists every license in the vendored code: ExecuTorch's BSD-3-Clause,
FlatBuffers' Apache-2.0, and the LLVM (Apache-2.0 WITH LLVM-exception,
NCSA) and musl (MIT) code in a few runtime and c10 headers. The component
ships LLVM's LICENSE.TXT, whose legacy NCSA text array_ref.h refers to,
next to FlatBuffers' license. sbom.yml with one sub-manifest each for
FlatBuffers and c10 describes the vendored code for esp-idf-sbom, and CI
validates them along with compote manifest lint.

The component ships examples/hello_executorch, a project that runs the
smoke model with the model and kernel bundle made at generation time by
the component's own gen_kernels.py. The generator therefore needs
ExecuTorch installed from the exact commit it packages, and checks that
before it starts. The example depends on exactly this component version,
since its bundle is made for it, through override_path. CI builds it
that way and runs it on the ESP32-S3 emulator, and the consumer test
also builds it as a registry user gets it, copied out of the packed
archive without override_path so the pin resolves from the mirror. The
component's README, which becomes its registry page, tells projects to
depend on one exact version and commit dependencies.lock, since a bundle
only fits the version it was made for.

Authored with Claude Code.

Author: rascani

.ci/scripts/test_espressif_idf_component.sh

@@ -11,9 +11,11 @@
 # build runs all three with a kernel bundle the build makes itself; an ESP32-C6
 # build runs the first two, with the Cadence kernels too, and must not link the
 # delegate; two more ESP32-S3 builds run the delegate alone, from a bundle with
-# no kernel libraries and from executorch_add_backend with no bundle. Also
-# checks that stale or mismatched bundles fail at configure time, and that an
-# app declaring nothing links to the same size as one without the component.
+# no kernel libraries and from executorch_add_backend with no bundle, and the
+# component's own example runs too. Also checks the version stamping, the
+# manifest and SBOM, that stale or mismatched bundles fail at configure time,
+# and that an app declaring nothing links to the same size as one without the
+# component.
 
 set -euo pipefail
 
@@ -37,6 +39,9 @@ collect_artifacts() {
 }
 trap collect_artifacts EXIT
 
+# Needs only git, so a stamp regression fails before the toolchain installs.
+bash examples/espressif/test_version_stamp.sh
+
 bash examples/espressif/setup.sh --minimal --targets=esp32s3,esp32c6 2>&1 \
     | tee "${build_dir}/setup.log"
 # shellcheck source=/dev/null
@@ -66,8 +71,10 @@ models="${cpu_models};${build_dir}/delegate.pte"
 
 component_dir="${build_dir}/components/executorch"
 kernels_dir="${build_dir}/smoke_kernels"
-bash examples/espressif/build_idf_component.sh --out "${component_dir}" \
+bash examples/espressif/build_idf_component.sh --allow-dev-stamp --out "${component_dir}" \
     2>&1 | tee "${build_dir}/generate.log"
+compote manifest lint "${component_dir}"
+(cd "${component_dir}" && esp-idf-sbom manifest validate idf_component.yml sbom.yml sbom_*.yml)
 gen_kernels() {
     local out=$1
     shift
@@ -261,9 +268,9 @@ run_firmware() {
     local firmware_dir="${build_dir}/firmware-${name}"
     IDF_TARGET="${chip}" idf.py -C "${project}" -B "${firmware_dir}" merge-bin \
         > "${build_dir}/merge-${name}.log" 2>&1
+    local elf=("${firmware_dir}"/*.elf)
     timeout 130s esp-emu --chip "${chip}" \
-        --firmware "${firmware_dir}/merged-binary.bin" \
-        --elf "${firmware_dir}/et_component_smoke.elf" \
+        --firmware "${firmware_dir}/merged-binary.bin" --elf "${elf[0]}" \
         --net user --timeout 120s --exit-on 'Program complete.' \
         2>&1 | tee "${build_dir}/emulator-${name}.log"
     for marker in "$@" "ET_COMPONENT_SMOKE: PASS" "Program complete."; do
@@ -349,4 +356,9 @@ if (( 16#${registry} > 64 )); then
     exit 1
 fi
 
+# The example the component ships, built through its override_path.
+example="${component_dir}/examples/hello_executorch"
+project="${example}" consumer example esp32s3 build
+project="${example}" run_firmware example esp32s3 "smoke: PASS"
+
 echo "PASS: generated ESP-IDF component ran every build on the emulator."

.ci/scripts/test_espressif_idf_consumer.sh

@@ -16,7 +16,8 @@
 # ESP32-S3 build an ESP-NN delegate program.
 # ESPRESSIF_IDF_VERSION selects the ESP-IDF release: v6.1 builds every target
 # an emulator runs (QEMU for ESP32, esp-emu for the rest), other releases
-# ESP32-S3 and ESP32-C6. Also lints the component's manifest.
+# ESP32-S3 and ESP32-C6. Also builds the component's example the way a registry
+# user gets it, and lints the component's manifest.
 
 set -euo pipefail
 
@@ -83,9 +84,9 @@ print(entry["atol"], entry["rtol"])
 python "${split}" "${assets}/delegate.bpte" --atol "${atol}" --rtol "${rtol}"
 
 component_dir="${build_dir}/component/executorch"
-bash examples/espressif/build_idf_component.sh --out "${component_dir}" \
+bash examples/espressif/build_idf_component.sh --allow-dev-stamp --out "${component_dir}" \
     2>&1 | tee "${build_dir}/generate.log"
-compote manifest lint "${component_dir}/idf_component.yml"
+compote manifest lint "${component_dir}"
 version=$(sed -n 's/^version: "\(.*\)"$/\1/p' "${component_dir}/idf_component.yml")
 (cd "${component_dir}" && compote component pack --name executorch --dest-dir "${build_dir}/dist")
 archive="${build_dir}/dist/executorch_${version}.tgz"
@@ -164,7 +165,7 @@ run_firmware() {
         consumer "${name}" "${target}" merge-bin > "${build_dir}/merge-${name}.log" 2>&1
         timeout 130s esp-emu --chip "${target}" \
             --firmware "${firmware_dir}/merged-binary.bin" \
-            --elf "${firmware_dir}/et_component_smoke.elf" \
+            --elf "$(find "${firmware_dir}" -maxdepth 1 -name '*.elf')" \
             --net user --timeout 120s --exit-on 'Program complete.' \
             > "${log}" 2>&1 || true
     fi
@@ -213,4 +214,25 @@ if (str(entry["version"]), entry.get("component_hash")) != (
     sys.exit(f"FAIL: pytorch/executorch did not come from the mirror: {entry}")
 PY
 done
+
+# The component's example as idf.py create-project-from-example gives it to a
+# registry user: copied out of the archive, without the override_path that the
+# registry removes, so its exact version pin resolves from the mirror. The
+# component manager cannot download examples from a file:// mirror itself.
+example="${sources}/hello_executorch"
+rm -rf "${example}"
+mkdir -p "${example}"
+tar -xzf "${archive}" -C "${example}" --strip-components=3 ./examples/hello_executorch
+python - "${example}/main/idf_component.yml" <<'PY'
+import sys
+
+import yaml
+
+manifest = yaml.safe_load(open(sys.argv[1]))
+del manifest["dependencies"]["pytorch/executorch"]["override_path"]
+yaml.safe_dump(manifest, open(sys.argv[1], "w"))
+PY
+project="${example}" consumer hello_executorch esp32s3 build \
+    > "${build_dir}/firmware-hello_executorch.log" 2>&1
+project="${example}" run_firmware hello_executorch esp32s3 "smoke: PASS"
 echo "PASS: the packed ESP-IDF component built from a mirror and ran on ${targets[*]}."

examples/espressif/README.md

@@ -15,10 +15,17 @@ cross-compile-then-link flow below, generate a self-contained ESP-IDF
 component from sources in this repository:
 
 ```bash
-./examples/espressif/build_idf_component.sh \
+./examples/espressif/build_idf_component.sh --allow-dev-stamp \
     --out /path/to/your_project/components/executorch
 ```
 
+Run it from a Python environment with ExecuTorch installed from this exact
+commit (reinstall after committing or pulling): the generator exports the
+component's example model with it. `--allow-dev-stamp` versions the component
+`X.Y.Z-dev.<commit date>`; without it the generator stamps a release
+version and requires the checkout to be at that release's tag (see the
+script's header for respin and RC stamps).
+
 The output is a registry-shaped component (`idf_component.yml`,
 `CMakeLists.txt`, `Kconfig`, `VERSION`) containing the runtime, the
 portable, quantized, Cadence generic and `esp_nn::` kernel libraries,