Skip to content

Commit e721ff8

Browse files
rascanirascani
committed
Register the ESP-IDF component's kernels from a project kernel bundle
The generated component no longer bakes in op profiles. It vendors every kernel source of the portable, quantized and Cadence generic kernel libraries into a plain static library, ships gen_kernels.py with the libraries' yamls and ExecuTorch's codegen templates under tools/, and registers the kernels of the project's kernel bundle. A project declares its models and bundle with executorch_add_model and executorch_add_kernels from project_include.cmake. At the end of the project's configure, a deferred step checks that the bundle was made for this component, by version and commit, and for exactly the declared models (by SHA-256), rejects delegates the component does not provide yet, and compiles the bundle's registration and prim op selection into the component. A mismatch fails at configure time and prints the gen_kernels.py command that fixes it. A bundle used without declared models is used as is, with a warning that names the models it was made from. Registration no longer needs WHOLE_ARCHIVE. Each registration file is compiled through a small wrapper that defines an extern "C" anchor, and the deferred step adds -u for each one, which reaches the link from a deferred call. Only the bundle's files are anchored, so a project that declares no models and no bundle links nothing from the component: CI checks that such an app is the same size with and without it (169,728 B on ESP32-S3). The kernel library is not even compiled without a bundle. et_model_create references a symbol that only the bundle's registration defines, so an app that runs a model without declaring one fails to link on et_requires_executorch_add_kernels instead of failing every load with missing operators. CONFIG_ET_ENABLE goes away with the profile choice. The deferred step reads the bundle's kernel count from its selected_max_kernel_num.h, which it requires, and sizes the registry with it unless CONFIG_ET_MAX_KERNEL_NUM, now relative to the bundle, sets a larger size for an application that registers its own kernels; configuring one smaller than the bundle is an error. The smoke image on ESP32-S3 is 253,668 B, down from 266,660 B with the smoke profile, because the bundle selects only the prim ops the model uses (none here); with all 28 prims selected it is 266,684 B. CI runs that image on the emulator with the registry sized from the bundle, and also builds the smoke consumer for ESP32-C6 (setup.sh gains --targets) with a bundle that adds the ops of the Cadence-quantized wake word model, so the Cadence generic kernels stay compiled and linked. It checks that a re-exported model, a model the bundle was not made from, a bundle made for another component version or commit or missing its kernel count, and models without a bundle each fail at configure time, that a bundle used as is names its models, and that an app running an undeclared model fails to link. Projects that use an op profile, such as CONFIG_ET_OP_PROFILE_WAKEWORD_INT8, drop CONFIG_ET_ENABLE and the profile and move to a kernel bundle made from their model. Authored with Claude Code.
1 parent 9d116a2 commit e721ff8

16 files changed

Lines changed: 820 additions & 638 deletions

File tree

‎.ci/scripts/test_espressif_idf_component.sh‎

Lines changed: 170 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,12 @@
55
# This source code is licensed under the BSD-style license found in the
66
# LICENSE file in the root directory of this source tree.
77

8-
# Generates the ExecuTorch ESP-IDF component with every shipped profile,
9-
# builds the idf_component_smoke consumer for ESP32-S3 against each, and runs
10-
# the smoke profile build on the emulator.
8+
# Generates the ExecuTorch ESP-IDF component and a kernel bundle for the smoke
9+
# model, builds the idf_component_smoke consumer for ESP32-S3 and, with the
10+
# Cadence kernels too, for ESP32-C6, and runs the ESP32-S3 build on the
11+
# emulator. Also checks that stale or mismatched bundles fail at configure time,
12+
# and that an app declaring nothing links to the same size as one without the
13+
# component.
1114

1215
set -euo pipefail
1316

@@ -21,7 +24,8 @@ cd "${et_root}"
2124
collect_artifacts() {
2225
if [[ -n "${RUNNER_ARTIFACT_DIR:-}" ]]; then
2326
mkdir -p "${RUNNER_ARTIFACT_DIR}/esp-idf-component"
24-
for file in "${build_dir}"/*.log "${build_dir}/components/executorch/VERSION"; do
27+
for file in "${build_dir}"/*.log "${build_dir}/components/executorch/VERSION" \
28+
"${build_dir}/smoke_kernels/kernels.json"; do
2529
if [[ -f "${file}" ]]; then
2630
cp "${file}" "${RUNNER_ARTIFACT_DIR}/esp-idf-component/"
2731
fi
@@ -30,7 +34,8 @@ collect_artifacts() {
3034
}
3135
trap collect_artifacts EXIT
3236

33-
bash examples/espressif/setup.sh --minimal 2>&1 | tee "${build_dir}/setup.log"
37+
bash examples/espressif/setup.sh --minimal --targets=esp32s3,esp32c6 2>&1 \
38+
| tee "${build_dir}/setup.log"
3439
# shellcheck source=/dev/null
3540
source "${ESPRESSIF_TOOLS_DIR}/setup_path.sh"
3641

@@ -39,43 +44,180 @@ python examples/espressif/export_smoke_model.py \
3944
2>&1 | tee "${build_dir}/export.log"
4045

4146
component_dir="${build_dir}/components/executorch"
42-
profiles=$(find examples/espressif/idf_profiles -name '*.ops' -exec basename {} .ops \; \
43-
| sort | paste -sd, -)
44-
bash examples/espressif/build_idf_component.sh --profiles "${profiles}" \
45-
--out "${component_dir}" 2>&1 | tee "${build_dir}/generate.log"
47+
kernels_dir="${build_dir}/smoke_kernels"
48+
bash examples/espressif/build_idf_component.sh --out "${component_dir}" \
49+
2>&1 | tee "${build_dir}/generate.log"
50+
python "${component_dir}/tools/gen_kernels.py" --pte "${build_dir}/smoke.pte" \
51+
--out "${kernels_dir}" 2>&1 | tee "${build_dir}/gen_kernels.log"
52+
# The ESP32-C6 build also registers the ops of a Cadence-quantized streaming
53+
# wake word model, so CI compiles and links the Cadence generic kernels.
54+
wakeword_ops=aten::_softmax.out,aten::_to_copy.out,aten::cat.out,aten::copy_
55+
wakeword_ops+=,aten::permute_copy.out,aten::slice_copy.Tensor_out
56+
wakeword_ops+=,aten::squeeze_copy.dim_out,aten::transpose_copy.int_out
57+
wakeword_ops+=,cadence::dequantize_per_tensor.out,cadence::quantize_per_tensor.out
58+
wakeword_ops+=,cadence::quantized_conv2d_nhwc.per_tensor_out
59+
wakeword_ops+=,cadence::quantized_fully_connected.per_tensor_out
60+
wakeword_ops+=,cadence::quantized_relu.per_tensor_out
61+
python "${component_dir}/tools/gen_kernels.py" --pte "${build_dir}/smoke.pte" \
62+
--ops "${wakeword_ops}" --out "${build_dir}/wakeword_kernels" \
63+
2>&1 | tee -a "${build_dir}/gen_kernels.log"
4664

4765
project=examples/espressif/idf_component_smoke
4866

67+
# Runs idf.py on the smoke consumer: build name, target, then idf.py arguments.
68+
consumer() {
69+
local name=$1 target=$2
70+
shift 2
71+
IDF_TARGET="${target}" idf.py -C "${project}" -B "${build_dir}/firmware-${name}" \
72+
-DSDKCONFIG="${build_dir}/sdkconfig-${name}" "$@" \
73+
> "${log:-${build_dir}/firmware-${name}.log}" 2>&1
74+
}
75+
76+
# Configures a case that must fail: build name, expected message, -D arguments.
77+
expect_configure_error() {
78+
local name=$1 expected=$2
79+
shift 2
80+
if consumer "${name}" esp32s3 -DET_COMPONENT_DIR="${component_dir}" "$@" reconfigure; then
81+
echo "FAIL: ${name} configured; expected an error with '${expected}'" >&2
82+
exit 1
83+
fi
84+
# CMake wraps long messages, so match with whitespace runs collapsed.
85+
if ! tr -s ' \n' ' ' < "${build_dir}/firmware-${name}.log" | grep -qF "${expected}"; then
86+
echo "FAIL: ${name} did not report '${expected}'; see firmware-${name}.log" >&2
87+
exit 1
88+
fi
89+
echo "OK: ${name} failed at configure with '${expected}'"
90+
}
91+
92+
# A re-exported smoke.pte.
93+
mkdir -p "${build_dir}/stale"
94+
cp "${build_dir}/smoke.pte" "${build_dir}/stale/smoke.pte"
95+
printf '\0' >> "${build_dir}/stale/smoke.pte"
96+
expect_configure_error stale-model "smoke.pte: changed since the bundle was made" \
97+
-DET_SMOKE_PTE="${build_dir}/stale/smoke.pte" -DET_SMOKE_KERNELS="${kernels_dir}"
98+
99+
rm -rf "${build_dir}/mismatch_kernels"
100+
cp -r "${kernels_dir}" "${build_dir}/mismatch_kernels"
101+
python - "${build_dir}/mismatch_kernels/kernels.json" <<'PY'
102+
import json
103+
import sys
104+
105+
bundle = json.load(open(sys.argv[1]))
106+
bundle["component_version"] = "0.0.0"
107+
json.dump(bundle, open(sys.argv[1], "w"))
108+
PY
109+
expect_configure_error version-mismatch component_version \
110+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" \
111+
-DET_SMOKE_KERNELS="${build_dir}/mismatch_kernels"
112+
113+
# A bundle made by a component of the same version from another commit, as any
114+
# two commits between version.txt bumps are.
115+
rm -rf "${build_dir}/mismatch_kernels"
116+
cp -r "${kernels_dir}" "${build_dir}/mismatch_kernels"
117+
python - "${build_dir}/mismatch_kernels/kernels.json" <<'PY'
118+
import json
119+
import sys
120+
121+
bundle = json.load(open(sys.argv[1]))
122+
bundle["component_commit"] = "0" * 40
123+
json.dump(bundle, open(sys.argv[1], "w"))
124+
PY
125+
expect_configure_error commit-mismatch component_commit \
126+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" \
127+
-DET_SMOKE_KERNELS="${build_dir}/mismatch_kernels"
128+
129+
expect_configure_error no-bundle "no kernel bundle is" -DET_SMOKE_PTE="${build_dir}/smoke.pte"
130+
131+
# A model the bundle was not made from, declared instead of the one it was.
132+
mkdir -p "${build_dir}/other"
133+
cp "${build_dir}/stale/smoke.pte" "${build_dir}/other/other.pte"
134+
expect_configure_error other-model \
135+
"other.pte: not in the bundle smoke.pte: in the bundle, not declared" \
136+
-DET_SMOKE_PTE="${build_dir}/other/other.pte" -DET_SMOKE_KERNELS="${kernels_dir}"
137+
138+
rm -rf "${build_dir}/no_count_kernels"
139+
cp -r "${kernels_dir}" "${build_dir}/no_count_kernels"
140+
rm -r "${build_dir}/no_count_kernels/include"
141+
expect_configure_error no-kernel-count "has no kernel count" \
142+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${build_dir}/no_count_kernels"
143+
144+
# A bundle used without declaring its models is used as is, with a warning that
145+
# names them.
146+
consumer as-is esp32s3 -DET_COMPONENT_DIR="${component_dir}" \
147+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_UNDECLARED=1 \
148+
-DET_SMOKE_KERNELS="${kernels_dir}" reconfigure
149+
if ! tr -s ' \n' ' ' < "${build_dir}/firmware-as-is.log" \
150+
| grep -qF "is not checked against smoke.pte"; then
151+
echo "FAIL: no warning naming smoke.pte; see ${build_dir}/firmware-as-is.log" >&2
152+
exit 1
153+
fi
154+
echo "OK: as-is warned that smoke.pte is not checked"
155+
156+
# An app that runs a model while declaring nothing fails to link.
157+
if consumer undeclared esp32s3 -DET_COMPONENT_DIR="${component_dir}" \
158+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_UNDECLARED=1 build; then
159+
echo "FAIL: undeclared linked; expected et_requires_executorch_add_kernels to be undefined" >&2
160+
exit 1
161+
fi
162+
if ! grep -qF "undefined reference to \`et_requires_executorch_add_kernels'" \
163+
"${build_dir}/firmware-undeclared.log"; then
164+
echo "FAIL: undeclared did not fail on et_requires_executorch_add_kernels; see firmware-undeclared.log" >&2
165+
exit 1
166+
fi
167+
echo "OK: undeclared failed to link on et_requires_executorch_add_kernels"
168+
169+
# A kernel registry smaller than the bundle; the C6 build below uses one larger
170+
# than it, and the emulator runs the one sized from the bundle.
171+
printf 'CONFIG_ET_MAX_KERNEL_NUM=1\n' > "${build_dir}/small_registry.defaults"
172+
printf 'CONFIG_ET_MAX_KERNEL_NUM=64\n' > "${build_dir}/large_registry.defaults"
173+
expect_configure_error small-registry "CONFIG_ET_MAX_KERNEL_NUM=1 is below" \
174+
"-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;${build_dir}/small_registry.defaults" \
175+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${kernels_dir}"
176+
177+
# An app that declares nothing must not grow because the component is in the
178+
# build. The baseline swaps the component for an empty one of the same name.
179+
mkdir -p "${build_dir}/empty_component/executorch"
180+
echo "idf_component_register()" > "${build_dir}/empty_component/executorch/CMakeLists.txt"
181+
for name in empty-baseline empty; do
182+
if [[ "${name}" == empty-baseline ]]; then
183+
dir="${build_dir}/empty_component/executorch"
184+
else
185+
dir="${component_dir}"
186+
fi
187+
consumer "${name}" esp32s3 -DET_COMPONENT_DIR="${dir}" build
188+
log="${build_dir}/size-${name}.log" consumer "${name}" esp32s3 \
189+
-DET_COMPONENT_DIR="${dir}" size --format json2 \
190+
--output-file "${build_dir}/size-${name}.json"
191+
done
192+
python - "${build_dir}/size-empty-baseline.json" "${build_dir}/size-empty.json" <<'PY'
193+
import json
194+
import sys
195+
196+
baseline, empty = (json.load(open(p))["total_size"] for p in sys.argv[1:])
197+
if baseline != empty:
198+
sys.exit(f"FAIL: empty app is {empty} B with the component, {baseline} B without")
199+
print(f"OK: empty app is {empty} B with and without the component")
200+
PY
201+
49202
# With newlib's nano printf, the component compiles ExecuTorch logging out and
50203
# says so.
51204
printf 'CONFIG_LIBC_NEWLIB=y\nCONFIG_LIBC_NEWLIB_NANO_FORMAT=y\n' > "${build_dir}/nano.defaults"
52-
idf.py -C "${project}" -B "${build_dir}/firmware-nano" -DIDF_TARGET=esp32s3 \
53-
-DSDKCONFIG="${build_dir}/sdkconfig-nano" \
205+
consumer nano esp32s3 -DET_COMPONENT_DIR="${component_dir}" \
54206
"-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;${build_dir}/nano.defaults" \
55-
-DET_COMPONENT_DIR="${component_dir}" -DET_SMOKE_PTE="${build_dir}/smoke.pte" \
56-
reconfigure > "${build_dir}/firmware-nano.log" 2>&1
57-
# CMake wraps long messages, so match with whitespace runs collapsed.
207+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${kernels_dir}" reconfigure
58208
if ! tr -s ' \n' ' ' < "${build_dir}/firmware-nano.log" \
59209
| grep -qF "ExecuTorch logging is DISABLED"; then
60210
echo "FAIL: no nano printf warning; see ${build_dir}/firmware-nano.log" >&2
61211
exit 1
62212
fi
63213

64-
for profile in ${profiles//,/ }; do
65-
profile_upper=$(echo "${profile}" | tr '[:lower:]-' '[:upper:]_')
66-
firmware_dir="${build_dir}/firmware-${profile}"
67-
printf 'CONFIG_ET_OP_PROFILE_%s=y\n' "${profile_upper}" \
68-
> "${build_dir}/${profile}.defaults"
69-
idf.py -C "${project}" -B "${firmware_dir}" \
70-
-DIDF_TARGET=esp32s3 \
71-
-DSDKCONFIG="${build_dir}/sdkconfig-${profile}" \
72-
"-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;${build_dir}/${profile}.defaults" \
73-
-DET_COMPONENT_DIR="${component_dir}" \
74-
-DET_SMOKE_PTE="${build_dir}/smoke.pte" build \
75-
2>&1 | tee "${build_dir}/firmware-${profile}.log"
76-
done
214+
consumer smoke-esp32s3 esp32s3 -DET_COMPONENT_DIR="${component_dir}" \
215+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${kernels_dir}" build
216+
consumer smoke-esp32c6 esp32c6 -DET_COMPONENT_DIR="${component_dir}" \
217+
"-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;${build_dir}/large_registry.defaults" \
218+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${build_dir}/wakeword_kernels" build
77219

78-
firmware_dir="${build_dir}/firmware-smoke"
220+
firmware_dir="${build_dir}/firmware-smoke-esp32s3"
79221
idf.py -C "${project}" -B "${firmware_dir}" merge-bin \
80222
2>&1 | tee "${build_dir}/firmware-merge.log"
81223
timeout 130s esp-emu --chip esp32s3 \

‎.github/workflows/test-espressif.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ jobs:
5454
ESPRESSIF_NN_KERNELS="${{ matrix.kernels }}" bash .ci/scripts/test_espressif.sh
5555
5656
test-idf-component:
57-
name: ESP-IDF component (ESP32-S3)
57+
name: ESP-IDF component
5858
needs: docker-image
5959
uses: pytorch/test-infra/.github/workflows/linux_job_v3.yml@main
6060
permissions:

‎examples/espressif/README.md‎

Lines changed: 14 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -15,26 +15,24 @@ cross-compile-then-link flow below, generate a self-contained ESP-IDF
1515
component from sources in this repository:
1616

1717
```bash
18-
./examples/espressif/build_idf_component.sh --profiles wakeword_int8 \
18+
./examples/espressif/build_idf_component.sh \
1919
--out /path/to/your_project/components/executorch
2020
```
2121

2222
The output is a registry-shaped component (`idf_component.yml`,
23-
`CMakeLists.txt`, `Kconfig`, `VERSION`) containing the runtime,
24-
per-profile kernel registration (selected via
25-
`CONFIG_ET_OP_PROFILE_*`; the component is off until
26-
`CONFIG_ET_ENABLE=y`), the shared ESP-IDF PAL from
27-
`platform/esp_pal.cpp` (ESP_LOG + `esp_timer` timing), and a C
28-
inference API (`et_model.h`, flash-XIP model bytes
29-
with internal/PSRAM arena placement). Application pipelines built on
30-
the runtime (for example wake word front ends and detectors) stay
31-
with the application. Op profiles are defined in
32-
`idf_profiles/<name>.ops`; `wakeword_int8` is an int8 wake-word
33-
example profile for a Cadence-backend export — profiles may list
34-
`cadence::` ops, which the generator binds to the Cadence backend's
35-
generic (portable) kernels under `backends/cadence/generic`, next to
36-
the `aten::` ops the same profile needs from the portable kernels.
37-
Consumers need only ESP-IDF — no host CMake build, no flatc.
23+
`CMakeLists.txt`, `Kconfig`, `VERSION`) containing the runtime, the
24+
portable, quantized and Cadence generic kernel libraries, the shared
25+
ESP-IDF PAL from `platform/esp_pal.cpp` (ESP_LOG + `esp_timer` timing),
26+
and a C inference API (`et_model.h`, flash-XIP model bytes with
27+
internal/PSRAM arena placement). Only the kernels a project's models
28+
use are registered: the component's `tools/gen_kernels.py` makes a
29+
kernel bundle from the project's `.pte` files, and the project declares
30+
both with `executorch_add_model` and `executorch_add_kernels` (see the
31+
component's `README.md`). Application pipelines built on the runtime
32+
(for example wake word front ends and detectors) stay with the
33+
application.
34+
Consumers need ESP-IDF, plus the `executorch` Python package of the
35+
same release to make kernel bundles — no host CMake build, no flatc.
3836
`idf_component_smoke/` is a minimal consumer that CI generates, builds
3937
and runs on the ESP32-S3 emulator
4038
(`.ci/scripts/test_espressif_idf_component.sh`).

0 commit comments

Comments
 (0)