Skip to content

Commit af61b8d

Browse files
rascanirascani
committed
Register the ESP-IDF component's kernels from a project kernel bundle
The generated component no longer bakes in op profiles. It vendors every kernel source of the portable, quantized and Cadence generic kernel libraries into a plain static library, ships gen_kernels.py with the libraries' yamls and ExecuTorch's codegen templates under tools/, and registers the kernels of the project's kernel bundle. A project declares its models and bundle with executorch_add_model and executorch_add_kernels from project_include.cmake. At the end of the project's configure, a deferred step checks that the bundle was made for this component, by version and commit, and for exactly the declared models (by SHA-256), rejects delegates the component does not provide yet, and compiles the bundle's registration and prim op selection into the component. A mismatch fails at configure time and prints the gen_kernels.py command that fixes it. A bundle used without declared models is used as is, with a warning that names the models it was made from. Registration no longer needs WHOLE_ARCHIVE. Each registration file is compiled through a small wrapper that defines an extern "C" anchor, and the deferred step adds -u for each one, which reaches the link from a deferred call. Only the bundle's files are anchored, so a project that declares no models and no bundle links nothing from the component: CI checks that such an app is the same size with and without it (169,728 B on ESP32-S3). The kernel library is not even compiled without a bundle. et_model_create references a symbol that only the bundle's registration defines, so an app that runs a model without declaring one fails to link on et_requires_executorch_add_kernels instead of failing every load with missing operators. CONFIG_ET_ENABLE goes away with the profile choice. The deferred step reads the bundle's kernel count from its selected_max_kernel_num.h, which it requires, and sizes the registry with it unless CONFIG_ET_MAX_KERNEL_NUM, now relative to the bundle, sets a larger size for an application that registers its own kernels; configuring one smaller than the bundle is an error. The smoke image on ESP32-S3 is 253,668 B, down from 266,660 B with the smoke profile, because the bundle selects only the prim ops the model uses (none here); with all 28 prims selected it is 266,684 B. CI also builds the smoke consumer for ESP32-C6 (setup.sh gains --targets) with a bundle that adds the ops of the Cadence-quantized wake word model, so the Cadence generic kernels stay compiled and linked. It checks that a re-exported model, a model the bundle was not made from, a bundle made for another component version or commit or missing its kernel count, and models without a bundle each fail at configure time, that a bundle used as is names its models, and that an app running an undeclared model fails to link. Projects that use an op profile, such as CONFIG_ET_OP_PROFILE_WAKEWORD_INT8, drop CONFIG_ET_ENABLE and the profile and move to a kernel bundle made from their model. Authored with Claude Code.
1 parent 39a872b commit af61b8d

16 files changed

Lines changed: 823 additions & 638 deletions

File tree

‎.ci/scripts/test_espressif_idf_component.sh‎

Lines changed: 169 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,12 @@
55
# This source code is licensed under the BSD-style license found in the
66
# LICENSE file in the root directory of this source tree.
77

8-
# Generates the ExecuTorch ESP-IDF component with every shipped profile,
9-
# builds the idf_component_smoke consumer for ESP32-S3 against each, and runs
10-
# the smoke profile build on the emulator.
8+
# Generates the ExecuTorch ESP-IDF component and a kernel bundle for the smoke
9+
# model, builds the idf_component_smoke consumer for ESP32-S3 and, with the
10+
# Cadence kernels too, for ESP32-C6, and runs the ESP32-S3 build on the
11+
# emulator. Also checks that stale or mismatched bundles fail at configure time,
12+
# and that an app declaring nothing links to the same size as one without the
13+
# component.
1114

1215
set -euo pipefail
1316

@@ -21,7 +24,8 @@ cd "${et_root}"
2124
collect_artifacts() {
2225
if [[ -n "${RUNNER_ARTIFACT_DIR:-}" ]]; then
2326
mkdir -p "${RUNNER_ARTIFACT_DIR}/esp-idf-component"
24-
for file in "${build_dir}"/*.log "${build_dir}/components/executorch/VERSION"; do
27+
for file in "${build_dir}"/*.log "${build_dir}/components/executorch/VERSION" \
28+
"${build_dir}/smoke_kernels/kernels.json"; do
2529
if [[ -f "${file}" ]]; then
2630
cp "${file}" "${RUNNER_ARTIFACT_DIR}/esp-idf-component/"
2731
fi
@@ -30,7 +34,8 @@ collect_artifacts() {
3034
}
3135
trap collect_artifacts EXIT
3236

33-
bash examples/espressif/setup.sh --minimal 2>&1 | tee "${build_dir}/setup.log"
37+
bash examples/espressif/setup.sh --minimal --targets=esp32s3,esp32c6 2>&1 \
38+
| tee "${build_dir}/setup.log"
3439
# shellcheck source=/dev/null
3540
source "${ESPRESSIF_TOOLS_DIR}/setup_path.sh"
3641

@@ -39,43 +44,179 @@ python examples/espressif/export_smoke_model.py \
3944
2>&1 | tee "${build_dir}/export.log"
4045

4146
component_dir="${build_dir}/components/executorch"
42-
profiles=$(find examples/espressif/idf_profiles -name '*.ops' -exec basename {} .ops \; \
43-
| sort | paste -sd, -)
44-
bash examples/espressif/build_idf_component.sh --profiles "${profiles}" \
45-
--out "${component_dir}" 2>&1 | tee "${build_dir}/generate.log"
47+
kernels_dir="${build_dir}/smoke_kernels"
48+
bash examples/espressif/build_idf_component.sh --out "${component_dir}" \
49+
2>&1 | tee "${build_dir}/generate.log"
50+
python "${component_dir}/tools/gen_kernels.py" --pte "${build_dir}/smoke.pte" \
51+
--out "${kernels_dir}" 2>&1 | tee "${build_dir}/gen_kernels.log"
52+
# The ESP32-C6 build also registers the ops of a Cadence-quantized streaming
53+
# wake word model, so CI compiles and links the Cadence generic kernels.
54+
wakeword_ops=aten::_softmax.out,aten::_to_copy.out,aten::cat.out,aten::copy_
55+
wakeword_ops+=,aten::permute_copy.out,aten::slice_copy.Tensor_out
56+
wakeword_ops+=,aten::squeeze_copy.dim_out,aten::transpose_copy.int_out
57+
wakeword_ops+=,cadence::dequantize_per_tensor.out,cadence::quantize_per_tensor.out
58+
wakeword_ops+=,cadence::quantized_conv2d_nhwc.per_tensor_out
59+
wakeword_ops+=,cadence::quantized_fully_connected.per_tensor_out
60+
wakeword_ops+=,cadence::quantized_relu.per_tensor_out
61+
python "${component_dir}/tools/gen_kernels.py" --pte "${build_dir}/smoke.pte" \
62+
--ops "${wakeword_ops}" --out "${build_dir}/wakeword_kernels" \
63+
2>&1 | tee -a "${build_dir}/gen_kernels.log"
4664

4765
project=examples/espressif/idf_component_smoke
4866

67+
# Runs idf.py on the smoke consumer: build name, target, then idf.py arguments.
68+
consumer() {
69+
local name=$1 target=$2
70+
shift 2
71+
IDF_TARGET="${target}" idf.py -C "${project}" -B "${build_dir}/firmware-${name}" \
72+
-DSDKCONFIG="${build_dir}/sdkconfig-${name}" "$@" \
73+
> "${build_dir}/firmware-${name}.log" 2>&1
74+
}
75+
76+
# Configures a case that must fail: build name, expected message, -D arguments.
77+
expect_configure_error() {
78+
local name=$1 expected=$2
79+
shift 2
80+
if consumer "${name}" esp32s3 -DET_COMPONENT_DIR="${component_dir}" "$@" reconfigure; then
81+
echo "FAIL: ${name} configured; expected an error with '${expected}'" >&2
82+
exit 1
83+
fi
84+
# CMake wraps long messages, so match with whitespace runs collapsed.
85+
if ! tr -s ' \n' ' ' < "${build_dir}/firmware-${name}.log" | grep -qF "${expected}"; then
86+
echo "FAIL: ${name} did not report '${expected}'; see firmware-${name}.log" >&2
87+
exit 1
88+
fi
89+
echo "OK: ${name} failed at configure with '${expected}'"
90+
}
91+
92+
# A re-exported smoke.pte.
93+
mkdir -p "${build_dir}/stale"
94+
cp "${build_dir}/smoke.pte" "${build_dir}/stale/smoke.pte"
95+
printf '\0' >> "${build_dir}/stale/smoke.pte"
96+
expect_configure_error stale-model "smoke.pte: changed since the bundle was made" \
97+
-DET_SMOKE_PTE="${build_dir}/stale/smoke.pte" -DET_SMOKE_KERNELS="${kernels_dir}"
98+
99+
rm -rf "${build_dir}/mismatch_kernels"
100+
cp -r "${kernels_dir}" "${build_dir}/mismatch_kernels"
101+
python - "${build_dir}/mismatch_kernels/kernels.json" <<'PY'
102+
import json
103+
import sys
104+
105+
bundle = json.load(open(sys.argv[1]))
106+
bundle["component_version"] = "0.0.0"
107+
json.dump(bundle, open(sys.argv[1], "w"))
108+
PY
109+
expect_configure_error version-mismatch component_version \
110+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" \
111+
-DET_SMOKE_KERNELS="${build_dir}/mismatch_kernels"
112+
113+
# A bundle made by a component of the same version from another commit, as any
114+
# two commits between version.txt bumps are.
115+
rm -rf "${build_dir}/mismatch_kernels"
116+
cp -r "${kernels_dir}" "${build_dir}/mismatch_kernels"
117+
python - "${build_dir}/mismatch_kernels/kernels.json" <<'PY'
118+
import json
119+
import sys
120+
121+
bundle = json.load(open(sys.argv[1]))
122+
bundle["component_commit"] = "0" * 40
123+
json.dump(bundle, open(sys.argv[1], "w"))
124+
PY
125+
expect_configure_error commit-mismatch component_commit \
126+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" \
127+
-DET_SMOKE_KERNELS="${build_dir}/mismatch_kernels"
128+
129+
expect_configure_error no-bundle "no kernel bundle is" -DET_SMOKE_PTE="${build_dir}/smoke.pte"
130+
131+
# A model the bundle was not made from, declared instead of the one it was.
132+
mkdir -p "${build_dir}/other"
133+
cp "${build_dir}/stale/smoke.pte" "${build_dir}/other/other.pte"
134+
expect_configure_error other-model \
135+
"other.pte: not in the bundle smoke.pte: in the bundle, not declared" \
136+
-DET_SMOKE_PTE="${build_dir}/other/other.pte" -DET_SMOKE_KERNELS="${kernels_dir}"
137+
138+
rm -rf "${build_dir}/no_count_kernels"
139+
cp -r "${kernels_dir}" "${build_dir}/no_count_kernels"
140+
rm -r "${build_dir}/no_count_kernels/include"
141+
expect_configure_error no-kernel-count "has no kernel count" \
142+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${build_dir}/no_count_kernels"
143+
144+
# A bundle used without declaring its models is used as is, with a warning that
145+
# names them.
146+
consumer as-is esp32s3 -DET_COMPONENT_DIR="${component_dir}" \
147+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_UNDECLARED=1 \
148+
-DET_SMOKE_KERNELS="${kernels_dir}" reconfigure
149+
if ! tr -s ' \n' ' ' < "${build_dir}/firmware-as-is.log" \
150+
| grep -qF "is not checked against smoke.pte"; then
151+
echo "FAIL: no warning naming smoke.pte; see ${build_dir}/firmware-as-is.log" >&2
152+
exit 1
153+
fi
154+
echo "OK: as-is warned that smoke.pte is not checked"
155+
156+
# An app that runs a model while declaring nothing fails to link.
157+
if consumer undeclared esp32s3 -DET_COMPONENT_DIR="${component_dir}" \
158+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_UNDECLARED=1 build; then
159+
echo "FAIL: undeclared linked; expected et_requires_executorch_add_kernels to be undefined" >&2
160+
exit 1
161+
fi
162+
if ! grep -qF "undefined reference to \`et_requires_executorch_add_kernels'" \
163+
"${build_dir}/firmware-undeclared.log"; then
164+
echo "FAIL: undeclared did not fail on et_requires_executorch_add_kernels; see firmware-undeclared.log" >&2
165+
exit 1
166+
fi
167+
echo "OK: undeclared failed to link on et_requires_executorch_add_kernels"
168+
169+
# A kernel registry smaller than the bundle; the emulator run below uses one
170+
# larger than it.
171+
printf 'CONFIG_ET_MAX_KERNEL_NUM=1\n' > "${build_dir}/small_registry.defaults"
172+
printf 'CONFIG_ET_MAX_KERNEL_NUM=64\n' > "${build_dir}/large_registry.defaults"
173+
expect_configure_error small-registry "CONFIG_ET_MAX_KERNEL_NUM=1 is below" \
174+
"-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;${build_dir}/small_registry.defaults" \
175+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${kernels_dir}"
176+
177+
# An app that declares nothing must not grow because the component is in the
178+
# build. The baseline swaps the component for an empty one of the same name.
179+
mkdir -p "${build_dir}/empty_component/executorch"
180+
echo "idf_component_register()" > "${build_dir}/empty_component/executorch/CMakeLists.txt"
181+
for name in empty-baseline empty; do
182+
if [[ "${name}" == empty-baseline ]]; then
183+
dir="${build_dir}/empty_component/executorch"
184+
else
185+
dir="${component_dir}"
186+
fi
187+
consumer "${name}" esp32s3 -DET_COMPONENT_DIR="${dir}" build
188+
consumer "${name}" esp32s3 -DET_COMPONENT_DIR="${dir}" size --format json2 \
189+
--output-file "${build_dir}/size-${name}.json"
190+
done
191+
python - "${build_dir}/size-empty-baseline.json" "${build_dir}/size-empty.json" <<'PY'
192+
import json
193+
import sys
194+
195+
baseline, empty = (json.load(open(p))["total_size"] for p in sys.argv[1:])
196+
if baseline != empty:
197+
sys.exit(f"FAIL: empty app is {empty} B with the component, {baseline} B without")
198+
print(f"OK: empty app is {empty} B with and without the component")
199+
PY
200+
49201
# With newlib's nano printf, the component compiles ExecuTorch logging out and
50202
# says so.
51203
printf 'CONFIG_LIBC_NEWLIB=y\nCONFIG_LIBC_NEWLIB_NANO_FORMAT=y\n' > "${build_dir}/nano.defaults"
52-
idf.py -C "${project}" -B "${build_dir}/firmware-nano" -DIDF_TARGET=esp32s3 \
53-
-DSDKCONFIG="${build_dir}/sdkconfig-nano" \
204+
consumer nano esp32s3 -DET_COMPONENT_DIR="${component_dir}" \
54205
"-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;${build_dir}/nano.defaults" \
55-
-DET_COMPONENT_DIR="${component_dir}" -DET_SMOKE_PTE="${build_dir}/smoke.pte" \
56-
reconfigure > "${build_dir}/firmware-nano.log" 2>&1
57-
# CMake wraps long messages, so match with whitespace runs collapsed.
206+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${kernels_dir}" reconfigure
58207
if ! tr -s ' \n' ' ' < "${build_dir}/firmware-nano.log" \
59208
| grep -qF "ExecuTorch logging is DISABLED"; then
60209
echo "FAIL: no nano printf warning; see ${build_dir}/firmware-nano.log" >&2
61210
exit 1
62211
fi
63212

64-
for profile in ${profiles//,/ }; do
65-
profile_upper=$(echo "${profile}" | tr '[:lower:]-' '[:upper:]_')
66-
firmware_dir="${build_dir}/firmware-${profile}"
67-
printf 'CONFIG_ET_OP_PROFILE_%s=y\n' "${profile_upper}" \
68-
> "${build_dir}/${profile}.defaults"
69-
idf.py -C "${project}" -B "${firmware_dir}" \
70-
-DIDF_TARGET=esp32s3 \
71-
-DSDKCONFIG="${build_dir}/sdkconfig-${profile}" \
72-
"-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;${build_dir}/${profile}.defaults" \
73-
-DET_COMPONENT_DIR="${component_dir}" \
74-
-DET_SMOKE_PTE="${build_dir}/smoke.pte" build \
75-
2>&1 | tee "${build_dir}/firmware-${profile}.log"
76-
done
213+
consumer smoke-esp32s3 esp32s3 -DET_COMPONENT_DIR="${component_dir}" \
214+
"-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;${build_dir}/large_registry.defaults" \
215+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${kernels_dir}" build
216+
consumer smoke-esp32c6 esp32c6 -DET_COMPONENT_DIR="${component_dir}" \
217+
-DET_SMOKE_PTE="${build_dir}/smoke.pte" -DET_SMOKE_KERNELS="${build_dir}/wakeword_kernels" build
77218

78-
firmware_dir="${build_dir}/firmware-smoke"
219+
firmware_dir="${build_dir}/firmware-smoke-esp32s3"
79220
idf.py -C "${project}" -B "${firmware_dir}" merge-bin \
80221
2>&1 | tee "${build_dir}/firmware-merge.log"
81222
timeout 130s esp-emu --chip esp32s3 \

‎.github/workflows/test-espressif.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ jobs:
5454
ESPRESSIF_NN_KERNELS="${{ matrix.kernels }}" bash .ci/scripts/test_espressif.sh
5555
5656
test-idf-component:
57-
name: ESP-IDF component (ESP32-S3)
57+
name: ESP-IDF component
5858
needs: docker-image
5959
uses: pytorch/test-infra/.github/workflows/linux_job_v3.yml@main
6060
permissions:

‎examples/espressif/README.md‎

Lines changed: 14 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -15,26 +15,24 @@ cross-compile-then-link flow below, generate a self-contained ESP-IDF
1515
component from sources in this repository:
1616

1717
```bash
18-
./examples/espressif/build_idf_component.sh --profiles wakeword_int8 \
18+
./examples/espressif/build_idf_component.sh \
1919
--out /path/to/your_project/components/executorch
2020
```
2121

2222
The output is a registry-shaped component (`idf_component.yml`,
23-
`CMakeLists.txt`, `Kconfig`, `VERSION`) containing the runtime,
24-
per-profile kernel registration (selected via
25-
`CONFIG_ET_OP_PROFILE_*`; the component is off until
26-
`CONFIG_ET_ENABLE=y`), the shared ESP-IDF PAL from
27-
`platform/esp_pal.cpp` (ESP_LOG + `esp_timer` timing), and a C
28-
inference API (`et_model.h`, flash-XIP model bytes
29-
with internal/PSRAM arena placement). Application pipelines built on
30-
the runtime (for example wake word front ends and detectors) stay
31-
with the application. Op profiles are defined in
32-
`idf_profiles/<name>.ops`; `wakeword_int8` is an int8 wake-word
33-
example profile for a Cadence-backend export — profiles may list
34-
`cadence::` ops, which the generator binds to the Cadence backend's
35-
generic (portable) kernels under `backends/cadence/generic`, next to
36-
the `aten::` ops the same profile needs from the portable kernels.
37-
Consumers need only ESP-IDF — no host CMake build, no flatc.
23+
`CMakeLists.txt`, `Kconfig`, `VERSION`) containing the runtime, the
24+
portable, quantized and Cadence generic kernel libraries, the shared
25+
ESP-IDF PAL from `platform/esp_pal.cpp` (ESP_LOG + `esp_timer` timing),
26+
and a C inference API (`et_model.h`, flash-XIP model bytes with
27+
internal/PSRAM arena placement). Only the kernels a project's models
28+
use are registered: the component's `tools/gen_kernels.py` makes a
29+
kernel bundle from the project's `.pte` files, and the project declares
30+
both with `executorch_add_model` and `executorch_add_kernels` (see the
31+
component's `README.md`). Application pipelines built on the runtime
32+
(for example wake word front ends and detectors) stay with the
33+
application.
34+
Consumers need ESP-IDF, plus the `executorch` Python package of the
35+
same release to make kernel bundles — no host CMake build, no flatc.
3836
`idf_component_smoke/` is a minimal consumer that CI generates, builds
3937
and runs on the ESP32-S3 emulator
4038
(`.ci/scripts/test_espressif_idf_component.sh`).

0 commit comments

Comments
 (0)