Improve supply chain reproducibility and traceability for binary blobs and C headers

[skerkour]

Hello,
The README states:

espradio uses the binary blobs provided by Espressif...

It could be great if we added better traceability for the binary blobs. As of now the blobs could have been modified anywhere in the process and we wouldn't know.

I guess that they come from repos like https://github.com/espressif/esp32-wifi-lib but as of now there is no way to be 100% sure.

I would like to suggest the addition of a tool (go script, lets call it blosbmanager) to automatically fetch the blobs for a given git commit of the upstream repo(s), hash all the blobs with SHA-512 and generate a checksums.txt file and README.md file in the blobs folder to indicate which upstream commits have been used.

The content of blobs/checksums.txt would look like:

58a82a3f8d8941b7c7e0e2db45003d1470f75ba80638821945cc54732b180e9ac163fbfba4701b701499685438aa519ad918e2d00aec30d51e250307223b3f11  libs/esp32/libcore.a
...

We could also add a verify command to the script to fetch the binaries from upstream (from the pinned commit) and compare the hashes with the local binaries. This verify command would run as a CI workflow.

Ideally we should be able to verify hashes and signatures directly from Espressif, but I couldn't find anything in their repos.

It would look like: https://nodejs.org/dist/v26.7.0/SHASUMS256.txt

Let me know what you think. I may be able to work on it if we define a clear scope and spec.

[deadprogram]

The blobs come from the submodule esp-wifi @ 238eb0f which is an official repo. No checksum in there, however. Maybe you want to submit a PR to them?

The blobs in espradio are updated via the existing make tasks in the espradio repo, see https://github.com/tinygo-org/espradio/blob/main/Makefile

Regarding hashing in espradio itself, that seems like a good idea.

I would suggest generation/storage of hashes based on both the original binaries from the upstream repo, and then also based on the patched binaries that end up stored in this this repo. See https://github.com/tinygo-org/espradio/blob/main/tools/patch_xtensa_literals.go for the LLD patches that are used.

Any resulting hashes can then be stored with the build/release. This should accomodate some SBOM checking for the blobs used.

[skerkour]

Good information, thank you.

I've opened some issues upstream to discuss adding hashes and provenance information.

I see now, esp-wifi is actually a git submodule of https://github.com/esp-rs/esp-wifi-sys and the blobs are copied in the Makefile.

I've dug a little bit, and indeed it seems that https://github.com/esp-rs/esp-wifi-sys is getting the blobs from Espressif's repos (https://github.com/espressif/esp32-wifi-lib , https://github.com/espressif/esp-phy-lib and probably more). I've matched https://github.com/esp-rs/esp-wifi-sys/blob/238eb0ff6fceb1a34f99677b0b714b5507b703cf/esp-wifi-sys-esp32c3/libs/libcore.a with https://github.com/espressif/esp32-wifi-lib/blob/e0befaa593277b4e80726079fbd521b4681754c2/esp32c3/libcore.a which is already 6 months old and hasn't received many bug fixes.

Here are my thoughts: it would require some work, but I think that we should get the blobs directly from Espressif's repositories.

Add Espressif's repos as submodules and build a tool / script to copy the relevant blobs from upstream repositories and hash them (e.g. blobs-copy). Store the hashes directly in the repository and commit them. They can be added in release files later as you said.

Build a tool / script to verify that the hashes match (e.g. blobs-verify)

One problem is that these Espressif repos are really huge (the git history is full of binary blobs), so the other option is to do the same but keep https://github.com/esp-rs/esp-wifi-sys as the submodule (less good as we add supply chain attack vectors).

It would require to make a clear inventory of which blobs are strictly needed.

[skerkour]

Inventory of used / unused blobs (may not be 100% accurate but to give a rough idea):

esp32 (radio_esp32.go:13):

• blobs/libs/esp32/libcoexist.a
• blobs/libs/esp32/libcore.a
• blobs/libs/esp32/libespnow.a
• blobs/libs/esp32/libmesh.a
• blobs/libs/esp32/libnet80211.a
• blobs/libs/esp32/libphy.a
• blobs/libs/esp32/libpp.a
• blobs/libs/esp32/libregulatory.a
• blobs/libs/esp32/librtc.a
• blobs/libs/esp32/libwpa_supplicant.a

esp32c3 (radio_esp32c3.go:11):

• blobs/libs/esp32c3/libbtbb.a
• blobs/libs/esp32c3/libbtdm_app.a
• blobs/libs/esp32c3/libcoexist.a
• blobs/libs/esp32c3/libcore.a
• blobs/libs/esp32c3/libespnow.a
• blobs/libs/esp32c3/libmesh.a
• blobs/libs/esp32c3/libnet80211.a
• blobs/libs/esp32c3/libphy.a
• blobs/libs/esp32c3/libpp.a
• blobs/libs/esp32c3/libregulatory.a
• blobs/libs/esp32c3/libwpa_supplicant.a

esp32s3 (radio_esp32s3.go:12):

• blobs/libs/esp32s3/libcoexist.a
• blobs/libs/esp32s3/libcore.a
• blobs/libs/esp32s3/libespnow.a
• blobs/libs/esp32s3/libmesh.a
• blobs/libs/esp32s3/libnet80211.a
• blobs/libs/esp32s3/libphy.a
• blobs/libs/esp32s3/libpp.a
• blobs/libs/esp32s3/libregulatory.a
• blobs/libs/esp32s3/libwpa_supplicant.a

Unused (in blobs/libs but never in any LDFLAGS):

• blobs/libs/esp32/libprintf.a
• blobs/libs/esp32/libsmartconfig.a
• blobs/libs/esp32/libwapi.a
• blobs/libs/esp32/libbtdm_app.a
• blobs/libs/esp32c3/libprintf.a
• blobs/libs/esp32c3/libsmartconfig.a
• blobs/libs/esp32c3/libwapi.a
• blobs/libs/esp32s3/libbtbb.a
• blobs/libs/esp32s3/libbtdm_app.a
• blobs/libs/esp32s3/libprintf.a
• blobs/libs/esp32s3/libsmartconfig.a
• blobs/libs/esp32s3/libwapi.a

---

Inventory of used / unused headers (may not be 100% accurate but to give a rough idea):

Used headers: 43 / 227

Determined by running the actual C preprocessor (gcc -MM) with the exact cgo CFLAGS for all three supported targets (esp32, esp32c3, esp32s3, incl. the esp32c3_qemu test build), across every C shim + every cgo preamble.

31 chip-independent (pulled via espradio.h → include.h, the phy adapters, or transitively):

Header	Pulled in by
include.h, esp_wifi.h, esp_private/wifi.h, esp_private/wifi_os_adapter.h, esp_wpa.h, esp_phy_init.h, phy.h, esp_timer.h, esp_eap_client.h, esp_coexist_internal.h, esp_coexist_adapter.h, esp_now.h	include.h aggregator
esp_wifi_types.h, esp_wifi_types_generic.h, esp_wifi_crypto_types.h, esp_wifi_default.h	via esp_wifi.h
esp_smartconfig.h	via esp_private/wifi.h
esp_private/wifi_types.h	via esp_private/wifi.h
esp_event.h, esp_event_base.h, esp_interface.h, esp_err.h, esp_etm.h, esp_assert.h, esp_compiler.h	via wifi/phy/coexist headers
esp_coexist.h, hal/gpio_types.h	via esp_coexist_internal.h→esp_coexist.h
esp_etm.h	via esp_timer.h
esp_phy.h	only the per-chip esp_phy_adapter.c
btbb.h	only bt_ble.c (BLE, esp32c3 build)
esp_netif.h	empty stub, still in include closure
local/esp_wifi_types_native.h	via esp_wifi_types.h

12 chip-specific (4 per chip): esp_bt.h, sdkconfig.h, soc/interrupts.h, soc/soc_caps.h in each of esp32/, esp32c3/, esp32s3/.

Unused headers: 184 / 227

38 common top-level : e.g. esp_attr.h, esp_bit_defs.h, esp_types.h (include.h self-defines the basic types), esp_mesh.h/esp_mesh_internal.h (mesh not exposed), esp_wpa2.h/esp_wps.h/esp_dpp.h/esp_rrm.h/esp_wnm.h/esp_mbo.h/esp_wifi_ap_get_sta_list.h (unused wifi features), esp_wifi_he.h/esp_wifi_he_types.h (Wi-Fi 6/HE), esp_wifi_netif.h, esp_system.h/esp_task.h/esp_timer_shim-adjacent task/wdt headers, esp_coexist_debug.h, esp_coex_i154.h (C6/H2/C5 only), nvs*.h, libunwind.h, smartconfig_ack.h, esp_bt_cfg.h (top-level stub), esp_debug_helpers.h, esp_freertos_hooks.h, esp_expression_with_stack.h, esp_ipc*.h, esp_modem_wrapper.h, esp_phy_cert_test.h, esp_supplicant_utils.h, esp_system_console.h, esp_systick_etm.h, esp_task_wdt.h, esp_xt_wdt.h, esp_event_loop.h.

6 in esp_private/ : esp_modem_clock.h, esp_pmu.h, esp_wifi_private.h, esp_wifi_types_private.h, esp_wifi_he_private.h, esp_wifi_he_types_private.h.

6 in hal/ : assert.h, efuse_hal.h, misc.h, modem_clock_hal.h, modem_clock_types.h, pmu_types.h.

2 in nimble/ : nimble_npl.h, nimble_npl_os.h (NimBLE controller).

132 in chip dirs :

• 21 for supported chips (7 each): esp_bt_vs.h, phy_init_data.h, phy_init_deps.h, soc/clk_tree_defs.h, soc/periph_defs.h, soc/reg_base.h, soc/soc.h in esp32/, esp32c3/, esp32s3/.
• 111 for unsupported chips (esp32c2/, esp32c5/, esp32c6/, esp32c61/, esp32h2/, esp32s2/), entirely unused today.

[skerkour]

Okay, I think I have now a clear understanding of where the blobs and headers come from, and how we could improve this far-from-ideal situation.

The blobs are copied from https://github.com/esp-rs/esp-wifi-sys which is the esp-wifi git submodule.

https://github.com/esp-rs/esp-wifi-sys get the blobs from https://github.com/esp-rs/esp-wireless-drivers-3rdparty

Some of the blobs are copied from a local esp-idf installation, and some blobs are actually compiled by https://github.com/esp-rs/esp-wireless-drivers-3rdparty

Problems

This is not ideal for at least 2 reasons:

• We have no assurance that the blobs and headers are matching upstream esp-idf and have not been modified in the process, especially since they use a local esp-idf install instead of a git submodule.
• tinygo is now bound to the update pace of https://github.com/esp-rs and may miss many bug fixes and security fixes from upstream esp-idf.

Also, https://github.com/esp-rs/esp-wireless-drivers-3rdparty compiles at least 3 libraries (libregulatory/libwpa_supplicant/lprintf) with toolchains from https://github.com/espressif/esp-idf

Solution

The goal is to have a reproducible and traceable supply chain. This is especially important with the new regulations.

Here is my proposal: Instead of getting the blobs and headers from https://github.com/esp-rs/esp-wifi-sys tinygo should get most of the blobs and headers directly from https://github.com/espressif/esp-idf

It will requires to create tools similar to https://github.com/esp-rs/esp-wireless-drivers-3rdparty

Here is what it would look like:

• update esp-wifi submodule to the current repo from https://github.com/esp-rs/esp-wifi to https://github.com/esp-rs/esp-wifi-sys
• Add https://github.com/espressif/esp-idf as submodule in addition to https://github.com/esp-rs/esp-wifi-sys with the same pinned commit for v5.5.3.
• create a tool to copy / hash headers and blobs from both https://github.com/espressif/esp-idf and https://github.com/esp-rs/esp-wifi-sys (libregulatory/libwpa_supplicant/lprintf). This tool will copy only the strictly necessary and allowlisted binary blobs and headers. Keep the exact current layout for folders (include/, libs/).
• (optional) create a tool to verify that the hashes of the (non-patched and non-compiled) binary blobs copied from esp-idf match upstream.

Later, tinygo can implement its own compilation process for libregulatory / libwpa_supplicant, remove all dependency on https://github.com/esp-rs/esp-wifi-sys and get updates directly from https://github.com/espressif/esp-idf at its own pace to get the latest bug and security fixes.

Appendix

Here is a short summary of the provenance of the headers and blobs that I've traced (may not be 100% accurate).

ESP-IDF use 4 binary repos as git submodules, pinned per release:

espradio lib	Upstream repo	ESP-IDF submodule path
libcore, libespnow, libmesh, libnet80211, libpp, libsmartconfig, libwapi	espressif/esp32-wifi-lib	components/esp_wifi/lib
libphy, libbtbb	espressif/esp-phy-lib	components/esp_phy/lib
libcoexist	espressif/esp-coex-lib	components/esp_coex/lib
libbtdm_app (C3/S3)	espressif/esp32c3-bt-lib	components/bt/controller/lib_esp32c3_family
libble_app (C6)	espressif/esp32c6-bt-lib	components/bt/controller/lib_esp32c6/esp32c6-bt-lib
libbtdm_app (ESP32)	espressif/esp32-bt-lib	components/bt/controller/lib_esp32

Three libs are NOT prebuilt blobs: esp-wireless-drivers-3rdparty builds them from IDF source: libregulatory.a (single file esp_wifi/src/regulatory/esp_wifi_regulatory.c), libwpa_supplicant.a (components/wpa_supplicant), and libprintf.a (its own helper-project main/ printf). These can't be fetched from a binary repo.

Where the headers come from: esp-idf

The xtask in esp-wireless-drivers-3rdparty copies specific include dirs out of ESP-IDF (that's what esp-wifi-sys c/include == blobs/include is):

• wpa_supplicant/esp_supplicant/include, esp_phy/include(+esp_private), esp_coex/include(+private), esp_wifi/include(+esp_private+local)
• esp_timer, esp_system, esp_event, nvs_flash include dirs
• individual files from esp_common, esp_hw_support, hal
• per-chip: components/soc/{chip}/include/soc/* + register/soc/reg_base.h + (C6/H2) pmu_struct/pmu_reg.h + modem/modem_syscon_struct.h + modem_lpcon_struct.h + components/hal/{chip}/include/hal/modem_*_ll.h
• sdkconfig.h = generated by the helper build
• plus include-path patches and the espradio-specific include.h aggregator

[skerkour]

@deadprogram let me know if the solution looks good to you so I can start working on a PR . If it's good, please let me know what I need to know (e.g. commands to run / code to compile) to test that everything is working.

Also, to keep the repo clean, I would like to move the submodules (https://github.com/esp-rs/esp-wifi-sys and https://github.com/espressif/esp-idf into the blobs folder).