Enhancements for the esp32_flash task

CHANGELOG.md

@@ -17,13 +17,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Added dialyzer task to simplify running dialyzer on AtomVM applications.
 - Added support for rp2350 devices to allow for default detection of the device mount path.
 - Added configuration paramenter for setting the path to picotool for the pico_flash task.
+- Added `app_partition` parameter to `esp32_flash` task. This is only needed to be provided for
+custom partition tables that do not use `main.avm` for the beam application partition name, or to
+flash to a custom alternate partition.
 
 ### Changed
 - The `uf2create` task now creates `universal` format uf2 files by default, suitable for both
 rp2040 or rp2350 devices.
 - The `pico_flash` task now checks that a device is an RP2 platform before resetting to `BOOTSEL`
 mode, preventing interference with other MCUs that may be attached to the host system.
 - The `pico_flash` task now aborts on all errors rather than trying to continue after a failure.
+- The `offset` used by the `esp32_flash` task is now read from the partition table of the device.
+When this parameter is provided it will be used to verify the offset of the application partition on
+flash matches the expected value.
+- The `esp32_flash` task now uses auto discovery for the `port` by default.
+- Stacktraces are not shown by default if the `esp32_flash` fails, instead a descriptive error
+message is displayed. To view the stacktrace use diagnostic mode.
+
+### Fixed
+- The `esp32_flash` task aborts when an error occurs, rather than attempt to continue after a step
+has failed.
 
 ## [0.7.5] (2025.05.27)
 

README.md

@@ -255,24 +255,30 @@ Running this AVM file will boot the `myapp` application automatically, without h
 
 You may use the `esp32_flash` task to flash the generated AtomVM packbeam application to the flash storage on an ESP32 device connected over a serial connection.
 
-    shell$ rebar3 help atomvm esp32_flash
-
-    Use this plugin to flash an AtomVM packbeam file to an ESP32 device.
-
-    Usage: rebar3 atomvm esp32_flash [-e <esptool>] [-c <chip>] [-p <port>]
-                                    [-b <baud>] [-o <offset>]
-
-    -e, --esptool  Path to esptool.py
-    -c, --chip     ESP chip (default auto)
-    -p, --port     Device port (default /dev/ttyUSB0)
-    -b, --baud     Baud rate (default 115200)
-    -o, --offset   Offset (default 0x210000)
+```shell
+shell$ rebar3 help atomvm esp32_flash
+
+Use this plugin to flash an AtomVM packbeam file to an ESP32 device.
+
+Usage: rebar3 atomvm esp32_flash [-e <esptool>] [-c <chip>] [-p <port>]
+                                 [-b <baud>] [-o <offset>]
+                                 [-a <app_partition>]
+
+  -e, --esptool        Path to esptool.py
+  -c, --chip           ESP chip (default auto)
+  -p, --port           Device port (default auto discovery)
+  -b, --baud           Baud rate (default 115200)
+  -o, --offset         Offset (default read from device) *old behavior
+                       deprecated, use app_partition. When given, verifies
+                       expected offset to actual
+  -a, --app_partition  Application partition name (default main.avm)
+```
 
 The `esp32_flash` task will use the `esptool.py` command to flash the ESP32 device.  This tool is available via the <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/">IDF SDK</a>, or directly via <a href="https://github.com/espressif/esptool">github</a>.  The `esptool.py` command is also available via many package managers (e.g., MacOS Homebrew).
 
-By default, the `esp32_flash` task will assume the `esptool.py` command is available on the user's executable path.  Alternatively, you may specify the full path to the `esptool.py` command via the `-e` (or `--esptool`) option
+By default, the `esp32_flash` task will assume the `esptool.py` command is available on the user's executable path.  Alternatively, you may specify the full path to the `esptool.py` command via the `-e` (or `--esptool`) option.
 
-By default, the `esp32_flash` task will write to port `/dev/ttyUSB0` at a baud rate of `115200`.  You may control the port and baud settings for connecting to your ESP device via the `-port` and `-baud` options to the `esp32_flash` task, e.g.,
+By default, the `esp32_flash` task uses port auto discovery at a baud rate of `115200`.  You may control the port and baud settings for connecting to your ESP device via the `-port` and `-baud` options to the `esp32_flash` task, e.g.,
 
     shell$ rebar3 atomvm esp32_flash --port /dev/tty.SLAB_USBtoUART --baud 921600
     ...
@@ -301,7 +307,8 @@ The following table enumerates the properties that may be defined in your projec
 | `chip` | `string()` | ESP32 chip type |
 | `port` | `string()` | Device port on which the ESP32 can be located |
 | `baud` | `integer()` | Device BAUD rate |
-| `offset` | `string()` | Offset into which to write AtomVM application |
+| `offset` | `string()` | Optionally verify offset on flash matches expected value. Original behavior deprecated: use `app_partition` for custom images or to target a different partition |
+| `app_partition` | `string()` | Name of application partition to write AtomVM application for custom partition tables |
 
 Example:
 
@@ -314,9 +321,21 @@ Alternatively, the following environment variables may be used to control the ab
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_PORT`
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_BAUD`
 * `ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET`
+* `ATOMVM_REBAR3_PLUGIN_ESP32_APP_PARTITION`
 
 Any setting specified on the command line take precedence over settings in `rebar.config`, which in turn take precedence over environment variable settings, which in turn take precedence over the default values specified above.
 
+```note
+The  behavior of the `offset` configuration option has changed, the correct offset for standard
+AtomVM builds are determined by the partition table flashed to the device. Elixir supported builds
+are recognized and the correct offset will be used. When using a custom partition table it is
+necessary to supply the `app_partition` name. If an offset is given it will be compared to the
+address of the discovered `app_partition` and an error will be raised if they do not match. The
+partition name of the application partition for standard AtomVM partition tables is `main.avm`, but
+it is not necessary to supply the name when using a release image or one of the standard partition
+tables.
+```
+
 The `esp32_flash` task depends on the `packbeam` task, so the packbeam file will get automatically built if any changes have been made to its dependencies.
 
 ### The `stm32_flash` task

UPDATING.md

@@ -6,6 +6,17 @@
 
 # `atomvm_rebar3_plugin` Update Instructions
 
+## (unreleased)
+
+- The `esp32_flash` task now reads the application `offset` from the partition table on the device.
+If you are using a custom partition table that does not use `main.avm` for the application partition
+name you should supply the name used with the `app_partition` parameter. An `offset` may optionally
+be supplied to assure the offset of the application partition matches the expected offset, this may
+be helpful to assure that specific applications are only flashed to devices with a custom build of
+AtomVM.
+- Pico 2 (RP2350) devices are recognized and now work with default parameters. Specifying device
+path and uf2 flavor for these chipsets is no longer necessary.
+
 ## 0.6.* -> 0.7.*
 
 - The `atomvm_rebar3_plugin` tasks have been moved into the `atomvm` namespace (from the [`rebar3`](https://rebar3.org) `default` namespace).  The "legacy" tasks in the `default` namespace are deprecated, and users will be issued a warning when used.  Be sure to use the `atomvm` namespace in any future usage of this plugin, as the deprecated tasks may be removed without warning.  E.g., `rebar3 atomvm packbeam ...`

src/atomvm_esp32_flash_provider.erl

@@ -30,17 +30,21 @@
 -define(OPTS, [
     {esptool, $e, "esptool", string, "Path to esptool.py"},
     {chip, $c, "chip", string, "ESP chip (default auto)"},
-    {port, $p, "port", string, "Device port (default /dev/ttyUSB0)"},
+    {port, $p, "port", string, "Device port (default auto discovery)"},
     {baud, $b, "baud", integer, "Baud rate (default 115200)"},
-    {offset, $o, "offset", string, "Offset (default 0x210000)"}
+    {offset, $o, "offset", string,
+        "Offset (default read from device) *old behavior deprecated, use app_partition."
+        " When given, verifies expected offset to actual"},
+    {app_partition, $a, "app_partition", string, "Application partition name (default main.avm)"}
 ]).
 
 -define(DEFAULT_OPTS, #{
     esptool => "esptool.py",
     chip => "auto",
-    port => "/dev/ttyUSB0",
-    baud => 115200,
-    offset => "0x210000"
+    port => "auto",
+    baud => "115200",
+    offset => auto,
+    app_partition => "main.avm"
 }).
 
 %%
@@ -81,16 +85,25 @@ do(State) ->
             maps:get(chip, Opts),
             maps:get(port, Opts),
             maps:get(baud, Opts),
-            maps:get(offset, Opts)
+            maybe_convert_string(maps:get(offset, Opts)),
+            list_to_binary(maps:get(app_partition, Opts)),
+            State
         ),
         {ok, State}
     catch
+        C:rebar_abort:S ->
+            rebar_api:error(
+                "A fatal error occurred in the ~p task.",
+                [?PROVIDER]
+            ),
+            rebar_api:debug("Class=~p, Error=~p~nSTACKTRACE:~n~p~n", [C, rebar_abort, S]),
+            {error, rebar_abort};
         C:E:S ->
             rebar_api:error(
-                "An error occurred in the ~p task.  Class=~p Error=~p Stacktrace=~p~n", [
-                    ?PROVIDER, C, E, S
-                ]
+                "An unhandled error occurred in the ~p task.  Error=~p",
+                [?PROVIDER, E]
             ),
+            rebar_api:debug("Class=~p, Error=~p~nSTACKTRACE:~n~p~n", [C, E, S]),
             {error, E}
     end.
 
@@ -105,6 +118,7 @@ format_error(Reason) ->
 %% @private
 get_opts(State) ->
     {ParsedArgs, _} = rebar_state:command_parsed_args(State),
+    rebar_api:debug("ParsedArgs = ~p", [ParsedArgs]),
     RebarOpts = atomvm_rebar3_plugin:get_atomvm_rebar_provider_config(State, ?PROVIDER),
     ParsedOpts = atomvm_rebar3_plugin:proplist_to_map(ParsedArgs),
     maps:merge(
@@ -133,20 +147,47 @@ env_opts() ->
                 maps:get(baud, ?DEFAULT_OPTS)
             )
         ),
-        offset => os:getenv(
-            "ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET",
-            maps:get(offset, ?DEFAULT_OPTS)
+        offset => maybe_convert_string(
+            os:getenv(
+                "ATOMVM_REBAR3_PLUGIN_ESP32_FLASH_OFFSET",
+                maps:get(offset, ?DEFAULT_OPTS)
+            )
+        ),
+        app_partition => os:getenv(
+            "ATOMVM_REBAR3_PLUGIN_ESP32_APP_PARTITION",
+            maps:get(app_partition, ?DEFAULT_OPTS)
         )
     }.
 
 %% @private
 maybe_convert_string(S) when is_list(S) ->
-    list_to_integer(S);
+    case lists:prefix("0x", S) of
+        true ->
+            list_to_integer(lists:subtract(S, "0x"), 16);
+        false ->
+            list_to_integer(S)
+    end;
 maybe_convert_string(I) ->
     I.
 
 %% @private
-do_flash(ProjectApps, EspTool, Chip, Port, Baud, Offset) ->
+do_flash(ProjectApps, EspTool, Chip, Port, Baud, Address, Partition, State) ->
+    Offset =
+        case Address of
+            auto ->
+                read_flash_offset(EspTool, Port, Partition, State);
+            Val ->
+                Offset0 = read_flash_offset(EspTool, Port, Partition, State),
+                case Val =:= Offset0 of
+                    true ->
+                        Offset0;
+                    false ->
+                        rebar_api:abort(
+                            "The configured offset 0x~.16B does not match the partition table on the device (0x~.16B).",
+                            [Val, Offset0]
+                        )
+                end
+        end,
     [ProjectAppAVM | _] = [get_avm_file(ProjectApp) || ProjectApp <- ProjectApps],
     Portparam =
         case Port of
@@ -172,9 +213,12 @@ do_flash(ProjectApps, EspTool, Chip, Port, Baud, Offset) ->
         "keep",
         "--flash_size",
         "detect",
-        Offset,
+        integer_to_list(Offset),
         ProjectAppAVM
     ]),
+    AVMApp = filename:basename(ProjectAppAVM),
+    rebar_api:info("Flashing ~s to device.", [AVMApp]),
+    %% The following log output is parsed by the tests and should not be changed or removed.
     rebar_api:info("~s~n", [Cmd]),
     rebar_api:console("~s", [os:cmd(Cmd)]),
     ok.
@@ -185,3 +229,44 @@ get_avm_file(App) ->
     Name = binary_to_list(rebar_app_info:name(App)),
     DirName = filename:dirname(OutDir),
     filename:join(DirName, Name ++ ".avm").
+
+%% @private
+read_flash_offset(Esptool, Port, PartName, State) ->
+    TempFile = get_part_tempfile(State),
+    rebar_api:info("Reading application partition offset from device...", []),
+    try esp_partition:read_app_offset(Esptool, Port, PartName, TempFile) of
+        Offset ->
+            Offset
+    catch
+        _:invalid_partition_table:_ ->
+            rebar_api:abort("Invalid partition data!", []);
+        _:no_device:_ ->
+            rebar_api:abort("No ESP32 device attached!", []);
+        _:{partition_not_found, Partition}:_ ->
+            rebar_api:error("The partition ~s was not fount on device partition table!", [Partition]),
+            rebar_api:abort(
+                "When using a custom partition table always specify the 'app_partition' NAME.", []
+            );
+        _:{invalid_subtype, Type}:_ ->
+            rebar_api:abort("The partition ~s was found, but used invalid subtype 0x~s.", [
+                PartName, Type
+            ]);
+        _:invalid_partition_data:_ ->
+            rebar_api:abort("The partition ~s was found, but partition data is invalid.", [PartName]);
+        _:Error:_ ->
+            rebar_api:abort("Unexpected error reading partition table from device, ~p.", [Error])
+    end.
+
+%% @private
+get_part_tempfile(State) ->
+    OutDir = filename:absname(rebar_dir:base_dir(State)),
+    TempFile = filename:absname_join(OutDir, "part.tmp"),
+    case filelib:is_file(TempFile) of
+        true ->
+            rebar_api:debug("Removing possibly stale partition dump data ~s", [TempFile]),
+            Cmd = lists:join(" ", ["rm", TempFile]),
+            os:cmd(Cmd);
+        false ->
+            ok
+    end,
+    TempFile.