Add support for partition-tables. (#67)

toitlang · Nov 20, 2024 · cd4a43f · cd4a43f
1 parent ee51f38
commit cd4a43f
Show file tree

Hide file tree

Showing 7 changed files with 201 additions and 105 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -23,6 +23,11 @@ on:
         required: false
         type: boolean
         default: true
+      build-partitions:
+        description: Build partitions
+        required: false
+        type: boolean
+        default: true
       toit-version:
         description: Toit SDK version to check out
         required: true
@@ -192,7 +197,7 @@ jobs:
           artifacts: |
             envelopes/*
           body: |
-            Envelopes for Toit SDK ${{ github.event.inputs.toit-version }}
+            Envelopes and partitions for Toit SDK ${{ github.event.inputs.toit-version }}
           allowUpdates: true
           token: ${{ secrets.GITHUB_TOKEN }}
           tag: ${{ github.event.inputs.toit-version }}
@@ -300,7 +305,7 @@ jobs:
           artifacts: |
             envelopes/*
           body: |
-            Envelopes for Toit SDK ${{ github.event.inputs.toit-version }}
+            Envelopes and partition tables for Toit SDK ${{ github.event.inputs.toit-version }}
           allowUpdates: true
           token: ${{ secrets.GITHUB_TOKEN }}
           tag: ${{ github.event.inputs.toit-version }}
@@ -371,7 +376,47 @@ jobs:
           artifacts: |
             envelopes/*
           body: |
-            Envelopes for Toit SDK ${{ github.event.inputs.toit-version }}
+            Envelopes and partition tables for Toit SDK ${{ github.event.inputs.toit-version }}
+          allowUpdates: true
+          token: ${{ secrets.GITHUB_TOKEN }}
+          tag: ${{ github.event.inputs.toit-version }}
+          commit: main
+
+  release-partition-tables:
+    name: "Release partition tables - ${{ github.event.inputs.toit-version }}"
+    if: ${{ github.event.inputs.build-partitions == 'true' }}
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Show inputs
+        run: echo "${{ toJSON(github.event.inputs) }}"
+
+      - uses: actions/checkout@v4
+
+      - name: Set the git version
+        run: |
+          echo "TOIT_GIT_VERSION=${{ github.event.inputs.toit-version }}" >> $GITHUB_ENV
+
+      - name: Prepare partition tables
+        shell: bash
+        run: |
+          mkdir -p build
+          shopt -s nullglob
+          for csv_file in partitions/esp32/*/partitions.csv; do
+            variant=$(basename "$(dirname "$csv_file")")
+            if [ -e "partitions/esp32/$variant/partitions.csv" ]; then
+              cp "$csv_file" "build/partitions-esp32-$variant.csv"
+            fi
+          done
+
+      - name: Release
+        if: ${{ github.event.inputs.upload-release == 'true' }}
+        uses: ncipollo/release-action@v1
+        with:
+          artifacts: |
+            build/partitions-*.csv
+          body: |
+            Envelopes and partition tables for Toit SDK ${{ github.event.inputs.toit-version }}
           allowUpdates: true
           token: ${{ secrets.GITHUB_TOKEN }}
           tag: ${{ github.event.inputs.toit-version }}

diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 # Envelopes
 
-Envelopes for Toit.
+Envelopes and partition tables for Toit.
 
 This repository contains a tool to generate envelopes for Toit. It
 also automatically builds envelopes for the variants that are stored in
@@ -19,105 +19,10 @@ compiled with and for SDK v2.0.0-alpha.90 are found on the
 [v2.0.0-alpha.90 release](https://github.com/toitlang/envelopes/releases/tag/v2.0.0-alpha.90)
 page.
 
-## Variants
+See the [variants/README.md](variants/README.md) for a list of the
+most commonly used variants. You can also browse the `variants`
+directory for more variants.
 
-### esp32
-
-A generic ESP32 variant. This is the default variant when using Toit.
-It is built for maximum compatibility.
-
-This variant supports Ethernet, but without the clock output.
-
-### esp32-spiram
-
-An ESP32 variant for boards with SPIRAM. Otherwise the same as the ESP32 variant.
-
-### esp32-eth-clk-out0 and esp32-eth-clk-out17
-
-A variant for ESP32 boards with Ethernet and a clock output on pin 0/17.
-
-Olimex boards with Ethernet should use this variant. The WROOM versions need
-`esp32-eth-clk-out17` and the WROVER versions need `esp32-eth-clk-out0`.
-
-### esp32c3
-
-A generic [ESP32-C3 variant](variants/esp32c). This is the default variant
-when using Toit on ESP32-C3 boards.
-
-### esp32s2
-
-A generic [ESP32-S2 variant](variants/esp32s2). This is the default variant
-when using Toit on ESP32-S2 boards.
-
-### esp32s3
-
-A generic [ESP32-S3 variant](variants/esp32s3). This is the default variant
-when using Toit on ESP32-S3 boards.
-
-This variant is configured for external Quad PSRAM.
-
-### esp32s3-spiram-octo
-
-A [variant](variants/esp32s3-spiram-octo/) for ESP32-S3 boards with external
-octal PSRAM.
-
-These boards are faster, but often more expensive.
-
-### esp32-no-ble
-
-A [variant](variants/esp32-no-ble/) for ESP32 boards.  This variant
-saves some RAM and flash space by removing the Bluetooth stack.
-The saved IRAM enables us to make the Toit interpreter a little faster
-and add support for external RAM (PSRAM, aka SPIRAM).
-
-## Contributing
-
-Feel free to open issues and pull requests with new variants. Make sure
-they have a description (README.md) with the purpose and the changes.
-We will automatically build them whenever a new Toit version is released.
-
-Note that some variants are featured here. Consult the
-Toit team before adding new variants to this README.
-
-### Creating a variant
-
-Variants are created by copying an existing variant and making the
-necessary changes. These consists of either overwriting existing files
-or by applying patches.
-
-#### Partition changes
-
-For partition changes, simply copy the new `partitions.csv` into the
-variant directory. The `esp32-ota-1c0000` is an example of this where
-the OTA partition size has been increased.
-
-#### sdkconfig changes
-
-For `sdkconfig` changes, a patch to the original `sdkconfig.defaults`
-file is typically preferred.
-
-For example, to create a variant `esp32s3-foo`.
-* Check out Toit (or use an existing checkout).
-* Copy the existing `toolchains/esp32s3` directory to `toolchains/esp32s3-foo`:
-  `cp -r toolchains/esp32s3 toolchains/esp32s3-foo`.
-* Run `make IDF_TARGET=esp32s3 ESP32_CHIP=esp32s3-foo menuconfig` and make the changes you want.
-  This automatically updates the `sdkconfig.defaults` as well.
-* Create patch by running:
-  ```
-  diff -aur \
-    --label toit/toolchains/esp32s3/sdkconfig.defaults \
-    --label synthesized/esp32s3-foo/sdkconfig.defaults \
-    toolchains/esp32s3/sdkconfig.defaults \
-    toolchains/esp32s3-foo/sdkconfig.defaults \
-    > toolchains/esp32s3-foo/sdkconfig.defaults.patch
-  ```
-  The labels are not crucial, but make it easier for us to update the
-  patch at a later time.
-* Create a new variant in this (`envelopes`) repository and copy the patch file into it.
-
-#### Main changes
-
-For changes to the `main` directory (be it the `toit.c` or the `CMakelists.txt` in it),
-use a recursive diff to create a patch on the original `main` directory.
-
-The synthetization tool will use the flag `-p1` when applying the patch.
+See the [partitions/esp32/README.md](partitions/esp32/README.md) for
+a list of the most commonly used partition tables. You can also browse
+the `partitions/esp32` directory for more partition tables.
diff --git a/partitions/esp32/README.md b/partitions/esp32/README.md
@@ -0,0 +1,15 @@
+# Partition tables
+
+This directory contains partition tables for ESP32 devices and their variants.
+
+During flashing, users can override the default partition table that comes
+with the envelope that they are flashing.
+
+Here is a list of the most commonly used partition tables:
+
+## OTA-1C0000
+
+A partition table that sets the size of the OTA partitions
+to 0x1C0000 (1835008) bytes. This table is typically used for
+use-cases where programs are bundled with the firmware (like with
+[Artemis](https://github.com/toitware/artemis)).
diff --git a/partitions/esp32/ota-1c0000/README.md b/partitions/esp32/ota-1c0000/README.md
@@ -0,0 +1,7 @@
+# OTA-1C0000
+
+A partition table that sets the size of the OTA partitions
+to 0x1C0000 (1835008) bytes.
+
+As a consequence the size of the programs partition is set to
+0x60000 (393216) bytes.
diff --git a/partitions/esp32/ota-1c0000/partitions.csv b/partitions/esp32/ota-1c0000/partitions.csv
@@ -0,0 +1,17 @@
+# Copyright (C) 2023 Toitware ApS.
+#
+# Use of this source code is governed by a BSD0-style license that can be
+# found in the LICENSE_BSD0 file.
+
+# Partition Table for Toit.
+
+# Name,   Type, SubType,  Offset,    Size
+# bootloader,,  ,         0x001000,  0x007000
+# partitions,,  ,         0x008000,  0x000c00
+secure,   0x42, 0x00,     0x009000,  0x004000,
+otadata,  data, ota,      0x00d000,  0x002000,
+phy_init, data, phy,      0x00f000,  0x001000,
+ota_0,    app,  ota_0,    0x010000,  0x1c0000,
+ota_1,    app,  ota_1,    0x1d0000,  0x1c0000,
+nvs,      data, nvs,      0x390000,  0x010000,
+programs, 0x40, 0x00,     0x3A0000,  0x060000, encrypted
diff --git a/variants/Contributing.md b/variants/Contributing.md
@@ -0,0 +1,51 @@
+# Contributing
+
+Feel free to open issues and pull requests with new variants. Make sure
+they have a description (README.md) with the purpose and the changes. If
+they are generally useful, we will add them to the repository.
+
+Variants in this repository are automatically built whenever a new Toit
+version is released.
+
+## Creating a variant
+
+Variants are created by copying an existing variant and making the
+necessary changes. These consists of either overwriting existing files
+or by applying patches.
+
+### Partition changes
+
+For partition changes, simply copy the new `partitions.csv` into the
+variant directory. The `esp32-ota-1c0000` is an example of this where
+the OTA partition size has been increased.
+
+### sdkconfig changes
+
+For `sdkconfig` changes, a patch to the original `sdkconfig.defaults`
+file is typically preferred.
+
+For example, to create a variant `esp32s3-foo`.
+* Check out Toit (or use an existing checkout).
+* Copy the existing `toolchains/esp32s3` directory to `toolchains/esp32s3-foo`:
+  `cp -r toolchains/esp32s3 toolchains/esp32s3-foo`.
+* Run `make IDF_TARGET=esp32s3 ESP32_CHIP=esp32s3-foo menuconfig` and make the changes you want.
+  This automatically updates the `sdkconfig.defaults` as well.
+* Create patch by running:
+  ```
+  diff -aur \
+    --label toit/toolchains/esp32s3/sdkconfig.defaults \
+    --label synthesized/esp32s3-foo/sdkconfig.defaults \
+    toolchains/esp32s3/sdkconfig.defaults \
+    toolchains/esp32s3-foo/sdkconfig.defaults \
+    > toolchains/esp32s3-foo/sdkconfig.defaults.patch
+  ```
+  The labels are not crucial, but make it easier for us to update the
+  patch at a later time.
+* Create a new variant in this (`envelopes`) repository and copy the patch file into it.
+
+### Main changes
+
+For changes to the `main` directory (be it the `toit.c` or the `CMakelists.txt` in it),
+use a recursive diff to create a patch on the original `main` directory.
+
+The synthetization tool will use the flag `-p1` when applying the patch.
diff --git a/variants/README.md b/variants/README.md
@@ -0,0 +1,56 @@
+# Variants
+
+Some commonly used variants are featured here. You can also browse this
+directory for more variants.
+
+See the [Contributing](Contributing.md) guide for how to create a new variant.
+
+
+## esp32
+
+A generic ESP32 variant. This is the default variant when using Toit.
+It is built for maximum compatibility.
+
+This variant supports Ethernet, but without the clock output.
+
+## esp32-spiram
+
+An ESP32 variant for boards with SPIRAM. Otherwise the same as the ESP32 variant.
+
+## esp32-eth-clk-out0 and esp32-eth-clk-out17
+
+A variant for ESP32 boards with Ethernet and a clock output on pin 0/17.
+
+Olimex boards with Ethernet should use this variant. The WROOM versions need
+`esp32-eth-clk-out17` and the WROVER versions need `esp32-eth-clk-out0`.
+
+## esp32c3
+
+A generic [ESP32-C3 variant](esp32c). This is the default variant
+when using Toit on ESP32-C3 boards.
+
+## esp32s2
+
+A generic [ESP32-S2 variant](esp32s2). This is the default variant
+when using Toit on ESP32-S2 boards.
+
+## esp32s3
+
+A generic [ESP32-S3 variant](esp32s3). This is the default variant
+when using Toit on ESP32-S3 boards.
+
+This variant is configured for external Quad PSRAM.
+
+## esp32s3-spiram-octo
+
+A [variant](esp32s3-spiram-octo/) for ESP32-S3 boards with external
+octal PSRAM.
+
+These boards are faster, but often more expensive.
+
+## esp32-no-ble
+
+A [variant](esp32-no-ble/) for ESP32 boards.  This variant
+saves some RAM and flash space by removing the Bluetooth stack.
+The saved IRAM enables us to make the Toit interpreter a little faster
+and add support for external RAM (PSRAM, aka SPIRAM).