Automate a large portion of our release tasks

.gitignore

@@ -149,3 +149,4 @@ share/container.tar
 share/container.tar.gz
 share/image-id.txt
 container/container-pip-requirements.txt
+.doit.db.db

CHANGELOG.md

@@ -18,8 +18,8 @@ since 0.4.1, and this project adheres to [Semantic Versioning](https://semver.or
 
 ### Development changes
 
-- Build Dangerzone MSI with Wix Toolset 5 ([#929](https://github.com/freedomofpress/dangerzone/pull/929)).
   Thanks [@jkarasti](https://github.com/jkarasti) for the contribution.
+- Automate a large portion of our release tasks with `doit` ([#1016](https://github.com/freedomofpress/dangerzone/issues/1016))
 
 ## [0.8.0](https://github.com/freedomofpress/dangerzone/compare/v0.8.0...0.7.1)
 

Makefile

@@ -66,6 +66,22 @@ test-large: test-large-init  ## Run large test set
 	python -m pytest --tb=no tests/test_large_set.py::TestLargeSet -v $(JUNIT_FLAGS) --junitxml=$(TEST_LARGE_RESULTS)
 	python $(TEST_LARGE_RESULTS)/report.py $(TEST_LARGE_RESULTS)
 
+.PHONY: build-clean
+build-clean:
+	doit clean
+
+.PHONY: build-macos-intel
+build-macos-intel: build-clean
+	doit -n 8
+
+.PHONY: build-macos-arm
+build-macos-arm: build-clean
+	doit -n 8 macos_build_dmg
+
+.PHONY: build-linux
+build-linux: build-clean
+	doit -n 8 fedora_rpm debian_deb
+
 # Makefile self-help borrowed from the securedrop-client project
 # Explaination of the below shell command should it ever break.
 # 1. Set the field separator to ": ##" and any make targets that might appear between : and ##

RELEASE.md

@@ -76,7 +76,16 @@ Once we are confident that the release will be out shortly, and doesn't need any
 
 ### macOS Release
 
-This needs to happen for both Silicon and Intel chipsets.
+> [!TIP]
+> You can automate these steps from your macOS terminal app with:
+>
+> ```
+> export APPLE_ID=<email>
+> make build-macos-intel  # for Intel macOS
+> make build-macos-arm    # for Apple Silicon macOS
+> ```
+
+The following needs to happen for both Silicon and Intel chipsets.
 
 #### Initial Setup
 
@@ -217,12 +226,16 @@ Rename `Dangerzone.msi` to `Dangerzone-$VERSION.msi`.
 
 ### Linux release
 
-> [!INFO]
-> Below we explain how we build packages for each Linux distribution we support.
+> [!TIP]
+> You can automate these steps from any Linux distribution with:
+>
+> ```
+> make build-linux
+> ```
 >
-> There is also a `release.sh` script available which creates all
-> the `.rpm` and `.deb` files with a single command.
+> You can then add the created artifacts to the appropriate APT/YUM repo.
 
+Below we explain how we build packages for each Linux distribution we support.
 
 #### Debian/Ubuntu
 

docs/developer/doit.md

@@ -0,0 +1,67 @@
+# Using the Doit Automation Tool
+
+Developers can use the [Doit](https://pydoit.org/) automation tool to create
+release artifacts. The purpose of the tool is to automate the manual release
+instructions in `RELEASE.md` file. Not everything is automated yet, since we're
+still experimenting with this tool. You can find our task definitions in this
+repo's `dodo.py` file.
+
+## Why Doit?
+
+We picked Doit out of the various tools out there for the following reasons:
+
+* **Pythonic:** The configuration file and tasks can be written in Python. Where
+  applicable, it's easy to issue shell commands as well.
+* **File targets:** Doit borrows the file target concept from Makefiles. Tasks
+  can have file dependencies, and targets they build. This makes it easy to
+  define a dependency graph for tasks.
+* **Hash-based caching:** Unlike Makefiles, doit does not look at the
+  modification timestamp of source/target files, to figure out if it needs to
+  run them.  Instead, it hashes those files, and will run a task only if the
+  hash of a file dependency has changed.
+* **Parallelization:** Tasks can be run in parallel with the `-n` argument,
+  which is similar to `make`'s `-j` argument.
+
+## How to Doit?
+
+First, enter your Poetry shell. Then, make sure that your environment is clean,
+and you have ample disk space. You can run:
+
+```bash
+doit clean --dry-run  # if you want to see what would happen
+doit clean  # you'll be asked to cofirm that you want to clean everything
+```
+
+Finally, you can build all the release artifacts with `doit`, or a specific task
+with:
+
+```
+doit <task>
+```
+
+## Tips and tricks
+
+* You can run `doit list --all -s` to see the full list of tasks, their
+  dependencies, and whether they are up to date.
+* You can run `doit info <task>` to see which dependencies are missing.
+* You can change this line in `pyproject.toml` to `true`, to allow using the
+  Docker/Podman build cache:
+
+  ```
+  use_cache = true
+  ```
+
+  > [!WARNING]
+  > Using caching may speed up image builds, but is not suitable for release
+  > artifacts. The ID of our base container image (Alpine Linux) does not change
+  > that often, but its APK package index does. So, if we use caching, we risk
+  > skipping the `apk upgrade` layer and end up with packages that are days
+  > behind.
+
+* You can pass the following environment variables to the script, in order to
+  affect some global parameters:
+  - `CONTAINER_RUNTIME`: The container runtime to use. Either `podman` (default)
+    or `docker`.
+  - `RELEASE_DIR`: Where to store the release artifacts. Default path is
+    `~/release-assets/<version>`
+  - `APPLE_ID`: The Apple ID to use when signing/notarizing the macOS DMG.