Rewrite-clj is verified on each push on macOS, Ubuntu and Windows via GitHub Actions.
All scripts are written in Clojure and most invoked via babashka tasks. This gives us a cross platform scripting language that is familiar, fun and consistent.
We make use of planck for cljs bootstrap (aka cljs self-hosted) testing. Planck is currently not available for Windows.
We test that rewrite-clj operates as expected when natively compile via GraalVM. Automated testing is setup using GraalVM for JDK 23. At this time we only test against the Community Edition.
-
Java JDK 1.8 or above (shadow-cljs tests require min of JDK 11)
-
NodeJs v12 or above
-
Clojure v1.10.1.697 or above for
clojurecommand-
Note that rewrite-clj v1 itself supports Clojure v1.8 and above
-
-
Babashka v0.3.7 or above
-
Current release of GraalVM for JDK 23, if you want to run GraalVM native image tests
The primary development OSes for rewrite-clj are macOS and Linux. Our line endings are LF only.
I’m not sure what Windows developers typically want for line endings while working on source. I expect, but don’t know, that most Windows editors automatically handle LF as line ending. Someone let me know if I am wrong.
Note that I do explicitly set git’s config core.autocrlf to false on our Windows CI unit test environment.
Our import vars code generation checks currently rely on line endings remaining unconverted.
The Clojure story on Windows is still in the early chapters. Scoop offers an easy way to install tools. @littleli is doing a great job w/maintaining scoop apps for Clojure, Babashka and other tools and this is how I installed Babashka.
Now that the Clojure core team is recommends clj-msi life is easier.
If you are not already installing via clj-msi on Windows, you’ll want to do so.
You’ll have your own preference, but I find it convenient to install GraalVM on Windows via scoop.
You’ll need to load the appropriate Visual C++ environment variables for GraalVM’s native-image to do its work. I found it oddly cumbersome to load them from PowerShell, so I work from a cmd shell instead. Here’s what works on my Windows dev environment:
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat"After checking out this project from GitHub,
-
Install JavaScript libraries and tools required by doo and shadow-cljs:
npm ci
-
If you are on macOS or linux, install planck.
-
Initialize cache for clj-kondo so it can lint against your dependencies
bb lint
We make use of babashka tasks for development related commands.
To see all available tasks with a short description run:
bb tasks
To run a task, for example, the lint task:
bb lint
Usage help for a task is requested via --help, for example:
bb lint --help
Tasks are described throughout this document.
Rewrite-clj v0 used a version of potemkin import-vars. Potemkin import-vars copies specified vars from a specified namespace to the current namespace at load time. Due to often mysterious issues related to import-vars, a general dislike for import-vars in the Clojure community, and associated maintenance costs, we’ve opted to instead generate code for rewrite-clj v1.
For any source that used potemkin import-vars, we now have a separate template clj (or cljc) file.
For example src/rewrite_clj/zip.cljc is generated by template template/rewrite_clj/zip.cljc.
The syntax of import-vars in the template remains familiar. The following old potemkin import-vars syntax:
(import-vars
[[my.ns1 my-var1 my-var2 my-var3]
[my.ns2 my-var4 my-var5]])Is expressed in our templates as:
#_{:import-vars/import
{:from [[my.ns1 my-var1 my-var2 my-var3]
[my.ns2 my-var4 my-var5]]}}Any :added and :deprecated metadata should be defined in the template and not on the reference var.
This keeps the metadata on the public API vars only and avoids having the ClojureScript compiler warn about deprecated calls on internal sources within rewrite-clj:
#_{:import-vars/import
{:from [[my.ns1
^{:deprecated "1.2.3"} obsolete-fn
^{:added "1.2.4"} new-fn]]}}We also carry over rewrite-cljc support for :import-vars/import-with-mods, via an optional :opts.
See template/rewrite_clj/zip.cljc for example usage.
Importing will generate delegates.
An import of (defn foo [a b] (+ a b)) from namespace my.ns1 will generate (defn foo [a b] (my.ns1/foo a b)).
No generation of requires is done, your template will have to require my.ns1 in normal Clojure code.
At this time, we don’t handle destructuring in arglists, and will throw unless args are all symbols.
To generate target source from templates run:
bb apply-import-vars gen-codeYou are expected to review the generated changes and commit the generated source to version control. We don’t lint templates, but we do lint the generated code.
To perform a read-only check, run:
bb apply-import-vars checkThe check command will exit with 0 if no changes are required, otherwise it will exit with 1. Our build script will run the check command and fail the build if there are any pending changes that have not been applied.
To launch a nREPL server:
bb dev-jvm
From your IDE, cider connect clj to this REPL server.
For a nREPL server that also includes ClojureScript support:
bb dev-cljs
From your IDE, cider connect cljs to this REPL server.
Your personal preference will likely be different, but during maintenance and refactoring, I found running tests continuously for Clojure and ClojureScript helpful.
For Clojure, I open a shell terminal window and run:
bb test-clj-watch
This launches kaocha in watch mode.
For ClojureScript, I open a shell terminal window and run:
bb test-cljs-watch
This launches fighweel main. After initialization, your default web browser will automatically be opened with the figwheel auto-testing page.
All documentation is written in AsciiDoc. We follow AsciiDoc best practice of one sentence per line.
Images are created and edited with draw.io desktop. We export to .png with a border of 10 and a transparent background. At the time of this writing draw.io does not remember export settings, so you’ll have to enter them in each time.
We use test-doc-blocks to verify that code blocks in our documentation are in good working order.
bb test-doc
This generates tests for doc code blocks and then runs them under Clojure and ClojureScript.
Before pushing, you likely want to mimic what is run on each push via GitHub Actions.
Unit tests can be run locally on your dev box. Some tests require a specific os and jdk, you will see warnings if a test is skipped for your current environment.
bb ci-unit-tests
Unit tests for our ci matrix are driven by:
bb script/ci_unit_tests.clj matrix-for-ci --format=json
To get a human readable version of the ci matrix:
bb ci-unit-tests matrix-for-ci
We also verify that rewrite-clj functions as expected when compiled via Graal’s native-image.
-
Tests and library natively compiled:
bb test-native
-
Library natively compiled and tests interpreted via sci
bb test-native-sci
To try to ensure our changes to rewrite-clj do not inadvertently break existing popular libraries, we run their tests, or a portion thereof, against rewrite-clj.
bb test-libs run
See README for current libs we test against.
Additional libs are welcome.
To see a list of available libs we currently test against:
bb test-libs list
If you are troubleshooting locally, and want to only run specific tests, you can specify which ones you’d like to run. For example:
bb test-libs run cljfmt zprint
Updating the test-libs script to run against current versions of libs is recommended, but care must be taken when updating. We want to make sure we are patching correctly to use rewrite-clj v1 and running a lib’s tests as intended.
To check for outdated libs:
bb test-libs outdated
Notes:
-
The
test-libstask was developed on macOS and is run on CI under Linux only under JDK 11 only. We can expand variations at some later date if there is any value to it. -
We test the current HEAD of rewrite-clj v1 against specific versions (latest at the time of this writing) of libs.
-
We patch lib deps and sometimes code (ex.
requireforrewrite-cljcbecomesrewrite-clj). -
As folks migrate to rewrite-clj v1, the need for current patches will lessen.
-
Updating what versions we test against is currently a manual, but not an overly burdensome, task.
To see what new dependencies are available, run:
bb outdated
This task uses:
-
antq for Clojure.
-
npm for JavaScript. It only checks against installed
./node_modules, so you may want to runnpm cifirst.
To bump JavaScript npm packages:
-
make appropriate changes to
package.json -
delete the
./node_modulesdirectory and./package-lock.json -
run
npm install -
after you are happy the updates work, commit both
package.jsonandpackage-lock.json
We fail the build on any lint violations. The CI server runs:
bb lint
and you can too. The lint script will build the clj-kondo cache when it is missing or stale. If you want to force a rebuild of the cache run:
bb lint --rebuild
Integrate clj-kondo into your editor to catch mistakes as they happen.
You can optionally:
-
bb -lint-kondoto run only clj-kondo linting -
bb -lint-eastwoodto run only the eastwood linting
Before a release, it can be comforting to preview what docs will look like on cljdoc.
Limitations
-
This task should be considered experimental, I have only tested running on macOS, but am fairly confident it will work on Linux. Not sure about Windows at this time.
-
You have to push your changes to GitHub to preview them. This allows for a full preview that includes any links (source, images, etc) to GitHub. This works fine from branches and forks - in case you don’t want to affect your main development branch for a preview.
Start Local Services
To start the local cljdoc docker container:
bb cljdoc-preview start
The local cljdoc server allows your ingested docs to be viewed in your web browser.
The start command also automatically checks docker hub for any updates so that our cljdoc preview matches the current production version of cljdoc.
Ingest Docs
To ingest rewrite-clj API and docs into the local cljdoc database:
bb cljdoc-preview ingest
The ingest command automatically publishes rewrite-clj to your local maven repository (cljdoc only works with published jars).
The locally published version will include a -cljdoc-preview suffix.
I find this distinction helps to reduce confusion around locally vs remotely installed artifacts.
You’ll have to remember to git commit and git push your changes before ingesting.
Repeat these steps any time you want to preview changes.
Preview Docs
To open a view to the ingested docs in your default web browser:
bb cljdoc-preview view
If you have just run the start command, be a bit patient, the cljdoc server can take a few moments to start up - especially on macOS due to poor file sharing performance.
Stop Local Services
When you are done, you’ll want to stop your docker container:
bb cljdoc-preview stop
This will also delete temporary files created to support your preview session, most notably the local cljdoc database.
Note that NO cleanup is done for any rewrite-clj artifacts published to your local maven repository.
Container Status
If you forget where you are at with your docker containers, run:
bb cljdoc-preview status
bb test-coverage
Our CI service is setup to automatically generate then upload reports to CodeCov.
We have no specific goals for code coverage, but new code is generally expected to have tests.
So why measure coverage? It simply offers us some idea of what code our test suite hits.