Automatic testing

We run automatic tests in various places. This page documents what we do where.

Buildbot

Buildbot only runs after pushes (which includes merges). It runs our pdns-builder based builds, including make check, for all platforms that we ship packages for. Additionally, it runs recursor tests and bulk tests on some recent Debian.

Travis

Travis has been removed from the master branch and will not appear on new release branches either.

For older releases, Travis tests 'almost everything'. Some small features are missing, and not all products run with all sanitizers.

GH Actions

We run several workflows in GitHub Actions:

• builder runs our pdns-builder based builds (including make check) for a few older distros, to catch errors there before merge (as most people develop on recent systems). This avoids surprises in Buildbot.
• codeql-analysis runs static analysis on every commit, to catch certain classes of errors.
• fuzz runs a few minutes of fuzz testing on specific components.
• spelling spell checks our docs.

GitHub Actions run on every commit, including PRs.

Circle

Our CircleCI setup builds all products (including ixfrdist) with UBSan and ASan, and runs all tests. There are no exclusions for any nosetests-based runs, and all auth backends are tested.

On non-master branches before January 2021 (so auth-4.4, rec-4.4, dnsdist-1.5, and older), CircleCI testing is somewhat less complete.

CircleCI runs on every commit, including PRs.