Skip to content

QA Procedures

Jump to bottom
rocodes edited this page Jul 22, 2020 · 25 revisions

Release QA

Hardware installation from scratch (clean install scenario):

  1. Download and verify Ubuntu Server and perform a clean Ubuntu install (http://releases.ubuntu.com/16.04/); configure as per the standard SecureDrop install process.

  2. (If applicable) Save all instance-specific files and credentials from existing Persistent/securedrop folder on the Admin workstation (see Tips below). Then delete the repo and start over with a fresh clone.

  3. Once cloned, git fetch --tags, verify the tag for the release branch you are testing, and check out that tag.

  4. Edit install_files/ansible_base/roles/install-fpf-repo/defaults/main.yml to use:

apt_repo_url: https://apt-test.freedom.press
...
apt_repo_pubkey_files:
    - apt-test-signing-key.pub

  1. ./securedrop-admin setup, then ./securedrop-admin sdconfig, then ./securedrop-admin install

  2. Kernel Testing. On app server:

    a. install paxtest (available via apt) and run paxtest blackhat The desired results are:

    • strcpy and strcpy, PIE should be paxtest: return address contains a NULL byte
    • Randomization tests: bit guesses in the high 20s or higher (there may be one or two in the mid-20s, this is still okay).
    • Everything else: "Killed"

    b. torify curl -L https://meltdown.ovh -o meltdown.sh on the Admin workstation. Then scp meltdown.sh app:~ so that you have the script on the app server. Make meltdown.sh executable, and run the script as sudo. Note: this should never be done on production hardware--this is for QA-only hardware. You should not see any tests that come back as STATUS:VULNERABLE.

The NUC7i5BNH and NUC7i7DNHEs generate a false-positive on one particular test (Foreshadow L1TF). This has been noted by our Security Engineer. See https://github.com/freedomofpress/securedrop/issues/5040#issuecomment-559597643

Note: you will generate OSSEC alerts about grsec when you run these two tests--this is normal/expected.

  1. Follow the QA matrix and the Github release ticket for further tests (end-to-end, acceptance tests, release-specific tests, etc).

Upgrade hardware installation (cron-apt scenario)

  1. Take a backup of your existing SecureDrop installation using the securedrop-admin backup command. You will need enough free space on your Admin Workstation USB to complete this backup.

  2. Modify /etc/apt/sources.list.d/apt_freedom_press.list and /etc/apt/security.list on the Admin workstation to use the apt-test.freedom.press repo instead of the apt.freedom.press repo, ensures the apt-test public key is present, and update the apt cache.

Note: @conorsch has an ansible playbook that accomplishes steps two and three, which has been forked by various FPFers: https://gist.github.com/conorsch/e7556624df59b2a0f8b81f7c0c4f9b7d

  1. SSH to each server, and run sudo cron-apt -i -s to upgrade your instance. You may use apt-cache policy <packagename> on each server to check a given package's version.

  2. Complete steps 6 and 7 (kernel testing, QA Matrix and Github release ticket testing).

More tips

  • Save a folder in your Tails Persistent directory called something like sdconf that contains: install_files/ansible_base/group_vars/all/site-specific, your Submission Public Key, and your admin/journalist email public key, and copy those into place (the latter two just to the ansible_base directory) on a new QA run to avoid retyping everything during sdconfig.
  • AppArmor can be checked with sudo aa-status
  • Kernel can be checked with uname -r
  • The submissions table is at /var/lib/securedrop/db.sqlite. Use sudo sqlite3 /var/lib/securedrop/db.sqlite to open, select * from submissions to see submission hashes, and so on.
  • There are some utilities (such as qa_loader.py) that may help you with testing; if their use is required, they should be documented in the test plan

Pull Request QA

  1. Perform a visual review of the code changes.

  2. Check out the pull request branch on your own machine using git fetch <remotename> pull/ID/head:BRANCHNAME (If your freedomofpress/securedrop repo remote is named upstream, and you want to review PR 1337 on a local branch called newfeature, run git fetch upstream pull/1337/head:newfeature, then git checkout newfeature.)

  3. Follow the test plan as documented in the PR, and run the appropriate unit/functional/configuration/linting tests as relevant.

Clone this wiki locally