Skip to content

QA Procedures

Jump to bottom
Allie Crevier edited this page Jan 22, 2021 · 25 revisions

Release QA

Hardware installation from scratch (clean install scenario):

  1. Download and verify Ubuntu Server and perform a clean Ubuntu install (http://releases.ubuntu.com/16.04/); configure as per the standard SecureDrop install process.

  2. (If applicable) Save all instance-specific files and credentials from existing Persistent/securedrop folder on the Admin workstation (see Tips below). Then delete the repo and start over with a fresh clone.

  3. Once cloned, git fetch --tags, verify the tag for the release branch you are testing, and check out that tag.

  4. Edit install_files/ansible_base/roles/install-fpf-repo/defaults/main.yml to use:

apt_repo_url: https://apt-test.freedom.press
...
apt_repo_pubkey_files:
    - apt-test-signing-key.pub
  1. To configure and install SecureDrop, run the following:
cd ~/Persistent/securedrop
./securedrop-admin setup
./securedrop-admin sdconfig
./securedrop-admin install
./securedrop-admin tailsconfig

  1. Kernel Testing. On app server:

    a. install paxtest (available via apt) and run paxtest blackhat The desired results are:

    • strcpy and strcpy, PIE should be paxtest: return address contains a NULL byte
    • Randomization tests: bit guesses in the high 20s or higher (there may be one or two in the mid-20s, this is still okay).
    • Everything else: "Killed"

    b. curl -L https://meltdown.ovh -o meltdown.sh to make sure you have the latest meltdown script. Then run sudo bash meltdown.sh -v so that you have the script on the app server. Note: this should never be done on production hardware--this is for QA-only hardware. You should not see any tests that come back as STATUS:VULNERABLE.

    ATTENTION: The NUC7i5BNH and NUC7i7DNHEs generate a false-positive on one particular test (Foreshadow L1TF). This has been noted by our Security Engineer. See https://github.com/freedomofpress/securedrop/issues/5040#issuecomment-559597643.

    Note: you will generate OSSEC alerts about grsec when you run these two tests--this is normal/expected.

  2. Follow the QA matrix and the Github release ticket for further tests (end-to-end, acceptance tests, release-specific tests, etc).

Upgrade hardware installation (cron-apt scenario)

  1. Take a backup of your existing SecureDrop installation using the securedrop-admin backup command. You will need enough free space on your Admin Workstation USB to complete this backup.

  2. Connect to each of the application and monitor servers via SSH and follow these steps:

    1. Modify /etc/apt/sources.list.d/apt_freedom_press.list and /etc/apt/security.list to use the apt-test.freedom.press repo instead of the apt.freedom.press repo, ensure the apt-test public key is present, and update the apt cache.

      Note: @conorsch has an ansible playbook that accomplishes steps two and three, which has been forked by various FPFers:
      https://gist.github.com/conorsch/e7556624df59b2a0f8b81f7c0c4f9b7d

    2. Run sudo cron-apt -i -s to upgrade your instance. Your SSH session may freeze or drop, but you should be able to reconnect thanks to tmux. You may use apt-cache policy <packagename> to check a given package's version.

  3. Complete steps 6 and 7 (kernel testing, QA Matrix and Github release ticket testing).

More tips

  • Save a folder in your Tails Persistent directory called something like sdconf that contains: install_files/ansible_base/group_vars/all/site-specific, your Submission Public Key, and your admin/journalist email public key, and copy those into place (the latter two just to the ansible_base directory) on a new QA run to avoid retyping everything during sdconfig.
  • AppArmor can be checked with sudo aa-status
  • Kernel can be checked with uname -r
  • The submissions table is at /var/lib/securedrop/db.sqlite. Use sudo sqlite3 /var/lib/securedrop/db.sqlite to open, select * from submissions to see submission hashes, and so on.
  • There are some utilities (such as qa_loader.py) that may help you with testing; if their use is required, they should be documented in the test plan

Pull Request QA

  1. Perform a visual review of the code changes.

  2. Check out the pull request branch on your own machine using git fetch <remotename> pull/ID/head:BRANCHNAME (If your freedomofpress/securedrop repo remote is named upstream, and you want to review PR 1337 on a local branch called newfeature, run git fetch upstream pull/1337/head:newfeature, then git checkout newfeature.)

  3. Follow the test plan as documented in the PR, and run the appropriate unit/functional/configuration/linting tests as relevant.

Clone this wiki locally