-
Notifications
You must be signed in to change notification settings - Fork 685
QA Procedures
-
Download and verify Ubuntu Server and perform a clean Ubuntu install (http://releases.ubuntu.com/16.04/); configure as per the standard SecureDrop install process.
-
(If applicable) Save all instance-specific files and credentials from existing
Persistent/securedrop
folder on the Admin workstation (see Tips below). Then delete the repo and start over with a fresh clone. -
Once cloned,
git fetch --tags
, verify the tag for the release branch you are testing, and check out that tag. -
Edit
install_files/ansible_base/roles/install-fpf-repo/defaults/main.yml
to use:
apt_repo_url: https://apt-test.freedom.press
...
apt_repo_pubkey_files:
- apt-test-signing-key.pub
-
./securedrop-admin setup,
then./securedrop-admin sdconfig
, then./securedrop-admin install
-
Kernel Testing. On app server:
a. install paxtest (available via apt) and run
paxtest blackhat
The desired results are:-
strcpy
andstrcpy, PIE
should bepaxtest: return address contains a NULL byte
- Randomization tests: bit guesses in the high 20s or higher (there may be one or two in the mid-20s, this is still okay).
- Everything else: "Killed"
b.
torify curl -L https://meltdown.ovh -o meltdown.sh
on the Admin workstation. Thenscp meltdown.sh app:~
so that you have the script on the app server. Makemeltdown.sh
executable, and run the script as sudo. Note: this should never be done on production hardware--this is for QA-only hardware. You should not see any tests that come back asSTATUS:VULNERABLE
. -
The NUC7i5BNH and NUC7i7DNHEs generate a false-positive on one particular test (Foreshadow L1TF). This has been noted by our Security Engineer. See https://github.com/freedomofpress/securedrop/issues/5040#issuecomment-559597643
Note: you will generate OSSEC alerts about grsec when you run these two tests--this is normal/expected.
- Follow the QA matrix and the Github release ticket for further tests (end-to-end, acceptance tests, release-specific tests, etc).
-
Take a backup of your existing SecureDrop installation using the
securedrop-admin backup
command. You will need enough free space on your Admin Workstation USB to complete this backup. -
Modify
/etc/apt/sources.list.d/apt_freedom_press.list
and/etc/apt/security.list
on the Admin workstation to use theapt-test.freedom.press
repo instead of theapt.freedom.press
repo, ensures theapt-test
public key is present, and update the apt cache.
Note: @conorsch has an ansible playbook that accomplishes steps two and three, which has been forked by various FPFers: https://gist.github.com/conorsch/e7556624df59b2a0f8b81f7c0c4f9b7d
-
SSH to each server, and run
sudo cron-apt -i -s
to upgrade your instance. You may useapt-cache policy <packagename>
on each server to check a given package's version. -
Complete steps 6 and 7 (kernel testing, QA Matrix and Github release ticket testing).
- Save a folder in your Tails
Persistent
directory called something likesdconf
that contains:install_files/ansible_base/group_vars/all/site-specific
, your Submission Public Key, and your admin/journalist email public key, and copy those into place (the latter two just to theansible_base
directory) on a new QA run to avoid retyping everything duringsdconfig
. - AppArmor can be checked with
sudo aa-status
- Kernel can be checked with
uname -r
- The submissions table is at
/var/lib/securedrop/db.sqlite
. Usesudo sqlite3 /var/lib/securedrop/db.sqlite
to open,select * from submissions
to see submission hashes, and so on. - There are some utilities (such as
qa_loader.py
) that may help you with testing; if their use is required, they should be documented in the test plan
-
Perform a visual review of the code changes.
-
Check out the pull request branch on your own machine using
git fetch <remotename> pull/ID/head:BRANCHNAME
(If yourfreedomofpress/securedrop
repo remote is namedupstream
, and you want to review PR 1337 on a local branch callednewfeature
, rungit fetch upstream pull/1337/head:newfeature
, thengit checkout newfeature
.) -
Follow the test plan as documented in the PR, and run the appropriate unit/functional/configuration/linting tests as relevant.