-
Notifications
You must be signed in to change notification settings - Fork 685
QA Procedures
You can check for unwrapped strings (e.g., "foo"
instead of gettext("foo")
) by enabling and switching to the Esperanto "pseudolocale" (sorry, Esperantists):
--- a/securedrop/i18n.json
+++ b/securedrop/i18n.json
@@ -1,5 +1,9 @@
{
"supported_locales": {
+ "eo": {
+ "name": "Esperanto",
+ "deoktop": "eo"
+ },
"ca": {
"name": "Catalan",
"desktop": "ca"
Any foo
that doesn't show up like [!foo!]
is unwrapped.
-
(Optional): Take a backup of your current instance.
-
(Optional, if reusing existing Admin workstation): Copy the following config and pubkey files from your existing Admin Workstation setup to a new folder in
~/Persistent
(eg~/Persistent/sdfiles
):
-
~/Persistent/securedrop/install_files/ansible-base/SecureDrop.asc
(or your instance's Submission Public Key) -
~/Persistent/securedrop/install_files/ansible-base/journalist.asc
(or your instance's journalist pubkey) -
~/Persistent/securedrop/install_files/ansible-base/alerts.asc
(or your instance's OSSEC alerts pubkey) ~/Persistent/securedrop/install_files/ansible-base/group_vars/all/site-specific
Then
- Delete the old
securedrop
repo from this Admin USB - Delete
~/.ssh/known_hosts
and.ssh/config
(assuming this Admin USB's SSH key is not being used for anything else)
-
Download and verify Ubuntu Server and perform a clean Ubuntu install (http://releases.ubuntu.com/20.04/ubuntu-20.04.3-live-server-amd64.iso) on hardware; configure Ubuntu Server as per the standard SecureDrop install process, including copying over SSH keys.
cd ~/Persistent
git clone https://github.com/freedomofpress/securedrop
Once cloned, git fetch --tags
, verify the tag for the release branch you are testing, and check out that tag.
- Manually edit
install_files/ansible-base/roles/install-fpf-repo/defaults/main.yml
to use:
--- a/install_files/ansible-base/roles/install-fpf-repo/defaults/main.yml
+++ b/install_files/ansible-base/roles/install-fpf-repo/defaults/main.yml
@@ -10,7 +10,7 @@
#
# For testing/QA, set this URL to another apt server. You must also update
# the associated public key for the apt repo for testing/QA.
-apt_repo_url: https://apt.freedom.press
+apt_repo_url: https://apt-test.freedom.press
# By default, install packages from the apt-repo, but under
# staging hosts we'll prefer locally-built deb packages
@@ -20,7 +20,7 @@ install_local_packages: False
# May be overridden in staging to install from a test/QA server,
# the Release file for which will *not* be signed with the prod key.
apt_repo_pubkey_files:
- - fpf-signing-key-2021.pub
+ - apt-test-signing-key.pub
# As of v2.0.0, only Focal is supported.
apt_repo_target_distro: "{{ ansible_distribution_release }}"
-
Copy the apt-test signing key to each server manually:
USER=sdadmin # or "vagrant", etc. IP=A.B.C.D # per server cat install_files/ansible-base/roles/install-fpf-repo/files/apt-test-signing-key.pub | ssh ${USER}@${IP} sudo apt-key add
-
cd ~/Persistent/securedrop/install_files/ansible-base/
and copy in the required .asc files, and optionally, copy thesite-specific
file into./group_vars/all/
(this will pre-populate your previous configuration during setup, and save you some time and typing).To configure and install SecureDrop, run:
cd ~/Persistent/securedrop ./securedrop-admin setup ./securedrop-admin sdconfig ./securedrop-admin install ./securedrop-admin tailsconfig
During
install
, ifapt-get update
fails, you'll need to do something like this against each server to injectapt-test
's signing key manually:cat install_files/ansible-base/roles/install-fpf-repo/files/apt-test-signing-key.pub | ssh [email protected] sudo apt-key add
-
Kernel Testing. On app server:
a. install paxtest (available via apt) and run
paxtest blackhat
The desired results are:-
strcpy
andstrcpy, PIE
will bepaxtest: return address contains a NULL byte
- Randomization tests: bit guesses in the high 20s or higher (there may be one or two in the mid-20s, this is still okay).
-
memcpy
andmemcpy, PIE
will show Vulnerable: this is expected on Ubuntu 20.04 - Everything else: "Killed"
b.
curl -L https://meltdown.ovh -o meltdown.sh
to make sure you have the latest meltdown script. Then runsudo bash meltdown.sh -v
so that you have the script on the app server. Note: this should never be done on production hardware--this is for QA-only hardware. You should not see any tests that come back asSTATUS:VULNERABLE
.ATTENTION: The NUC7i5BNH, NUC7i7DNHEs, NUC8s and NUC10s and generate a false-positive on one particular test (Foreshadow L1TF). This has been noted by our Security Engineer. See https://github.com/freedomofpress/securedrop/issues/5040#issuecomment-559597643.
Note: you will generate OSSEC alerts about grsec when you run these two tests--this is normal/expected.
-
-
Follow the QA matrix and the Github release ticket for further tests (end-to-end, acceptance tests, release-specific tests, etc).
-
Take a backup of your existing SecureDrop installation using the
securedrop-admin backup
command. You will need enough free space on your Admin Workstation USB to complete this backup. -
Either run the ansible QA playbook:
cd ~/Persistent/securedrop
source admin/.venv3/bin/activate
cd install_files/ansible-base
ansible-playbook -vv --diff securedrop-qa.yml
OR
Connect to each of the application and monitor servers via SSH and follow these steps:
- Modify
/etc/apt/sources.list.d/apt_freedom_press.list
and/etc/apt/security.list
to use theapt-test.freedom.press
repo instead of theapt.freedom.press
repo, ensure theapt-test
public key is present, and update the apt cache. - Run
sudo unattended-upgrades -d
to upgrade your instance. Your SSH session may freeze or drop, but you should be able to reconnect thanks totmux
. You may useapt-cache policy <packagename>
to check a given package's version.
- Complete steps 6 and 7 of clean install QA instructions (kernel testing, QA Matrix and Github release ticket testing).
-
Perform a visual review of the code changes.
-
Check out the pull request branch on your own machine using
git fetch <remotename> pull/ID/head:BRANCHNAME
(If yourfreedomofpress/securedrop
repo remote is namedupstream
, and you want to review PR 1337 on a local branch callednewfeature
, rungit fetch upstream pull/1337/head:newfeature
, thengit checkout newfeature
.) -
Follow the test plan as documented in the PR, and run the appropriate unit/functional/configuration/linting tests as relevant.
- Save a folder in your Tails
Persistent
directory called something likesdconf
that contains:install_files/ansible_base/group_vars/all/site-specific
, your Submission Public Key, and your admin/journalist email public key, and copy those into place (the latter two just to theansible_base
directory) on a new QA run to avoid retyping everything duringsdconfig
. - AppArmor can be checked with
sudo aa-status
- Kernel can be checked with
uname -r
- The submissions table is at
/var/lib/securedrop/db.sqlite
. Usesudo sqlite3 /var/lib/securedrop/db.sqlite
to open,select * from submissions
to see submission hashes, and so on. - There are some utilities (such as
qa_loader.py
) that may help you with testing; if their use is required, they should be documented in the test plan
The goal of preflight testing is ensure that the release has been properly deployed, and is ready for final deployment to production instances. For preflight testing, we host the new packages at https://apt-qa.freedom.press/, rather than https://apt.freedom.press/. The exact same packages, Release file, and corresponding detached signature, generated with the prod key, are used. We deploy those artifacts to apt-qa to provide final verification that the release will function well for real-world prod instances.
With more automation around the release process, we should be able to do away with the need to perform preflight testing. For now, it's done in support of the RM as one of the final steps in preparing a release.
On release day, prepare either hardware or prod VMs suitable for installation. VMs are usually preferable, since any hardware-specific changes will have been tested thoroughly as part of normal QA. Choose whether you'll be performing a clean install, or testing an upgrade.
For upgrade, it's important to perform these steps ahead of time, because it can take ~1h to install SD before it's ready for preflight. Here are the steps:
# on dev workstation:
molecule create -s libvirt-prod-focal
# then log into Tails Admin VM, and run:
./securedrop-admin install
./securedrop-admin tailsconfig
# then "ssh app" and make an account, log in, make sure it works
Now you're prepped. To test, log into both servers, and edit /etc/apt/sources.list.d/apt_freedom_press.list
, replacing apt.freedom.press
with apt-qa.freedom.press
. No other changes are required.
Once the release is available on apt-qa, run sudo unattended-upgrades -d
to upgrade to the new release. Perform functional testing, and report any problems.
For clean install preflight testing, the procedure is similar to the above:
# on dev workstation:
molecule create -s libvirt-prod-focal
# then log into Tails Admin VM
# edit ~/Persistent/securedrop/install_files/ansible-base/roles/install-fpf-repo/defaults/main.yml
# and change "apt.freedom.press" to "apt-qa.freedom.press".
# wait for the go-ahead from the RM, then proceed with install:
./securedrop-admin install
./securedrop-admin tailsconfig
# then "ssh app" and make an account, log in, make sure it works