-
Notifications
You must be signed in to change notification settings - Fork 43
0.5.0 Test Plan
This test plan covers release 0.5.0 of the SecureDrop Workstation, which will ship consolidated templates via the preflight updater (#471), as well as other unreleased changes from main
.
In addition to an RPM update, we will simultaneously issue new releases of most SecureDrop Workstation components, including the SecureDrop Client. These have their own versioning, but for simplicity, this issue uses "0.5.0" as a shorthand for the cross-component release.
Before beginning to test, please claim a column in the 0.5.0 test matrix and fill out the first 3 fields.
To set up the environment for testing correctly, you must work through the Template Consolidation scenario first, choosing one of the 3 install types:
- Fresh install (self explanatory)
- Happy Path - assumes the user follows the upgrade instructions for 0.5.0
- Sad Path - assumes the user does not follow the upgrade instructions for 0.5.0
If time allows, standard acceptance tests should also be completed for each release candidate.
Then complete the rest of the test scenarios (or as many as possible).
Once finished, please append your completed test plan to the release issue and update the test matrix with your results for each tested scenario.
- Hardware target: e.g. Thinkpad X1 20KH002JUS
- Install type: fresh install/"happy path" upgrade/"sad path" upgrade
- securedrop-workstation-dom0-config version: Check via
dnf list
in dom0 - Release candidate version: 0.5.0-rcN
To prepare the test environment, complete the Server setup section below. Then complete the section for your choice of QA scenarios.
- Ensure that your server is populated with at least 10 sources and with some file submissions.
- Create a user via the Journalist Interface and record their login credentials, referenced as account X. Log in as account X via the Journalist Interface, and reply to some sources.
-
If an existing production install is being used for testing, remove it completely with the following commands in
dom0
:securedrop-admin --uninstall qvm-kill securedrop-workstation-buster sudo dnf remove qubes-template-securedrop-workstation-buster sudo dnf clean all
-
Follow the installation instructions to copy the submission key and journalist interface config details into place, then follow the steps below to download and install the workstation RPM.
-
in a networked VM (
work
, say):- download the latest
securedrop-workstation-dom0-config
version (sdw-latest.rpm
, say) fromhttps://yum-test.securedrop.org/workstation/dom0/f25/
- clone the
https://github.com/freedomofpress/securedrop-workstation.git
repo and ensure that it is on themain
branch
- download the latest
-
in
dom0
:- transfer the RPM to
dom0
usingqvm-run --pass-io work "cat sdw-latest.rpm" > sdw-latest.rpm
- transfer the repo to
dom0
usingqvm-run --pass-io work "tar -c -C /home/user securedrop-workstation" | tar xvf -
- install the RPM using
sudo dnf install sdw-latest.rpm
- update the Salt config to use QA repos using
sudo bash ~/securedrop-workstation/utils/qa-switch.sh
- set up the workstation using the command
sdw-admin --apply
- transfer the RPM to
-
Verify that the installation completes successfully.
-
In
dom0
, runqvm-ls --tags sd-workstation
, and verify that:- the only TemplateVMs listed are
sd-(large|small)-buster-template)
andsecuredrop-workstation-buster
. -
sd-app
,sd-log
,sd-proxy
, andsd-gpg
are based onsd-small-buster-template
-
sd-devices-dvm
andsd-viewer
are based onsd-large-buster-template
- the only TemplateVMs listed are
- Complete a production workstation setup following the installation instructions.
- Start the SecureDrop Client and Verify that its version in the login dialog footer is
0.2.1
- As the primary test journalist user, submit a reply to a source via the client.
- in a networked VM (
work
, say):- clone the
https://github.com/freedomofpress/securedrop-workstation.git
repo and ensure that it is on themain
branch
- clone the
- in
dom0
:- transfer the repo to
dom0
usingqvm-run --pass-io work "tar -c -C /home/user securedrop-workstation" | tar xvf -
- update the Salt config to use QA repos using
sudo bash ~/securedrop-workstation/utils/qa-switch.sh
- update the workstation config RPM via
sudo qubes-dom0-update -y
and verify that it is now the release candidate withdnf list securedrop-workstation-dom0-config
- verify that the pre-consolidation templates are still in use via
qvm-ls --tags sd-workstation
- update the Salt config again, using
sudo bash ~/securedrop-workstation/utils/qa-switch.sh
- remove the updater flag files, if they exist, with
rm ~/.securedrop_launcher/sdw-{last_updated,update-status}
- transfer the repo to
- Double-click the SecureDrop icon to start the updater, and run
tail -f ~/securedrop_launcher/logs/launcher.log
indom0
- Verify that the installation completes successfully.
- In
dom0
, run `qvm-ls --tags sd-workstation, and verify that:- the only TemplateVMs listed for AppVMs are
sd-(large|small)-buster-template)
andsecuredrop-workstation-buster
. -
sd-app
,sd-log
,sd-proxy
, andsd-gpg
are based onsd-small-buster-template
-
sd-devices
andsd_viewer
are based onsd-large-buster-template
- the only TemplateVMs listed for AppVMs are
- Perform a timed run of the updater via the
dom0
command:time /opt/securedrop/launcher/sdw-launcher.py --skip-delta=0
and record the time here: e.g. 12 minutes 32 seconds
- Complete a production workstation setup following the installation instructions.
- Start the SecureDrop Client and Verify that its version in the login dialog footer is
0.2.1
- As the primary test journalist user, submit a reply to a source via the client.
- in a networked VM (
work
, say):- clone the
https://github.com/freedomofpress/securedrop-workstation.git
repo and ensure that it is on themain
branch
- clone the
- in
dom0
:- transfer the repo to
dom0
usingqvm-run --pass-io work "tar -c -C /home/user securedrop-workstation" | tar xvf -
- update the Salt config to use QA repos using
sudo bash ~/securedrop-workstation/utils/qa-switch.sh
- remove the updater flag files, if they exist, with
rm ~/.securedrop_launcher/sdw-{last_updated,update-status}
- transfer the repo to
- Reboot the machine. When the GUI updater launches, leave it open without clicking Continue
- in
dom0
:- verify the config RPM version is
0.4.0.1-fc25
, withdnf list securedrop-workstation-dom0-config
- verify that the pre-consolidation templates are still in use via
qvm-ls --tags sd-workstation
- verify the config RPM version is
- In the updater, click Continue to start the update, and run
tail -f ~/securedrop_launcher/logs/launcher.log
indom0
- Verify that the updater completes without error
- Verify that the client fails to start when the Securedrop icon is double-clicked.
- in
dom0
:- verify the config RPM version is now
0.5.0.*-fc25
, withdnf list securedrop-workstation-dom0-config
- update the Salt config to use QA repos using
sudo bash ~/securedrop-workstation/utils/qa-switch.sh
- run
sdw-admin --apply
to apply the correct template configuration.
- verify the config RPM version is now
- Verify that the installation completes successfully.
- Verify that the client starts when the SecureDrop desktop icon is double-clicked, and its version is
0.2.1-dev-20201029-164632
- In
dom0
, runqvm-ls --tags sd-workstation
, and verify that:- the only TemplateVMs listed are
sd-(large|small)-buster-template)
andsecuredrop-workstation-buster
. -
sd-app
,sd-log
,sd-proxy
, andsd-gpg
are based onsd-small-buster-template
-
sd-devices
andsd-viewer
are based onsd-large-buster-template
- the only TemplateVMs listed are
- Verify that client login and basic functionality are working.
- Perform a timed run of the updater via the
dom0
command:time /opt/securedrop/launcher/sdw-launcher.py --skip-delta=0
and record the time here: e.g. 12 minutes 32 seconds
- on
sd-log
:-
archive the logging directory with:
tar -zcvf log_archive.tar.gz ~/QubesIncomingLogs rm -rf ~/QubesIncomingLogs/*
-
- on
dom0
:- restart
sd_log
viaqvm-kill sd-log && qvm-start sd-log
- restart
- Verify that the standard acceptance tests pass.
- Observe that
securedrop-admin
can no longer be run indom0
- Observe that you can now access the tool with
sdw-admin
indom0
- Observe that
sdw-admin --validate
still functions as expected
- Under "System ▶ Appearance ▶ Fonts" in
dom0
, note the current font size of the "Default Font", then increase it to a very high value, e.g., 17. - Run the GUI updater, via
/opt/securedrop/launcher/sdw-launcher.py --skip-delta 0
. Complete the updater process while observing the following: - Observe that no text in the updater GUI is cut off
- Observe that you cannot cause text to be cut off by resizing the window
- After the update completes, restore the font size to the previous value.
Test case: Preview snippets (Issue: https://github.com/freedomofpress/securedrop-client/issues/1121; PR: https://github.com/freedomofpress/securedrop-client/pull/1131)
- Log into the SecureDrop Client
- Before the client has finished decrypting all messages and replies, send a reply to an existing user
- Observe that a preview of that reply is shown underneath the source name in the source list to the left, once it has been successfully sent
- Wait for all messages/replies to be decrypted
- Observe that during the sync, the preview snippet continues to show the reply you sent
Test case: Window resizing (Issues: https://github.com/freedomofpress/securedrop-client/issues/1109 and https://github.com/freedomofpress/securedrop-client/issues/1120; PRs: https://github.com/freedomofpress/securedrop-client/pull/1130 and https://github.com/freedomofpress/securedrop-client/pull/1145)
- Observe that the client window is maximized with no part of the client hidden from view
- Click on the square resize icon in the upper right-hand corner
- Observe that the client window is changed to a smaller size, with no part of the window hidden from view (horizontal scrolling within the conversation view may be required)
- Resize the client window manually by dragging its corner
- Observe for multiple sources that during resizing, source abbreviations are elided at lower resolutions as shown in this GIF
- Click a few times on the square resize icon in the upper right corner
- Observe that the client switches back and forth between maximum size and your chosen size, with no part of the window hidden from view
- If a second monitor is available, move the client window to a second monitor and repeat the previous step
- Observe that the client switches back and forth between maximum size and your chosen size, with no part of the window hidden from view
Test case: Reply badges feature (Issue: https://github.com/freedomofpress/securedrop-client/issues/76; PR: https://github.com/freedomofpress/securedrop-client/pull/1142)
- Log into the SecureDrop Client and wait for it to sync.
- Observe that all previously sent replies show a two-letter badge representing the reply author. If first name and last name are set, lower-case initials will be used; otherwise, the first two letters of the username will be used.
- Send a reply.
- Observe that the sent reply shows a two-letter badge representing your account.
- Observe that the badge for replies you have sent is in a different color than the badge for replies sent by other users.
- Through the Journalist Interface, delete account X. (Stay logged into the JI for a later step.)
- Wait for the SecureDrop Client to synchronize with the server.
- Observe that replies sent by account X are now attributed to a placeholder user, represented by a star cluster icon.
- Through the Journalist Interface, change the first and last name of the user logged into the SecureDrop Client to names with different initial.
- Wait for the SecureDrop Client to synchronize with the server.
- Observe that the updated initials are shown in reply badges, and in the login badge (top left corner).
- Switch to offline mode
- Observe that all reply badges are still present, but are now in a uniform color, even if sent by the previously logged in user.
Test case: Decryption error styling (Issue: https://github.com/freedomofpress/securedrop-client/issues/1150; PR: https://github.com/freedomofpress/securedrop-client/pull/1151)
- Log out of the SecureDrop Client
- Via the Source Interface, send a message as a new source
- Via the Journalist Interface, send a reply to that source
- In
sd-gpg
, temporarily move~/.gnupg
to~/.gnupg.bak
to cause decryption errors - Log into the SecureDrop Client and wait for it to sync
- Observe that message and reply are styled identically, in gray and italics: "cannot decrypt message" / "cannot decrypt reply"
- In
sd-gpg
, move~/.gnupg.bak
back to~/.gnupg
Test case: Sorting logs by hostname (Issue: #583; PR: https://github.com/freedomofpress/securedrop-log/pull/18)
- Observe that no
host
directory exists in~/QubesIncomingLogs
onsd-log
(only relevant for "Fresh install" scenario, otherwise the directory will be present) - Run the command
logger LOGME
insd-app
,sd-whonix
, andsd-log
. - Observe that the line was added to
~/QubesIncomingLogs/sd-app
and~/QubesIncomingLogs/sd-whonix
- Observe that no
sd-log
directory exists in~/QubesIncomingLogs
onsd-log
.