Release SecureDrop 2.11.0

[zenmonkeykstop]

This is a tracking issue for the release of SecureDrop 2.11.0

Tentatively scheduled as follows:

Pre-release announcement: 2024-12-10
Release date: 2024-12-17 (tentative)

Release manager: @zenmonkeykstop
Deputy release manager: @legoktm
Localization manager: @cfm
Communications manager: @nathandyer

SecureDrop maintainers and testers: As you QA 2.11.0, please report back your testing results as comments on this ticket. File GitHub issues for any problems found, tag them "QA: Release".

Test debian packages will be posted on https://apt-test.freedom.press signed with the test key.

QA Matrix for 2.11.0

Test Plan for 2.11.0

Prepare release candidate (2.11.0~rc1)

• Link to latest version of Tails, including release candidates, to test against during QA
• Prepare 2.11.0~rc1 release changelog
• Branch off release/2.11.0 from develop
• Prepare 2.11.0-rc1
• Build debs, preserving build log, and put up 2.11.0~rc1 on test apt server
• Commit build log.

Prepare release candidate (2.11.0~rc2)

• Link to latest version of Tails, including release candidates, to test against during QA
• Prepare 2.11.0~rc2 release changelog
• Prepare 2.11.0-rc2
• Build debs, preserving build log, and put up 2.11.0~rc2 on test apt server
• Commit build log.

After each test, please update the QA matrix and post details for Basic Server Testing, Application Acceptance Testing and release-specific testing below in comments to this ticket.

Final release

• Ensure builder in release branch is updated and/or update builder image (no longer in use)
• Push signed tag
• Pre-Flight: Test updater logic in Tails (apt-qa tracks the release branch in the LFS repo)
• Build final Debian packages(and preserve build log)
• Commit package build log to https://github.com/freedomofpress/build-logs
• Pre-Flight: Test that install and upgrade from 2.10.1 to 2.11.0 works w/ prod repo debs (apt-qa.freedom.press polls the release branch in the LFS repo for the debs)
• Flip apt QA server to prod status (merge to main in the LFS repo)
• Merge Docs branch changes to main and verify new docs build in securedrop-docs repo
• Prepare release messaging

Post release

• Create GitHub release object
• Once release object is created, update versions in securedrop-docs and Wagtail
• Verify new docs show up on https://docs.securedrop.org
• Publish announcements
• Merge changelog back to develop
• Update roadmap wiki page: https://github.com/freedomofpress/securedrop/wiki/Development-Roadmap

[zenmonkeykstop]

2.11.0 QA - IN PROGRESS

Environment

• Install target: Prod VMs
• Tails version: 6.10
• Test Scenario: upgrade
• SSH over Tor?: no
• Tor PoW enabled?: yes
• Release candidate: rc1 rc2
• General notes:

Basic Server Testing

• After installing the testinfra dependencies, all tests in ./securedrop-admin verify are passing:
  • Install dependencies on Admin Workstation with cd ~/Persistent/securedrop && ./securedrop-admin setup -t
  • Run tests with ./securedrop-admin verify (this will take a while)
  • Remove test dependencies: rm -rf admin/.venv3/ && ./securedrop-admin setup
• QA Matrix checks pass

Command Line User Generation

• Can successfully add admin user and login

(Optional) Administration

• I have backed up and successfully restored the app server following the backup documentation
• If doing upgrade testing, make a backup on 2.10.1 and restore this backup on this release candidate, verifying that the Redis password in /etc/redis/redis.conf changes
• "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
• Can successfully add journalist account with HOTP authentication

(Optional) Application Acceptance Testing

Source Interface

Landing page base cases

• JS warning bar does not appear when using Security Slider high
• JS warning bar does appear when using Security Slider Low

First submission base cases

• On generate page, refreshing page produces a new 7-word codename
• On submit page, empty submissions produce flashed message
• On submit page, short message submitted successfully
• On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
• On submit page, file less than 500 MB submitted successfully

Returning source base cases

• Nonexistent codename cannot log in
• Empty codename cannot log in
• Legitimate codename can log in
• Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

• Can log in with 2FA tokens
• incorrect password cannot log in
• invalid 2fa token cannot log in
• 2fa immediate reuse cannot log in
• Journalist account with HOTP can log in

Index base cases

• Filter by codename works
• Starring and unstarring works
• Click select all selects all submissions
• Selecting all and clicking "Download" works

Individual source page

• You can submit a reply and a flashed message and new row appears
• You cannot submit an empty reply
• Clicking "Delete Source Account" and the source and docs are deleted
• You can click on a document and successfully decrypt using application private key

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

• The Updater GUI appears on boot

2.11.0 release-specific changes

Noble prep (misc PRs)

• Verify that ufw and haveged are not installed on either server
• Verify that sdssh user group exists containing SD admin acccount, and that the ssh group is empty, on both servers

Check script and banner (#7334, #7363, 7348)

• After installation and reboot, verify that a file /etc/securedrop-noble-migration.json has been created on the application server
• Install the ufw package on the application server and run sudo /usr/bin/securedrop-noble-migration-check - verify that the ufw value in the json file is now False
• Uninstall the ufw package and run sudo /usr/bin/securedrop-noble-migration-check again - verify that the ufw value in the json file is now False
• Edit the json file and ensure at least one value is false
• Log into the JI and verify that a banner is displayed with the Noble readiness warning message
• Edit the json file again, flipping all values to True
• Log into the JI and verify that the banner is not displayed (Note: a sudo systemctl restart apache2 is required here coz lru_cache)
• In /var/www/securedrop/server_os.py, set FOCAL_ENDOFLIFE to a past date.
• Log into the JI and verify that a banner is displayed with the Focal EOL warning message
• In /var/www/securedrop/server_os.py, reset FOCAL_ENDOFLIFE to its original value

Preflight testing

Basic testing

• Install or upgrade occurs without error (from apt-qa.freedom.press per preflight procedure)
• Source interface is available and version string indicates it is 2.11.0
• A message can be successfully submitted

Tails

• The updater GUI appears on boot
• The update successfully occurs to 2.11.0
• After reboot, updater GUI no longer appears

[nathandyer]

2.11.0 QA - IN PROGRESS

Environment

• Install target: NUC13/NUC12
• Tails version: 6.9
• Test Scenario: upgrade
• SSH over Tor?: no
• Tor PoW enabled?: yes
• Release candidate: rc1
• General notes:

Basic Server Testing

• After installing the testinfra dependencies, all tests in ./securedrop-admin verify are passing:
  • Install dependencies on Admin Workstation with cd ~/Persistent/securedrop && ./securedrop-admin setup -t
  • Run tests with ./securedrop-admin verify (this will take a while)
  • Remove test dependencies: rm -rf admin/.venv3/ && ./securedrop-admin setup

Note: originally failed, requiring an apparmor patch and removing a bad temp file in /etc/apt/apt.conf.d

Full original output:

=================================== FAILURES ===================================
_____________ test_unattended_upgrades_functional[paramiko://mon] ______________
[gw0] linux -- Python 3.11.2 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python

host = <testinfra.host.Host paramiko://mon>

    def test_unattended_upgrades_functional(host):
        """
        Ensure unattended-upgrades completes successfully and ensures all packages
        are up-to-date.
        """
        c = host.run("sudo unattended-upgrades --dry-run --debug")
        assert c.rc == 0
        distro = host.system_info.codename
        expected_origins = (
            f"Allowed origins are:"
            f" origin=Ubuntu,archive={distro}, origin=Ubuntu,archive={distro}-security"
            f", origin=Ubuntu,archive={distro}-updates, origin=SecureDrop,codename={distro}"
        )
        expected_result = (
            "No packages found that can be upgraded unattended and no pending auto-removals"
        )
    
>       assert expected_origins in c.stdout
E       assert 'Allowed origins are: origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-security, origin=Ubuntu,archive=focal-updates, origin=SecureDrop,codename=focal' in "Starting unattended upgrades script\nAllowed origins are: o=Ubuntu,a=focal, o=Ubuntu,a=focal-security, o=UbuntuESMApps,a=focal-apps-security, o=UbuntuESM,a=focal-infra-security, origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-security, origin=Ubuntu,archive=focal-updates, origin=SecureDrop,codename=focal\nInitial blacklist: \nInitial whitelist (not strict): \nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_focal-backports_universe_i18n_Translation-en'  a=focal-backports,c=universe,v=20.04,o=Ubuntu,l=Ubuntu arch='' site='us.archive.ubuntu.com' IndexType='Debian Translation Index' Size=54888 ID:30> with -32768 pin\nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_focal-backports_universe_binary-amd64_Packages'  a=focal-backports,c=universe,v=20.04,o=Ubuntu,l=Ubuntu arch='amd64' site='us.archive.ubuntu.com' IndexType='Debian Package Index' Size=109755 ID:29> with -32768 pin\nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_focal-backports_main_i18n_Translation-en'  a=focal-backports,...Using (^linux-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^linux-.*-5\\.15\\.173\\-1$|^kfreebsd-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^kfreebsd-.*-5\\.15\\.173\\-1$|^gnumach-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^gnumach-.*-5\\.15\\.173\\-1$|^.*-modules-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-modules-5\\.15\\.173\\-1$|^.*-kernel-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-kernel-5\\.15\\.173\\-1$|^linux-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^linux-.*-5\\.15\\.173\\-1$|^kfreebsd-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^kfreebsd-.*-5\\.15\\.173\\-1$|^gnumach-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^gnumach-.*-5\\.15\\.173\\-1$|^.*-modules-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-modules-5\\.15\\.173\\-1$|^.*-kernel-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-kernel-5\\.15\\.173\\-1$) regexp to find running kernel packages\npkgs that look like they should be upgraded: \n\rFetched 0 B in 0s (0 B/s)                                                       \nfetch.run() result: 0\ndpkg is configured not to cause conffile prompts\nNo packages found that can be upgraded unattended and no pending auto-removals\nThe list of kept packages can't be calculated in dry-run mode.\n"
E        +  where "Starting unattended upgrades script\nAllowed origins are: o=Ubuntu,a=focal, o=Ubuntu,a=focal-security, o=UbuntuESMApps,a=focal-apps-security, o=UbuntuESM,a=focal-infra-security, origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-security, origin=Ubuntu,archive=focal-updates, origin=SecureDrop,codename=focal\nInitial blacklist: \nInitial whitelist (not strict): \nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_focal-backports_universe_i18n_Translation-en'  a=focal-backports,c=universe,v=20.04,o=Ubuntu,l=Ubuntu arch='' site='us.archive.ubuntu.com' IndexType='Debian Translation Index' Size=54888 ID:30> with -32768 pin\nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_focal-backports_universe_binary-amd64_Packages'  a=focal-backports,c=universe,v=20.04,o=Ubuntu,l=Ubuntu arch='amd64' site='us.archive.ubuntu.com' IndexType='Debian Package Index' Size=109755 ID:29> with -32768 pin\nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_focal-backports_main_i18n_Translation-en'  a=focal-backports,...Using (^linux-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^linux-.*-5\\.15\\.173\\-1$|^kfreebsd-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^kfreebsd-.*-5\\.15\\.173\\-1$|^gnumach-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^gnumach-.*-5\\.15\\.173\\-1$|^.*-modules-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-modules-5\\.15\\.173\\-1$|^.*-kernel-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-kernel-5\\.15\\.173\\-1$|^linux-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^linux-.*-5\\.15\\.173\\-1$|^kfreebsd-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^kfreebsd-.*-5\\.15\\.173\\-1$|^gnumach-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^gnumach-.*-5\\.15\\.173\\-1$|^.*-modules-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-modules-5\\.15\\.173\\-1$|^.*-kernel-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-kernel-5\\.15\\.173\\-1$) regexp to find running kernel packages\npkgs that look like they should be upgraded: \n\rFetched 0 B in 0s (0 B/s)                                                       \nfetch.run() result: 0\ndpkg is configured not to cause conffile prompts\nNo packages found that can be upgraded unattended and no pending auto-removals\nThe list of kept packages can't be calculated in dry-run mode.\n" = CommandResult(command=b'sudo unattended-upgrades --dry-run --debug', exit_status=0, stdout=b"Starting unattended upgrades script\nAllowed origins are: o=Ubuntu,a=focal, o=Ubuntu,a=focal-security, o=UbuntuESMApps,a=focal-apps-security, o=UbuntuESM,a=focal-infra-security, origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-security, origin=Ubuntu,archive=focal-updates, origin=SecureDrop,codename=focal\nInitial blacklist: \nInitial whitelist (not strict): \nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_focal-backports_universe_i18n_Translation-en'  a=focal-backports,c=universe,v=20.04,o=Ubuntu,l=Ubuntu arch='' site='us.archive.ubuntu.com' IndexType='Debian Translation Index' Size=54888 ID:30> with -32768 pin\nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_focal-backports_universe_binary-amd64_Packages'  a=focal-backports,c=universe,v=20.04,o=Ubuntu,l=Ubuntu arch='amd64' site='us.archive.ubuntu.com' IndexType='Debian Package Index' Size=109755 ID:29> with -32768 pin\nMarking not allowed <apt_pkg.PackageFile object: filename:'/var/lib/apt/lists/us.a....*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^linux-.*-5\\.15\\.173\\-1$|^kfreebsd-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^kfreebsd-.*-5\\.15\\.173\\-1$|^gnumach-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^gnumach-.*-5\\.15\\.173\\-1$|^.*-modules-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-modules-5\\.15\\.173\\-1$|^.*-kernel-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-kernel-5\\.15\\.173\\-1$|^linux-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^linux-.*-5\\.15\\.173\\-1$|^kfreebsd-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^kfreebsd-.*-5\\.15\\.173\\-1$|^gnumach-.*-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^gnumach-.*-5\\.15\\.173\\-1$|^.*-modules-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-modules-5\\.15\\.173\\-1$|^.*-kernel-5\\.15\\.173\\-1\\-grsec\\-securedrop$|^.*-kernel-5\\.15\\.173\\-1$) regexp to find running kernel packages\npkgs that look like they should be upgraded: \n\rFetched 0 B in 0s (0 B/s)                                                       \nfetch.run() result: 0\ndpkg is configured not to cause conffile prompts\nNo packages found that can be upgraded unattended and no pending auto-removals\nThe list of kept packages can't be calculated in dry-run mode.\n", stderr=None).stdout

common/test_automatic_updates.py:161: AssertionError
__________________ test_fpf_apt_repo_present[paramiko://app] ___________________
[gw0] linux -- Python 3.11.2 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python

host = <testinfra.host.Host paramiko://app>

    def test_fpf_apt_repo_present(host):
        """
        Ensure the FPF apt repo, apt.freedom.press, is configured.
        This repository is necessary for the SecureDrop Debian packages,
        including:
    
          * securedrop-app-code
          * securedrop-keyring
          * securedrop-grsec
    
        Depending on the host, additional FPF-maintained packages will be
        installed, e.g. for OSSEC. Install state for those packages
        is tested separately.
        """
    
        # If the var fpf_apt_repo_url test var is apt-test, validate that the
        # apt repository is configured on the host
        if test_vars.fpf_apt_repo_url == "https://apt-test.freedom.press":
            f = host.file("/etc/apt/sources.list.d/apt_test_freedom_press.list")
        else:
            f = host.file("/etc/apt/sources.list.d/apt_freedom_press.list")
        repo_regex = rf"^deb \[arch=amd64\] {re.escape(test_vars.fpf_apt_repo_url)} "
        rf"{re.escape(host.system_info.codename)} main$"
>       assert f.contains(repo_regex)
E       AssertionError: assert False
E        +  where False = <bound method File.contains of <file /etc/apt/sources.list.d/apt_freedom_press.list>>('^deb \\[arch=amd64\\] https://apt\\.freedom\\.press ')
E        +    where <bound method File.contains of <file /etc/apt/sources.list.d/apt_freedom_press.list>> = <file /etc/apt/sources.list.d/apt_freedom_press.list>.contains

common/test_fpf_apt_repo.py:33: AssertionError
__________________ test_fpf_apt_repo_present[paramiko://mon] ___________________
[gw0] linux -- Python 3.11.2 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python

host = <testinfra.host.Host paramiko://mon>

    def test_fpf_apt_repo_present(host):
        """
        Ensure the FPF apt repo, apt.freedom.press, is configured.
        This repository is necessary for the SecureDrop Debian packages,
        including:
    
          * securedrop-app-code
          * securedrop-keyring
          * securedrop-grsec
    
        Depending on the host, additional FPF-maintained packages will be
        installed, e.g. for OSSEC. Install state for those packages
        is tested separately.
        """
    
        # If the var fpf_apt_repo_url test var is apt-test, validate that the
        # apt repository is configured on the host
        if test_vars.fpf_apt_repo_url == "https://apt-test.freedom.press":
            f = host.file("/etc/apt/sources.list.d/apt_test_freedom_press.list")
        else:
            f = host.file("/etc/apt/sources.list.d/apt_freedom_press.list")
        repo_regex = rf"^deb \[arch=amd64\] {re.escape(test_vars.fpf_apt_repo_url)} "
        rf"{re.escape(host.system_info.codename)} main$"
>       assert f.contains(repo_regex)
E       AssertionError: assert False
E        +  where False = <bound method File.contains of <file /etc/apt/sources.list.d/apt_freedom_press.list>>('^deb \\[arch=amd64\\] https://apt\\.freedom\\.press ')
E        +    where <bound method File.contains of <file /etc/apt/sources.list.d/apt_freedom_press.list>> = <file /etc/apt/sources.list.d/apt_freedom_press.list>.contains

common/test_fpf_apt_repo.py:33: AssertionError
_____________________ test_migration_check[paramiko://app] _____________________
[gw0] linux -- Python 3.11.2 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python

host = <testinfra.host.Host paramiko://app>

    def test_migration_check(host):
        """Verify our migration check script works"""
        if host.system_info.codename != "focal":
            pytest.skip("only applicable/testable on focal")
    
        with host.sudo():
            # remove state file so we can see if it works
            if host.file("/etc/securedrop-noble-migration.json").exists:
                host.run("rm /etc/securedrop-noble-migration.json")
            cmd = host.run("systemctl start securedrop-noble-migration-check")
            assert cmd.rc == 0
            while host.service("securedrop-noble-migration-check").is_running:
                time.sleep(1)
    
            # JSON state file was created
            assert host.file("/etc/securedrop-noble-migration.json").exists
    
            cmd = host.run("cat /etc/securedrop-noble-migration.json")
            assert cmd.rc == 0
    
            contents = json.loads(cmd.stdout)
            print(contents)
            # The script did not error out
            if "error" in contents:
                # Run the script manually to get the error message
                cmd = host.run("securedrop-noble-migration-check")
                print(cmd.stdout)
                # We'll fail in the next line after this
            assert "error" not in contents
            # staging CI jobs don't have enough free space, so just check
            # that it returned a value for it
            assert "free_space" in contents
            del contents["free_space"]
            # All the values should be True
>           assert all(contents.values())
E           AssertionError: assert False
E            +  where False = all(dict_values([True, False, True, True]))
E            +    where dict_values([True, False, True, True]) = <built-in method values of dict object at 0x71d6d86003c0>()
E            +      where <built-in method values of dict object at 0x71d6d86003c0> = {'apt': True, 'ssh': True, 'systemd': True, 'ufw': False}.values

common/test_release_upgrades.py:70: AssertionError
----------------------------- Captured stdout call -----------------------------
{'ssh': True, 'ufw': False, 'free_space': True, 'apt': True, 'systemd': True}
_____________________ test_migration_check[paramiko://mon] _____________________
[gw0] linux -- Python 3.11.2 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python

host = <testinfra.host.Host paramiko://mon>

    def test_migration_check(host):
        """Verify our migration check script works"""
        if host.system_info.codename != "focal":
            pytest.skip("only applicable/testable on focal")
    
        with host.sudo():
            # remove state file so we can see if it works
            if host.file("/etc/securedrop-noble-migration.json").exists:
                host.run("rm /etc/securedrop-noble-migration.json")
            cmd = host.run("systemctl start securedrop-noble-migration-check")
            assert cmd.rc == 0
            while host.service("securedrop-noble-migration-check").is_running:
                time.sleep(1)
    
            # JSON state file was created
            assert host.file("/etc/securedrop-noble-migration.json").exists
    
            cmd = host.run("cat /etc/securedrop-noble-migration.json")
            assert cmd.rc == 0
    
            contents = json.loads(cmd.stdout)
            print(contents)
            # The script did not error out
            if "error" in contents:
                # Run the script manually to get the error message
                cmd = host.run("securedrop-noble-migration-check")
                print(cmd.stdout)
                # We'll fail in the next line after this
            assert "error" not in contents
            # staging CI jobs don't have enough free space, so just check
            # that it returned a value for it
            assert "free_space" in contents
            del contents["free_space"]
            # All the values should be True
>           assert all(contents.values())
E           AssertionError: assert False
E            +  where False = all(dict_values([True, False, False, True]))
E            +    where dict_values([True, False, False, True]) = <built-in method values of dict object at 0x71d6d85aea80>()
E            +      where <built-in method values of dict object at 0x71d6d85aea80> = {'apt': False, 'ssh': True, 'systemd': True, 'ufw': False}.values

common/test_release_upgrades.py:70: AssertionError
----------------------------- Captured stdout call -----------------------------
{'ssh': True, 'ufw': False, 'free_space': True, 'apt': False, 'systemd': True}
____________________ test_iptables_packages[paramiko://app] ____________________
[gw0] linux -- Python 3.11.2 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python

host = <testinfra.host.Host paramiko://app>

    def test_iptables_packages(host):
        """
        Focal hosts should use iptables-persistent for enforcing
        firewall config across reboots.
        """
        assert host.package("iptables-persistent").is_installed
>       assert not host.package("ufw").is_installed
E       AssertionError: assert not True
E        +  where True = <package ufw>.is_installed
E        +    where <package ufw> = <class 'testinfra.modules.base.DebianPackage'>('ufw')
E        +      where <class 'testinfra.modules.base.DebianPackage'> = <testinfra.host.Host paramiko://app>.package

common/test_system_hardening.py:178: AssertionError
____________________ test_iptables_packages[paramiko://mon] ____________________
[gw0] linux -- Python 3.11.2 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python

host = <testinfra.host.Host paramiko://mon>

    def test_iptables_packages(host):
        """
        Focal hosts should use iptables-persistent for enforcing
        firewall config across reboots.
        """
        assert host.package("iptables-persistent").is_installed
>       assert not host.package("ufw").is_installed
E       AssertionError: assert not True
E        +  where True = <package ufw>.is_installed
E        +    where <package ufw> = <class 'testinfra.modules.base.DebianPackage'>('ufw')
E        +      where <class 'testinfra.modules.base.DebianPackage'> = <testinfra.host.Host paramiko://mon>.package

common/test_system_hardening.py:178: AssertionError

• QA Matrix checks pass

Command Line User Generation

• Can successfully add admin user and login

(Optional) Administration

• I have backed up and successfully restored the app server following the backup documentation
• If doing upgrade testing, make a backup on 2.10.1 and restore this backup on this release candidate, verifying that the Redis password in /etc/redis/redis.conf changes
• "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
• Can successfully add journalist account with HOTP authentication

(Optional) Application Acceptance Testing

Source Interface

Landing page base cases

• JS warning bar does not appear when using Security Slider high
• JS warning bar does appear when using Security Slider Low

First submission base cases

• On generate page, refreshing page produces a new 7-word codename
• On submit page, empty submissions produce flashed message
• On submit page, short message submitted successfully
• On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
• On submit page, file less than 500 MB submitted successfully

Returning source base cases

• Nonexistent codename cannot log in
• Empty codename cannot log in
• Legitimate codename can log in
• Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

• Can log in with 2FA tokens
• incorrect password cannot log in
• invalid 2fa token cannot log in
• 2fa immediate reuse cannot log in
• Journalist account with HOTP can log in

Index base cases

• Filter by codename works
• Starring and unstarring works

Unclear: I'm not seeing the select all button now?

• Click select all selects all submissions
• Selecting all and clicking "Download" works

Individual source page

• You can submit a reply and a flashed message and new row appears
• You cannot submit an empty reply
• Clicking "Delete Source Account" and the source and docs are deleted
• You can click on a document and successfully decrypt using application private key

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

• The Updater GUI appears on boot

2.11.0 release-specific changes

Noble prep (misc PRs)

• Verify that ufw and haveged are not installed on either server
• Verify that sdssh user group exists containing SD admin acccount, and that the ssh group is empty, on both servers

Check script and banner (#7334, #7363, 7348)

• After installation and reboot, verify that a file /etc/securedrop-noble-migration.json has been created on the application server
• Install the ufw package on the application server and run sudo /usr/bin/securedrop-noble-migration-check - verify that the ufw value in the json file is now False
• Uninstall the ufw package and run sudo /usr/bin/securedrop-noble-migration-check again - verify that the ufw value in the json file is now False
• Edit the json file and ensure at least one value is false
• Log into the JI and verify that a banner is displayed with the Noble readiness warning message
• Edit the json file again, flipping all values to True
• Log into the JI and verify that the banner is not displayed
• In /var/www/securedrop/server_os.py, set FOCAL_ENDOFLIFE to a past date.
• Log into the JI and verify that a banner is displayed with the Focal EOL warning message
• In /var/www/securedrop/server_os.py, reset FOCAL_ENDOFLIFE to its original value

Preflight testing

Basic testing

• Install or upgrade occurs without error (from apt-qa.freedom.press per preflight procedure)
• Source interface is available and version string indicates it is 2.11.0
• A message can be successfully submitted

Tails

• The updater GUI appears on boot
• The update successfully occurs to 2.11.0
• After reboot, updater GUI no longer appears

[cfm]

• Install target: NUC11s
• Tails version: 6.9
• Test Scenario: upgrade
• SSH over Tor?: yes
• Tor PoW enabled?: yes
• Release candidate: 2.11.0-rc2
• General notes: On this long-running QA instance I've prioritized release-specific upgrade testing, all of which has passed. I'll come back to application-level acceptance testing if I have time.

Basic Server Testing

• After installing the testinfra dependencies, all tests in ./securedrop-admin verify are passing:
  • Install dependencies on Admin Workstation with cd ~/Persistent/securedrop && ./securedrop-admin setup -t
  • Run tests with ./securedrop-admin verify (this will take a while)
  • Remove test dependencies: rm -rf admin/.venv3/ && ./securedrop-admin setup
• QA Matrix checks pass

Command Line User Generation

• [-] Can successfully add admin user and login

(Optional) Administration

• I have backed up and successfully restored the app server following the backup documentation
• If doing upgrade testing, make a backup on 2.10.1 and restore this backup on this release candidate, verifying that the Redis password in /etc/redis/redis.conf changes
• [-] "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
• [-] Can successfully add journalist account with HOTP authentication

(Optional) Application Acceptance Testing

Source Interface

Landing page base cases

• JS warning bar does not appear when using Security Slider high
• JS warning bar does appear when using Security Slider Low

First submission base cases

• On generate page, refreshing page produces a new 7-word codename
• On submit page, empty submissions produce flashed message
• On submit page, short message submitted successfully
• On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
• On submit page, file less than 500 MB submitted successfully

Returning source base cases

• Nonexistent codename cannot log in
• Empty codename cannot log in
• Legitimate codename can log in
• Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

• Can log in with 2FA tokens
• incorrect password cannot log in
• invalid 2fa token cannot log in
• 2fa immediate reuse cannot log in
• Journalist account with HOTP can log in

Index base cases

• Filter by codename works
• Starring and unstarring works
• Click select all selects all submissions
• Selecting all and clicking "Download" works

Individual source page

• You can submit a reply and a flashed message and new row appears
• You cannot submit an empty reply
• Clicking "Delete Source Account" and the source and docs are deleted
• You can click on a document and successfully decrypt using application private key

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

• The Updater GUI appears on boot

2.11.0 release-specific changes

Noble prep (misc PRs)

• Verify that ufw and haveged are not installed on either server

Via ssh {app,mon} sudo apt-cache policy ufw haveged.

• Verify that sdssh user group exists containing SD admin acccount, and that the ssh group is empty, on both servers

Via ssh {app,mon} getent group {ssh,sdssh}.

Check script and banner (#7334, #7363, 7348)

• After installation and reboot, verify that a file /etc/securedrop-noble-migration.json has been created on the application server

• Install the ufw package on the application server and run sudo /usr/bin/securedrop-noble-migration-check - verify that the ufw value in the json file is now False

• Run sudo systemctl start securedrop-remove-packages and then run sudo /usr/bin/securedrop-noble-migration-check again - verify that the ufw value in the json file is now False

• Edit the json file and ensure at least one value is false; then run sudo systemctl restart apache2 to restart the JI

• Log into the JI and verify that a banner is displayed with the Noble readiness warning message

• Edit the json file again, flipping all values to True and restart the JI with sudo systemctl restart apache2

• Log into the JI and verify that the banner is not displayed

• In /var/www/securedrop/server_os.py, set FOCAL_ENDOFLIFE to a past date; and restart the JI with sudo systemctl restart apache2

• Log into the JI and verify that a banner is displayed with the Focal EOL warning message

  • ...and the Source Interface is disabled.

• In /var/www/securedrop/server_os.py, reset FOCAL_ENDOFLIFE to its original value; and restart the JI with sudo systemctl restart apache2

  • The banner is gone...
  • ...and the Source Interface is reenabled.

[cfm]

• Install target: VMs
• Tails version: 6.10
• Test Scenario: clean installation
• SSH over Tor?: yes
• Tor PoW enabled?: yes
• Release candidate: 2.11.0-rc2
• General notes: I've prioritized release-specific upgrade testing, which has passed except as noted below.

Basic Server Testing

• After installing the testinfra dependencies, all tests in ./securedrop-admin verify are passing:
  • Install dependencies on Admin Workstation with cd ~/Persistent/securedrop && ./securedrop-admin setup -t
  • Run tests with ./securedrop-admin verify (this will take a while)
  • Remove test dependencies: rm -rf admin/.venv3/ && ./securedrop-admin setup
• QA Matrix checks pass

Except for:

• [2.11.0-rc2] recent Focal releases include an ubuntu_pro_apt_news that AppArmor blocks #7385

Command Line User Generation

• Can successfully add admin user and login

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

• The Updater GUI appears on boot

2.11.0 release-specific changes

Noble prep (misc PRs)

• Verify that ufw and haveged are not installed on either server
• Verify that sdssh user group exists containing SD admin acccount, and that the ssh group is empty, on both servers

Check script and banner (#7334, #7363, 7348)

• After installation and reboot, verify that a file /etc/securedrop-noble-migration.json has been created on the application server

• Install the ufw package on the application server and run sudo /usr/bin/securedrop-noble-migration-check - verify that the ufw value in the json file is now False

• Run sudo systemctl start securedrop-remove-packages and then run sudo /usr/bin/securedrop-noble-migration-check again - verify that the ufw value in the json file is now False

• Edit the json file and ensure at least one value is false; then run sudo systemctl restart apache2 to restart the JI

• Log into the JI and verify that a banner is displayed with the Noble readiness warning message

• Edit the json file again, flipping all values to True and restart the JI with sudo systemctl restart apache2

• Log into the JI and verify that the banner is not displayed

• In /var/www/securedrop/server_os.py, set FOCAL_ENDOFLIFE to a past date; and restart the JI with sudo systemctl restart apache2

• Log into the JI and verify that a banner is displayed with the Focal EOL warning message

• In /var/www/securedrop/server_os.py, reset FOCAL_ENDOFLIFE to its original value; and restart the JI with sudo systemctl restart apache2
  with sudo systemctl restart apache2

• Log into the JI and verify that the banner is not displayed

• In /var/www/securedrop/server_os.py, set FOCAL_ENDOFLIFE to a past date; and restart the JI with sudo systemctl restart apache2

• Log into the JI and verify that a banner is displayed with the Focal EOL warning message

  • ...and the Source Interface is disabled.

• In /var/www/securedrop/server_os.py, reset FOCAL_ENDOFLIFE to its original value; and restart the JI with sudo systemctl restart apache2

  • The banner is gone...
  • ...and the Source Interface is reenabled.