Bump supported Fedora release to 37 #859

eaon · 2023-02-27T18:53:37Z

Description of Changes

Fedora 37 will be shipped with Qubes OS R4.1.2. As soon as Qubes R4.1.2 has been released, this should probably be merged so that the next release doesn't download Fedora 36 unnecessarily.

Fixes #841

Testing

There are no references to Fedora 36 left in the repository
sd-fedora-37-dvm is used in place of sd-fedora-dvm through the repo (except for the salt state removing the latter)
Check out this branch in your sd-dev vm and ensure you set the SDW environment variables. Ensure SD VMs (lookin at you, sd-whonix) are all shut down (otherwise, during make test, you could encounter tests that report that VMs aren't fully updated). In dom0 make clone then make dev
- make dev completes without error
- sd-fedora-37-dvm exists and is based on fedora-37
- sd-fedora-dvm is removed
- make test passes in dom0

Note that removing the f36 template/f36-dvm template is not part of the template update process.

eloquence · 2023-03-06T22:09:09Z

(I've taken the liberty to push a small commit to update the dom0 tests to the new template version.)

eaon · 2023-03-07T13:17:04Z

The Fedora 37 template is available now so technically this is already ready for review.

eloquence · 2023-03-08T04:10:24Z

I consistently get the following error when attempting to migrate from fedora-36:

qubesadmin.exc.QubesVMNotHaltedError: Cannot change template while there are running DispVMs based on this DVM template

To test the migration, I built an RPM from the branch, installed it in dom0, and then ran the updater with /opt/securedrop/launcher/sdw-launcher.py, including once after a clean reboot. (I wanted to see the migration behavior a user would see.)

Fedora 37 template was released on 2023-03-03, and will be shipped by default by Qubes OS R4.1.2. For upgrades, we also need to ensure we're not changing the template for sd-fedora-dvm before the appropriate time.

eloquence

Thanks for working through this! Unfortunately I'm still hitting an error during the migration. That said, it looks like the migration did complete -- the non-disposable sys- templates are using fedora-37, and sd-fedora-dvm uses the new template. 🎉

This Gist has the Salt error I'm getting: https://gist.github.com/eloquence/e5bca6fcd12ed6d7b7364c180bfe8bac

You may have an immediate hypothesis on what's causing this, but I'll leave one note at the line level on a part of the logic that's confusing to me.

eloquence · 2023-03-15T00:03:17Z

dom0/sd-sys-vms.sls

 sd-{{ sys_vm }}-fedora-version-update:
  qvm.vm:
    - name: {{ sys_vm }}
    - prefs:
      - template: {{ sd_supported_fedora_template }}
    - require:
      - cmd: sd-{{ sys_vm }}-fedora-version-halt-wait
-{% if sd_supported_fedora_template.endswith("-dvm") %}
+{% if not dvm_needs_updating and sd_supported_fedora_template.endswith("-dvm") %}


I'm confused by the not dvm_needs_updating addition here -- what's the purpose of this additional check?

That said, what appears to be happening on my system is that this evaluates to true, but the create- block itself is never defined, because the VM is already present and the {% if required_dispvm not in existing_vms %} block evaluates to false.

eaon · 2023-03-20T14:25:59Z

I'm assuming you ran everything again without doing any manual Fedora 37 related cleaning operations, correct? If that's the case, your machine is in a state that the current patch would not get to by itself, which is also why it's not covered. That being said, I think we shouldn't assume that SDW is used in a completely isolated environment, power users may have done some manual template updating, so I'll take another look.

Just to spell out the permutations of cases we would run into:

Fresh install: SDW is not installed, neither fedora-37 nor sd-fedora-dvm exist
Regular upgrade: SDW is installed, fedora-37 does not exist, sd-fedora-dvm needs updating
Manual intervention or previous broken run: SDW is installed, fedora-37 exists, sd-fedora-dvm needs updating
All up-to-date: SDW is installed, fedora-37 exists, sd-fedora-dvm does not need updating

eloquence · 2023-03-21T00:32:45Z

Yes, that's correct, I did not manually remove fedora-37 before trying again. What's odd about my run, though, is that it seemed to eventually fit the "All up-to-date" description, but still kept failing with the error above.

The only other state I can think of is with Qubes 4.1.2, which was released last week. A fresh install on 4.1.2 would be:

Fresh install: SDW is not installed, fedora-37 exists but sd-fedora-dvm does not.

rocodes · 2023-03-21T21:36:29Z

(Stepping through now to put some more eyes on this, will first try with no fedora-37 preinstall aka "regular update" scenario, then with fedora-37 template preinstalled aka "manual intervention or previous broken run")

@gonzalo-bulnes

Shout out to @gonzalo-bulnes for bringing this up as a potential solution

eaon · 2023-03-22T14:35:51Z

I pushed some changes after a note from @gonzalo-bulnes came to mind again.

I have not tested this, but I think this should cover all cases including the one @eloquence pointed out.

This is all significantly more complex than https://github.com/freedomofpress/securedrop-workstation/pull/802/files in part due what I've introduced with #764 and #784. I consider the numbered approach now taken by this PR a workaround as removing and recreating an AppVM with the same contents wastes time and resources. However, this is a restriction of our deployment strategy which is IMO in dire need of fundamental restructuring. More notes on the direction I think would remove classes of pain-points is documented in https://github.com/freedomofpress/securedrop-engineering/pull/22

zenmonkeykstop · 2023-03-27T17:06:55Z

Given 4.1.2 is out and recommended, the fresh install case should actually be "SDW not installed, f37 exists, sd-fedora-dvm doesn't exist." For update cases we still have to allow for the 4.1.1 cases.

eloquence · 2023-03-28T23:45:34Z

On my system which has fedora-37 already installed, I am still encountering the error below. Note that fedora-37-dvm also exists on this system. This is why I think this Jinja check fails to include the create-fedora-37 step, which is then later required.

I'm going to try removing the fedora-37-dvm AppVM to see if that causes a successful run.

----------
          ID: sd-sys-firewall-fedora-version-update
    Function: qvm.vm
        Name: sys-firewall
      Result: False
     Comment: The following requisites were not found:
                                 require:
                                     qvm: create-fedora-37-dvm
     Started: 18:33:02.926407
    Duration: 0.002 ms
     Changes:   
----------
          ID: sd-sys-firewall-fedora-version-start
    Function: qvm.start
        Name: sys-firewall
      Result: False
     Comment: One or more requisite failed: sd-sys-vms.sd-sys-firewall-fedora-version-update
     Started: 18:33:02.926714
    Duration: 0.003 ms
     Changes:

eloquence · 2023-03-29T00:12:46Z

Yes, deleting fedora-37-dvm resulted in a successful run with all expected updates completed. 🎉

Repeated runs after that also complete without errors.

I think this is an edge case that resulted from a partially completed upgrade. I don't think you would encounter it on a Qubes 4.1.2 system with fedora-37-dvm already created, because such a system would also have sys-firewall already set to fedora-37.

If we haven't tested that already, I think that would be useful to verify. Assuming that is the case, I think addressing this issue would be mostly a matter of overall resilience.

rocodes · 2023-03-29T02:34:17Z

Thanks for the sleuthing - you're right, I thought initially you were encountering the issue after only having downloaded the f37 template (as opposed to having set up the dvm as well).

This is fixable, thanks for flagging.

rocodes · 2023-03-29T19:38:14Z

(A little bit annoying to remove the jinja optimization that was in there, but for now it's fine. We'll want to revisit our overall strategy here, as was suggested earlier)

rocodes · 2023-03-29T22:42:18Z

I'm stepping through one last time for good measure on a brand new 4.1.2 install. If someone wants to review on an existing 4.1 workstation by following the original test plan, that would be great (since I'm the last one to push some code I shouldn't review).

cfm

Without any fedora-37 yet installed, and with the caveat that this test plan wasn't written when I started the make clone && sdw-admin --apply sequence :-):

Check out this branch in your sd-dev vm and ensure you set the SDW environment variables. ~~Ensure SD VMs (lookin at you, sd-whonix) are all shut down (otherwise, during make test, you could encounter tests that report that VMs aren't fully updated).~~ In dom0 make clone then ~~make dev~~ sdw-admin --apply
- ~~make dev~~ sdw-admin-apply completes without error
- sd-fedora-37-dvm exists and is based on fedora-37
- sd-fedora-dvm is removed
- make test passes in dom0
  - ...modulo qrexec policy changes I've made because this is my lowercase-W workstation too... :-)
  - ...with one meaningful exception:

securedrop-workstation/tests/test_vms_exist.py

Line 9 in d704568

EXPECTED_KERNEL_VERSION = "5.15.41-grsec-workstation"

[user@dom0 securedrop-workstation]$ qvm-run --pass-io sd-app uname -r
5.15.89-grsec-workstation

later changes address this issue

rocodes · 2023-03-30T00:33:10Z

@cfm I still have the 5.15.41 grsec kernel when I've only ever built from dev on the SDW laptop. I'm guessing you had this machine on staging or prod before, since we don't do nightly kernel builds - that would explain the test failure.

Thank you for the review :) I will finish clean install QA then merge.

eloquence

Didn't do a full re-run, but did do another make dev with latest changes and all LGTM, so adding another +1 for good measure :)

rocodes · 2023-03-30T19:29:50Z

Thanks both. Tested on fresh 4.1.2 install; fedora 37 template handled just fine, although the first time, the installation errored out because our own template (workstation-bullseye) was not successfully downloaded. It was successful on a second run. Will keep an eye out for that issue in QA, but I think we're OK to proceed here.

eaon marked this pull request as draft February 27, 2023 18:56

eaon self-assigned this Mar 1, 2023

eaon marked this pull request as ready for review March 7, 2023 13:16

eaon marked this pull request as draft March 9, 2023 12:44

eaon mentioned this pull request Mar 14, 2023

securedrop-workstation-bullseye template install fails after uninstall #813

Open

eaon and others added 2 commits March 14, 2023 18:11

Bump supported Fedora release to 37

15d3e7d

Fedora 37 template was released on 2023-03-03, and will be shipped by default by Qubes OS R4.1.2. For upgrades, we also need to ensure we're not changing the template for sd-fedora-dvm before the appropriate time.

Update dom0 tests

088b88e

eaon force-pushed the fedora-37 branch from 1bf2669 to 088b88e Compare March 14, 2023 17:12

eaon marked this pull request as ready for review March 14, 2023 17:13

eaon requested review from eloquence and rocodes March 14, 2023 17:13

eloquence previously requested changes Mar 14, 2023

View reviewed changes

eloquence reviewed Mar 15, 2023

View reviewed changes

rocodes assigned rocodes and unassigned rocodes Mar 15, 2023

Version SD's Fedora based DVM template.

438200c

Shout out to @gonzalo-bulnes for bringing this up as a potential solution

rocodes self-assigned this Mar 27, 2023

Remove unused Jinja variable, cleanup fedora-dvm removal state

a627feb

rocodes force-pushed the fedora-37 branch from 4cf2db1 to a627feb Compare March 27, 2023 23:38

Update tests and provisioning scripts

5f55c27

Always run qvm.vm on required vms so prereq can succeed

d58ee62

rocodes requested review from eloquence and a team March 29, 2023 21:12

rocodes mentioned this pull request Mar 29, 2023

Release securedrop-workstation-dom0-config-0.8.0 #866

Closed

15 tasks

cfm approved these changes Mar 30, 2023

View reviewed changes

eloquence approved these changes Mar 30, 2023

View reviewed changes

rocodes merged commit 67a125e into main Mar 30, 2023

legoktm deleted the fedora-37 branch May 28, 2024 15:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump supported Fedora release to 37 #859

Bump supported Fedora release to 37 #859

eaon commented Feb 27, 2023 •

edited by rocodes

Loading

eloquence commented Mar 6, 2023

eaon commented Mar 7, 2023

eloquence commented Mar 8, 2023

eloquence left a comment

eloquence Mar 15, 2023

eaon commented Mar 20, 2023 •

edited

Loading

eloquence commented Mar 21, 2023

rocodes commented Mar 21, 2023 •

edited

Loading

eaon commented Mar 22, 2023

zenmonkeykstop commented Mar 27, 2023 •

edited

Loading

eloquence commented Mar 28, 2023 •

edited

Loading

eloquence commented Mar 29, 2023 •

edited

Loading

rocodes commented Mar 29, 2023

rocodes commented Mar 29, 2023

rocodes commented Mar 29, 2023 •

edited

Loading

cfm left a comment

rocodes commented Mar 30, 2023 •

edited

Loading

eloquence left a comment

rocodes commented Mar 30, 2023 •

edited

Loading

Bump supported Fedora release to 37 #859

Bump supported Fedora release to 37 #859

Conversation

eaon commented Feb 27, 2023 • edited by rocodes Loading

Description of Changes

Testing

eloquence commented Mar 6, 2023

eaon commented Mar 7, 2023

eloquence commented Mar 8, 2023

eloquence left a comment

Choose a reason for hiding this comment

eloquence Mar 15, 2023

Choose a reason for hiding this comment

eaon commented Mar 20, 2023 • edited Loading

eloquence commented Mar 21, 2023

rocodes commented Mar 21, 2023 • edited Loading

eaon commented Mar 22, 2023

zenmonkeykstop commented Mar 27, 2023 • edited Loading

eloquence commented Mar 28, 2023 • edited Loading

eloquence commented Mar 29, 2023 • edited Loading

rocodes commented Mar 29, 2023

rocodes commented Mar 29, 2023

rocodes commented Mar 29, 2023 • edited Loading

cfm left a comment

Choose a reason for hiding this comment

rocodes commented Mar 30, 2023 • edited Loading

eloquence left a comment

Choose a reason for hiding this comment

rocodes commented Mar 30, 2023 • edited Loading

eaon commented Feb 27, 2023 •

edited by rocodes

Loading

eaon commented Mar 20, 2023 •

edited

Loading

rocodes commented Mar 21, 2023 •

edited

Loading

zenmonkeykstop commented Mar 27, 2023 •

edited

Loading

eloquence commented Mar 28, 2023 •

edited

Loading

eloquence commented Mar 29, 2023 •

edited

Loading

rocodes commented Mar 29, 2023 •

edited

Loading

rocodes commented Mar 30, 2023 •

edited

Loading

rocodes commented Mar 30, 2023 •

edited

Loading