Skip to content

fix corner cases that allow stale properties on user slices - #33

Merged
mergify[bot] merged 5 commits into
flux-framework:mainfrom
grondo:deviceallow-stale
Sep 23, 2026
Merged

mergify[bot] merged 5 commits into
flux-framework:mainfrom
grondo:deviceallow-stale

Conversation

@grondo

@grondo grondo commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

This PR fixes a few bugs that would allow stale properties, i.e. access to resources that aren't allocated, in user slices.

This is built on top of #32

@garlick garlick left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@grondo grondo added the merge-when-passing mergify will merge this PR once all tests are passing label Sep 23, 2026
@grondo

grondo commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks!

@mergify mergify Bot added the queued label Sep 23, 2026
@mergify

mergify Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • ✅ Entered queue — 2026-09-23 19:54 UTC · Rule: default · triggered by rule default
  • ❌ Checks failed · in-place
  • 🚫 Left the queue — 2026-09-23 19:57 UTC · at 59c816baa88f9058ada59b7aebb4c0bf00413f5c

This pull request spent 3 minutes 50 seconds in the queue, with no time running CI.

Waiting for
  • any of: [🛡 GitHub branch protection]
    • check-neutral = el8
    • check-skipped = el8
    • check-success = el8
  • any of: [🛡 GitHub branch protection]
    • check-neutral = fedora40
    • check-skipped = fedora40
    • check-success = fedora40
  • any of: [🛡 GitHub branch protection]
    • check-neutral = el9
    • check-skipped = el9
    • check-success = el9
  • any of: [🛡 GitHub branch protection]
    • check-neutral = el10
    • check-skipped = el10
    • check-success = el10
  • any of: [🛡 GitHub branch protection]
    • check-neutral = noble
    • check-skipped = noble
    • check-success = noble
  • any of: [🛡 GitHub branch protection]
    • check-neutral = el8 system tests
    • check-skipped = el8 system tests
    • check-success = el8 system tests
All conditions

Reason

The merge conditions cannot be satisfied due to failing checks

- validate commits
- fedora40
- el10
- noble
- spelling
- python linting

Hint

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio queue comment.

Requeued — the merge queue status continues in this comment ↓.

Problem: When a user has multiple jobs on a node, and the last job that
allocated any GPUs ends, the user slice could keep access to devices
that are no longer allocated. This occurs because `set-property` only
changes properties that are named, and the resource mapper may omit
DeviceAllow entirely when no GPUs are allocated, so the previous
list remains assigned to the slice.

Always emit a bare "DeviceAllow=" ahead of the entries, which resets
the list, so the slice ends up with just the devices computed for
the jobs running now. It precedes them because a reset is order
sensitive.

Update the argument-splitting test, which asserts the full list of
DeviceAllow arguments, for the added reset.

Assisted-by: Claude:Opus-5
Problem: Nothing covers how a DeviceAllow list is reset, so a stale
device left on the slice by an earlier update would not be caught.

Add tests that modify_slice resets the list when the mapper grants no
devices, and that the reset precedes the entries, since a reset after
them discards them.

Assisted-by: Claude:Opus-5
Problem: A user's login sessions can be held to a device policy set
for a job that has already ended. set-property only changes the
properties it names, so a DevicePolicy from an earlier update stays
on the slice when a later one omits it.

A mapper that leaves device access unrestricted does so by returning
DevicePolicy=auto or omitting it. Omitting it after a job that set
"closed" leaves the slice closed, so sessions in it are confined to
the standard pseudo devices even though nothing asked for that.

Emit DevicePolicy=auto alongside the DeviceAllow reset, ahead of the
mapper's own properties, which override it when set. The default
value is used rather than an empty one, which systemd rejects.

Assisted-by: Claude:Opus-5
Problem: Nothing covers how DevicePolicy is reset, or that a mapper
leaving device access unrestricted gets that, so neither a stale
policy nor a reset that wrongly restricted a job would be caught.

Add tests that modify_slice resets the policy when the mapper omits
it, that a policy from the mapper overrides the reset, and that an
"auto" policy with no devices granted is passed through, which is how
access is left unrestricted.

Assisted-by: Claude:Opus-5
Problem: The user slice is left with no resource constraints at all
when the sdexec-mapper lookup fails. The exception is caught and
reported as an empty property set, which the caller takes to mean
there is nothing to constrain, so no cpuset and no device containment
are applied and the prolog exits successfully.

Let the error propagate, as the other constraint steps already do.
The prolog and housekeeping report it and exit non-zero, draining the
node, rather than admitting sessions to an unconstrained slice.
Resource constraints are disabled with exec.sdexec-constrain-
resources, not by an unreachable mapper.

Update the lookup failure test, which asserted the empty result.

Assisted-by: Claude:Opus-5
@mergify
mergify Bot force-pushed the deviceallow-stale branch from f647fdc to 59c816b Compare September 23, 2026 19:55
@mergify

mergify Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • ✅ Entered queue — 2026-09-23 20:12 UTC · Rule: default · triggered by rule default
  • ✅ Checks skipped · PR is already up-to-date
  • ✅ Merged — 2026-09-23 20:13 UTC · at 2a3a82064321a0fa620e1c4438b66cb0a850e442 · merge

This pull request spent 46 seconds in the queue, including 8 seconds running CI.

Required conditions to merge

@mergify
mergify Bot merged commit 2a3a820 into flux-framework:main Sep 23, 2026
19 of 21 checks passed
@mergify mergify Bot removed the queued label Sep 23, 2026
@grondo
grondo deleted the deviceallow-stale branch September 24, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dequeued merge-when-passing mergify will merge this PR once all tests are passing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants