test: harden three flaky test areas against CI races - #7342
Conversation
Fix three independently-flaking test areas that caused the last four main merges to fail CI. **Status-expiry E2E (profile-custom-emoji-status.spec.ts)** The test seeded a status expiring 2 s ahead then waited on wall-clock time for it to disappear, racing slow CI workers. Fix: install Playwright's fake clock at +3 s and call a new E2E bridge function (__BUZZ_E2E_EXPIRE_USER_STATUS_QUERIES__) to sweep the query cache immediately. The fake Date.now() makes the dialog's expirationIsFuture re-evaluate to false on its next render; the cache sweep removes the sidebar indicator. No timers, no polling. **Workflow E2Es (workflow-local-controls.spec.ts, workflows.spec.ts)** One-shot boundingBox() reads during inspector-open transitions produced null or reversed coordinates. Fix: await all three operator buttons visible + waitForAnimations before reading geometry; use expect.poll to re-measure until the layout invariant holds. In workflows.spec.ts, await the Trigger event button visible before clicking it (inspector may still be animating in after the Trigger: Message Posted click). **Rust cancellation test (fake_llm.rs)** h.send(session/cancel) is request-like; releasing the round-2 gate before the ACK arrived let round 2 win the race, producing stopReason: null. Fix: drain stdout into a buffer until the cancel ACK is received, then release the gate. Cancel token is set before round 2 can respond. Replay buffered messages through the same classification logic. **Pre-existing biome lint (utilities.css)** Add biome-ignore for dynamic-range-limit (emerging CSS property, Chromium 133+ / WebKit 26+) that was blocking the pre-commit hook for all committers. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
🔐 Codex Security Review
Review SummaryOverall Risk: NONE
FindingsNo concrete security, correctness, or reliability findings were identified. Notes
Generated by Codex Security Review | |
…d, reopenWorkflow, openCreateWorkflow, openTriggerInspector Replace four one-shot isVisible() reads in workflow-local-controls helpers with poll-based state resolution: - openCreateWorkflow: poll until channelList appears, clicking combobox when not yet visible, instead of a racy one-shot isVisible() check on mount - openTriggerInspector: poll until Trigger event button appears, clicking the collapsed Trigger: header when needed, instead of one-shot isVisible() during tab-switch/dialog-mount transitions - createEnabled: poll until confirmation dialog appears OR create dialog closes (narrow triggers skip the dialog entirely), then await explicit create dialog closure — removes the racy one-shot isVisible() that let execution proceed before state was determined - reopenWorkflow: await the named card visible before clicking its action button; createEnabled() now guarantees dialog closure but the card render is async These were the synchronization races Thufir identified as the root cause of the workflow E2E historical failures in CI runs 33891898259/33896008491. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
|
Review feedback at exact head P1 — status-expiry E2E bypasses the production scheduler
As a result, the test would still pass if the scheduler stopped arming or firing. The sweep already has unit coverage; this E2E should bind the production lifecycle seam, consistent with Suggested fix: install Playwright's clock before P2 — cancel-ACK assertion is tautologicalIn Please remove The workflow polling/readiness changes also look sound; they replace one-shot visibility reads with condition-based synchronization. |
The prefix locator `[data-testid^="workflow-card-"]` matched both the
container div (`workflow-card-<id>`) and the child `<p
data-testid="workflow-card-name">`, causing Playwright strict-mode to
resolve to 2 elements and fail the `toBeVisible()` assertion in
`reopenWorkflow()` — deterministic CI failure in all 3 retries.
Scope to the container via `filter({ has: getByTestId("workflow-card-name")
.filter({ hasText: name }) })`, which uniquely selects the parent div
(the only element that *contains* the name child with the right text).
The previous local 12/12 pass came from a stale build that was serving
the pre-R1 spec without the new `toBeVisible()` call; the strict-mode
violation is deterministic in CI where the DOM structure is identical.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Chessing234
left a comment
There was a problem hiding this comment.
the expire-via-e2e-bridge + fake clock approach is solid. one hang risk: the cancel-ack drain is now an unbounded loop { recv } — if the agent never acks cancel the test wedges instead of failing the old 40-iter budget. worth a timeout or iter cap around that drain.
Chessing234
left a comment
There was a problem hiding this comment.
poll helpers that click on every failed check can toggle the combobox/inspector if isVisible flickers. also the cancel-ack drain has no timeout now — if the ack never shows up that test hangs instead of failing.
Fix three independently-flaking test areas that caused the last four
mainmerges to fail CI (runs 33891898259, 33895981950, 33896008491, 33901221965).Status-expiry E2E (
profile-custom-emoji-status.spec.ts)The test seeded a status expiring 2 s ahead using wall-clock time, then raced the UI waiting for the expiry indicator to disappear. Under CI load the indicator never disappeared within the timeout.
Fix: install Playwright's fake clock at
Date.now() + 3_000(past the 2 s expiry) and call a new E2E bridge function (__BUZZ_E2E_EXPIRE_USER_STATUS_QUERIES__) that callsexpireUserStatusQuerieson the app'sQueryClient. The fakeDate.now()makes the dialog'sexpirationIsFutureguard re-evaluate tofalseon its next render; the cache sweep removes the sidebar indicator. No timers, no polling.Workflow E2Es (
workflow-local-controls.spec.ts,workflows.spec.ts)One-shot
boundingBox()reads during inspector-open transitions producednullor reversed coordinates, causing alternating assertion failures.Fix: await all three operator buttons visible +
waitForAnimationsbefore reading geometry; wrap layout assertions inexpect.pollto re-measure until stable. Inworkflows.spec.ts, await the "Trigger event" button visible before clicking it (the inspector may still be animating in after clicking "Trigger: Message Posted").Rust cancellation test (
fake_llm.rs)h.send("session/cancel", ...)is request-like; releasing the round-2 gate before the cancel ACK arrived let round 2 win the race, producingstopReason: null.Fix: drain stdout into a buffer until the cancel ACK is received, then release the gate. The cancel token is guaranteed set before round 2 can respond. Buffered messages are replayed through the same classification logic.
Pre-existing biome lint (
utilities.css)Add
biome-ignorefordynamic-range-limit(emerging CSS property, Chromium 133+ / WebKit 26+) that was blocking the pre-commit hook for all committers. Unrelated to the flake fixes.Local verification
Rust cancellation test — 25/25 iterations:
Full hook suite passed clean on push (desktop-check, desktop-typecheck, desktop-test, desktop-tauri-checks, rust-tests).