Skip to content

Flaky: learning::startup::learning_subscriber_fires_with_no_channel_configured fails under full-suite coverage run (#5003 test isolation) #5076

Description

@senamakel

Summary

openhuman::learning::startup::tests::learning_subscriber_fires_with_no_channel_configured (added in #5003 / #5025) fails deterministically in the Rust Core Coverage (cargo-llvm-cov) CI job whenever the coverage module filter widens to the whole openhuman suite (i.e. any PR that touches src/openhuman/mod.rs), but passes in isolation and when only the learning module runs.

Observed failure:

---- openhuman::learning::startup::tests::learning_subscriber_fires_with_no_channel_configured stdout ----
thread '...' panicked at src/openhuman/learning/startup.rs:291:9:
assertion `left == right` failed: email-signature subscriber must push the parsed
identity candidates with no channel configured anywhere (#5003)
  left: 0
 right: 4

First surfaced on PR #5073 (an unrelated revert), which is the first PR since #5025 landed to edit src/openhuman/mod.rs and therefore the first to run the full openhuman suite in one process under coverage.

Root cause

The test publishes a single DocumentCanonicalized event onto the process-wide global event bus and asserts that the EmailSignatureSubscriber pushed its parsed candidates into the shared, bounded candidate::global() ring buffer (candidate.rs doc: "Tests should construct their own buffer via Buffer::new").

Under the full-suite coverage run — thousands of tests in one instrumented process — that global bus is under heavy concurrent load, so this test's event/candidates are lost before the assertion reads them (tokio broadcast lag and/or ring eviction). It is a test-isolation defect, not a product bug: the production wiring path is unaffected.

Notes from investigation (on PR #5073)

  • GLOBAL_BUS is a stable OnceLock (created once, never replaced), and EMAIL_SIG_HANDLE is only ever touched by the two learning::startup tests (both call init_global first), so the subscriber is not left unregistered — this is a delivery/eviction problem, not a registration one.
  • Re-publishing the event on every poll tick for 4s did not clear it in CI → sustained starvation under load, not a transient drop.
  • Nondeterministic locally: running the exact broad filter on a many-core dev box surfaces a rotating set of 1–5 failures across learning, model_council, and memory — all the same "assert on a shared global under parallel load" shape. CI's lower core count makes learning the consistent casualty.

Suggested fix

Give the EmailSignatureSubscriber an injectable bus + candidate buffer (or otherwise let the test run the subscriber on a dedicated, non-global EventBus/Buffer::new) so the assertion no longer depends on process-wide event traffic. The sibling model_council / memory tests that share candidate::global() (e.g. reflection_tests) likely want the same treatment.

Impact

Blocks the coverage gate on any PR that edits src/openhuman/mod.rs. Pure test-infrastructure flake — no product behaviour affected.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions