Keep an ordinary process's exit for its owner when it missed it

[pcarrier]

Stacked on #92. Its two commits come first here. Review the last one, "Keep an ordinary process's exit for its owner when it missed it". It goes after #92: its WAIT looks again after a CONFLICT, and this PR relies on that.

A design change in #85/#86's area. The server now keeps some final records of ordinary processes, not only detachable ones. It keeps them for the owning session alone, and only when that session missed the exit. The choices (what is kept, for whom, how many, how long) are below.

The problem

An ordinary process belongs to its spawning session. yas-client sends WAIT for its exit when the attachment that would have reported it goes first. That happens on a stream reset or drop, on a DETACH, and when a connection drops and reconnects just as the process exits. If the exit came in between, that WAIT answered NOT_FOUND, and the exit was lost:

How the owner missed the exit	What happened
(a) Its look went before the exit, so it had no binding when the exit was queued.	finish_terminal → release_record: removed from live, with nothing kept, because only a detachable process goes to finals.
(b) The exit found the owner's endpoint queue full.	The exit event was dropped, and the attachment waited forever for an exit that would never come.
(c) Its route failed (was evicted) as the exit was queued.	The adapter's Exit arm found no route and dropped the exit without a replay. #92's comment there describes this.

This is known follow-up 1 in #92, and an_owner_that_drops_its_stream_after_the_exit_still_waits_for_it flaked on it. That test failed 1 of 4 yas-server lib runs on #92's branch, with NOT_FOUND.

The change

• (c), in the adapter: dispatch_outbound records the replay of every exit it is handed, before it looks for the route and whether or not the route is there. NativeEvent::Exit now carries process_handle for that.
• (a) and (b), native: EndpointState.missed holds the finals of this endpoint's ordinary processes whose exits it missed.
  • try_queue_terminal works out whether the owner has a binding among those that take the exit.
  • WriterGuard's action is now told whether its event was dispatched, so an owner exit that is dropped before dispatch also counts as missed.
  • finish_terminal stores the final before release_record moves the catalogue. A WAIT that found the exit in flight (CONFLICT, Answer a WAIT that looks too soon with the exit, not CONFLICT #92) waits for that change, so when it looks again it finds the final.
  • watch_native checks missed after the public finals. WAIT, and an owner's ATTACH, then get the exit, exactly as for a detachable process's final.
• (b): a binding whose exit can't be queued is evicted, just as one that falls a window behind is. Its attachment fails with RESOURCE_EXHAUSTED and its client WAITs.

Design choices, for review

• Who sees them: only the owning endpoint. Other sessions still get NOT_FOUND for an ordinary process that has gone, as before. The public catalogue is unchanged.
• Which exits: only those the owner missed. An exit its route took is already in the adapter's exit replays.
• How many and how long: the newest ProcessMaxima::exit_replays() (the process generations maximum, never fewer than 64), until the endpoint shuts down. This is the same bound, and the same lifetime, as the adapter's per-session exit replays. There are no timers; the alternative was a TTL like the 5-minute detached-result one.
• Accounting: kept finals take no process slot, no generation and no buffer bytes. Their memory is the final record (argv0 and cwd included) times the bound, per session.
• Docs: processes.md and yas.md say that the spawning session's WAIT finds such an exit.

Tests

New, each failing with NotFound before the change:

• an_owner_whose_look_went_before_the_exit_still_waits_for_it (a)
• an_owner_whose_queue_was_full_as_the_exit_came_still_finds_it (b): a native endpoint whose one-event queue the output fills. It also checks the eviction.
• an_owner_whose_route_failed_as_the_exit_came_still_waits_for_it (c)

Reworked (#92's known follow-up 2):

• a_watcher_whose_queue_fills_after_the_exit_fails_alone assumed 80 lines make 80 frames. A loaded reader breaks that by taking several lines in one read.
  • Its child now writes each line only after the test has received the last one, as a FIFO tells it. It runs without stdin, so the watcher's queue holds output alone.
  • The test checks each frame (x\n at its offset), and checks that the z read after the exit evicted only the watcher's attachment (ROUTE_EVICTED). The watcher's session goes on.
• an_owner_that_drops_its_stream_after_the_exit_still_waits_for_it: its comment now says why it holds whether the detach comes before or after the exit.

Checks

• cargo fmt --check: passes. cargo clippy -p yas-server --all-targets -D warnings: clean.
• The three new tests, test B (reworked) and test A: 10/10 runs each.
• Answer a WAIT that looks too soon with the exit, not CONFLICT #92's tests a_wait_that_finds_its_own_look_still_bound_answers_once_it_goes and a_wait_right_after_a_kill_of_a_detached_process_gets_its_exit: 10/10 each.
• yas-server lib: 941/941, twice.
• client_host: 38/38.

All on a machine at load 50–290.

Fork CI: see the linked indent-com draft.

[ghost]

Reviewed 45ff11e in 8m 57s: 1 issue · Session

[ghost]

Ultimator's review of 45ff11e

Keeps an ordinary process's final for its owner when the owner missed the exit: no binding, the exit dropped before dispatch, or its route gone in the adapter, which now records every exit's replay before it looks for the route. It also evicts a binding whose exit finds its queue full. The server change checks out against the code, and the new and changed yas_process tests pass on their own. One new test is racy: under load it failed 7 of 24 parallel runs of yas_process::tests::a, and 3 of 40 parallel runs of the test alone.

1 issue: 1 bug

No issue is open since the review of 29350d9.

_{Session · @ultimator-agent review reviews again · @ultimator-agent fix fixes the open issues}

[ghost]

Reviewed 29350d9 in 1m 35s: no new issues, 1 resolved · Session

[ghost]

Ultimator's review of 29350d9

The new commit waits on evictions.notified() before counting the evictions, which closes the race in the queue-full test. On the test binary built from 29350d9, the same load that failed 7 of 24 runs at 45ff11e now passes: 24 of 24 parallel yas_process::tests::a runs and 40 of 40 runs of the test alone. Nothing new.

No issues found · 1 resolved

_{Session · @ultimator-agent review reviews again · @ultimator-agent fix fixes the open issues}