Skip to content

Commit c99b450

Browse files
authored
test(daemon): split the two replay test files over the size tripwire (#2455)
* test(daemon): split the two replay test files over the size tripwire No daemon test file should sit over the 1,000-line tripwire, so the size ratchet can fail on new size, not only on growth. Split the two files PR #2417 left over into sibling `<base>-<concern>.test.ts` files: - session-replay-divergence.test.ts -> suggestions, chrome-filter, overlay, capture-policy, observation (13 cases). - session-replay-runtime-maestro.test.ts -> dispatch, run-script, targeting, swipe, keyboard, run-flow, shell-artifacts (43 cases). Shared setup moves to a dedicated fixtures module per family: - session-replay-divergence.fixtures.ts holds the device-resolution and snapshot-interactor mocks and the shared reset, and re-exports the SUTs so siblings load them under the mocks. The freshness-retry `sleep` stub lives in a separate retry fixtures module the two retry siblings import first, so it cannot no-op the retry delay in siblings that never retry. - session-replay-runtime-maestro.fixtures.ts declares the Maestro device-resolution vi.mock once, so the split does not copy it per file. The diff is moves, import paths and fixture extraction. The case count under src/daemon/replay/internal/__tests__ is unchanged (13 + 43 = 56); no test is deleted or skipped, and assertions are identical. Closes #2446 * refactor(test): inline the divergence replay mocks, drop the fixtures indirection The divergence split did not need a shared fixtures module. Its only shared setup was the dispatch-resolve + interactor mock pair and a reset that legacy-snapshot-capture-fixture already provides, so routing the SUTs back through a frozen object and import-order-first fixtures bought nothing and left a load-bearing import invariant that a reordering could silently break. Each sibling now declares its own vi.mock (Vitest hoists a file's own mock above its own imports) and imports the SUT directly, matching the existing session-replay-divergence-android-occlusion.test.ts sibling. The freshness-retry sleep stub is declared only in the two siblings that exercise a retry branch, so it stays local. Maestro keeps its fixtures: the 24-line resolveTargetDevice mock cannot be copied per file (one mock per module per file forbids merging with session-replay-runtime.test.ts) and the runReplayFixture harness is genuinely shared. 56 tests unchanged.
1 parent 3bbeb61 commit c99b450

17 files changed

Lines changed: 3247 additions & 3065 deletions

src/__tests__/test-utils/android-ui-hierarchy-fixtures.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,7 @@ export function parseUiHierarchy(
121121
* node (`status_bar_container`, `status_bar_contents`, ...), unlike a default
122122
* capture. Shared across the chrome-classification tests
123123
* (`core/__tests__/snapshot-chrome-android-statusbar.test.ts`) and the replay
124-
* divergence route test (`daemon/replay/internal/__tests__/session-replay-divergence.test.ts`)
124+
* divergence route test (`daemon/replay/internal/__tests__/session-replay-divergence-chrome-filter.test.ts`)
125125
* so both exercise the exact same real screen through `walkNonRawAndroidFixture`.
126126
*/
127127
export const ANDROID_IME_CAPTURE_RAW_NODES: RawSnapshotNode[] = imeCapture;
Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
import path from 'node:path';
2+
import { beforeEach, expect, test, vi } from 'vitest';
3+
import { mkdtempForTestSync } from '../../../../__tests__/test-utils/tmp-dir.ts';
4+
import { makeAndroidSession } from '../../../../__tests__/test-utils/session-factories.ts';
5+
import { SessionStore } from '../../../session-store.ts';
6+
import { buildReplayFailureDivergence } from '../session-replay-divergence.ts';
7+
import { replayDivergenceForTest } from './replay-session-fixture.ts';
8+
import {
9+
legacyDispatchCapture,
10+
resetLegacySnapshotCapture,
11+
} from '../../../__tests__/legacy-snapshot-capture-fixture.ts';
12+
import { captureSnapshotWithInteractor } from '../../../snapshot-interactor-capture.ts';
13+
14+
vi.mock('@agent-device/device-selection/dispatch-resolve', async (importOriginal) => {
15+
const actual =
16+
await importOriginal<typeof import('@agent-device/device-selection/dispatch-resolve')>();
17+
return { ...actual, resolveTargetDevice: vi.fn() };
18+
});
19+
vi.mock('../../../snapshot-interactor-capture.ts', () => ({
20+
captureSnapshotWithInteractor: vi.fn(),
21+
}));
22+
// Stubs the Android freshness-retry delay to a no-op so the retry branch runs without
23+
// a wall-clock wait. Declared only here (and in the observation sibling) — the two
24+
// siblings that exercise a retry branch — so it cannot no-op the delay elsewhere.
25+
vi.mock('@agent-device/host-kit/retry', async (importOriginal) => {
26+
const actual = await importOriginal<typeof import('@agent-device/host-kit/retry')>();
27+
return { ...actual, sleep: vi.fn(async () => {}) };
28+
});
29+
30+
const mockDispatchCommand = legacyDispatchCapture;
31+
32+
// #1264 capture parity: the divergence capture must reach the device through the SAME
33+
// `captureSnapshot` wrapper a plain `snapshot` uses (so it inherits Android freshness +
34+
// post-action retry and can never be STALER than a snapshot), and it must build its
35+
// flags from a fixed diagnostic policy — never from the failed action's narrowing flags.
36+
37+
beforeEach(() => resetLegacySnapshotCapture(vi.mocked(captureSnapshotWithInteractor)));
38+
39+
// #1264 (capture parity, point 1): the divergence capture must go through the
40+
// SAME `captureSnapshot` wrapper as a plain `snapshot`, so it inherits Android
41+
// freshness + post-action retry. Otherwise a divergence could consume the first
42+
// stale / app-scoped dump while a plain `snapshot` retries to the fresh
43+
// full-window tree — a divergence STALER than `snapshot`. Here the session
44+
// carries an active Android freshness marker (baselineCount 20); the first
45+
// on-device dump is a stale, near-empty tree (sharp node-count drop, no
46+
// meaningful content → the `sharp-drop` retry trigger), and only the RETRIED
47+
// second dump contains the system overlay. The divergence must reflect the
48+
// retried tree. The retry delay (`sleep`) is stubbed to a no-op at the top of
49+
// this file, so the retry BRANCH runs without a real wall-clock wait.
50+
test('buildReplayFailureDivergence: routes through the freshness-retry wrapper and uses the retried fresh tree, not the first stale dump (#1264 capture parity)', async () => {
51+
const root = mkdtempForTestSync('agent-device-replay-divergence-fresh-');
52+
const sessionStore = new SessionStore(path.join(root, 'sessions'));
53+
const sessionName = 'default';
54+
const appBundleId = 'com.callstack.agentdevicelab';
55+
const session = makeAndroidSession(sessionName, { appBundleId });
56+
// Active freshness marker: a navigation-sensitive action just ran, and the
57+
// pre-action baseline had 20 nodes, so a near-empty next dump is suspicious.
58+
session.androidSnapshotFreshness = {
59+
action: 'press',
60+
markedAt: Date.now(),
61+
baselineCount: 20,
62+
routeComparable: false,
63+
};
64+
sessionStore.set(sessionName, session);
65+
66+
// Capture 1: stale, near-empty dump (a single bare view — no hittable/label/
67+
// id) → `sharp-drop` vs the 20-node baseline → triggers a retry.
68+
const staleDump = {
69+
nodes: [
70+
{
71+
index: 0,
72+
type: 'android.view.View',
73+
bundleId: appBundleId,
74+
rect: { x: 0, y: 0, width: 10, height: 10 },
75+
},
76+
],
77+
truncated: false,
78+
backend: 'android',
79+
};
80+
// Capture 2: the fresh full-window tree, holding the app control AND the
81+
// separate-window system overlay's dismiss target.
82+
const freshDump = {
83+
nodes: [
84+
{
85+
index: 0,
86+
type: 'android.widget.Button',
87+
bundleId: appBundleId,
88+
label: 'App control',
89+
identifier: `${appBundleId}:id/control`,
90+
rect: { x: 20, y: 100, width: 200, height: 44 },
91+
hittable: true,
92+
},
93+
{
94+
index: 1,
95+
type: 'android.widget.ImageButton',
96+
bundleId: 'com.android.systemui',
97+
identifier: 'com.android.systemui:id/volume_new_ringer_active_icon_container',
98+
label: 'Ringer volume',
99+
rect: { x: 300, y: 300, width: 44, height: 44 },
100+
hittable: true,
101+
},
102+
],
103+
truncated: false,
104+
backend: 'android',
105+
};
106+
mockDispatchCommand.mockReset();
107+
mockDispatchCommand.mockResolvedValueOnce(staleDump).mockResolvedValueOnce(freshDump);
108+
109+
const action = {
110+
ts: 0,
111+
command: 'press',
112+
positionals: ['label="App control"'],
113+
flags: {},
114+
result: { selectorChain: ['label="App control"'] },
115+
};
116+
const divergence = await buildReplayFailureDivergence({
117+
error: { code: 'COMMAND_FAILED', message: 'not hittable' },
118+
action,
119+
index: 0,
120+
sourcePath: path.join(root, 'flow.ad'),
121+
sourceLine: 1,
122+
...replayDivergenceForTest(sessionStore, sessionName),
123+
logPath: path.join(root, 'daemon.log'),
124+
responseLevel: 'default',
125+
planActions: [action],
126+
planDigest: 'test-plan-digest',
127+
});
128+
129+
// The freshness wrapper retried past the stale dump (2 on-device captures).
130+
expect(mockDispatchCommand).toHaveBeenCalledTimes(2);
131+
132+
expect(divergence.screen.state).toBe('available');
133+
const screen = divergence.screen as Extract<typeof divergence.screen, { state: 'available' }>;
134+
// The overlay only exists in the RETRIED capture, so its presence proves the
135+
// divergence used the fresh tree — parity with what a plain `snapshot` sees.
136+
expect(screen.refs.some((ref) => ref.label === 'Ringer volume')).toBe(true);
137+
expect(screen.refs.some((ref) => ref.label === 'App control')).toBe(true);
138+
});
139+
140+
// #1264 (clean flags policy, point 2): a failed `snapshot --raw`/scoped/`-d`
141+
// action must never narrow the DIAGNOSTIC divergence tree. The divergence
142+
// capture builds its flags from a fixed policy (full-window, non-raw, default
143+
// depth), NOT from the failed action's flags — so `snapshotRaw`/`snapshotScope`/
144+
// `snapshotDepth` on the action do not reach the capture. This inspects the
145+
// context handed to the snapshot dispatch and asserts those narrowing flags are
146+
// dropped while the interactive-only policy is still applied.
147+
test('buildReplayFailureDivergence: divergence capture drops the action snapshotRaw/scope/depth flags (#1264 clean flags policy)', async () => {
148+
const root = mkdtempForTestSync('agent-device-replay-divergence-flags-');
149+
const sessionStore = new SessionStore(path.join(root, 'sessions'));
150+
const sessionName = 'default';
151+
const appBundleId = 'com.callstack.agentdevicelab';
152+
sessionStore.set(sessionName, makeAndroidSession(sessionName, { appBundleId }));
153+
154+
mockDispatchCommand.mockReset();
155+
mockDispatchCommand.mockResolvedValue({
156+
nodes: [
157+
{
158+
index: 0,
159+
type: 'android.widget.Button',
160+
bundleId: appBundleId,
161+
label: 'Submit',
162+
identifier: `${appBundleId}:id/submit`,
163+
rect: { x: 20, y: 100, width: 200, height: 44 },
164+
hittable: true,
165+
},
166+
],
167+
truncated: false,
168+
backend: 'android',
169+
});
170+
171+
// A failed action that itself requested a raw, ref-scoped, depth-limited
172+
// snapshot — none of which may reshape the divergence diagnostic tree.
173+
const action = {
174+
ts: 0,
175+
command: 'press',
176+
positionals: ['label="Submit"'],
177+
flags: { snapshotRaw: true, snapshotScope: '@e5', snapshotDepth: 2 },
178+
result: { selectorChain: ['label="Submit"'] },
179+
};
180+
await buildReplayFailureDivergence({
181+
error: { code: 'COMMAND_FAILED', message: 'not hittable' },
182+
action,
183+
index: 0,
184+
sourcePath: path.join(root, 'flow.ad'),
185+
sourceLine: 1,
186+
...replayDivergenceForTest(sessionStore, sessionName),
187+
logPath: path.join(root, 'daemon.log'),
188+
responseLevel: 'default',
189+
planActions: [action],
190+
planDigest: 'test-plan-digest',
191+
});
192+
193+
expect(mockDispatchCommand).toHaveBeenCalled();
194+
const context = mockDispatchCommand.mock.calls[0]?.[4] as
195+
| {
196+
snapshotRaw?: boolean;
197+
snapshotScope?: string;
198+
snapshotDepth?: number;
199+
snapshotInteractiveOnly?: boolean;
200+
}
201+
| undefined;
202+
// The action's narrowing flags are stripped by the fixed divergence policy.
203+
expect(context?.snapshotRaw).not.toBe(true);
204+
expect(context?.snapshotScope).toBeUndefined();
205+
expect(context?.snapshotDepth).toBeUndefined();
206+
// The interactive-only policy (press → interactive) is still applied.
207+
expect(context?.snapshotInteractiveOnly).toBe(true);
208+
});

0 commit comments

Comments
 (0)