|
| 1 | +--- |
| 2 | +name: testing-cli |
| 3 | +description: Test the Detail CLI locally — run lint/tests, verify CLI behavior end-to-end, and diagnose flaky/parallel-test failures. Use when verifying CLI changes or investigating CI failures. |
| 4 | +--- |
| 5 | + |
| 6 | +# Testing the Detail CLI |
| 7 | + |
| 8 | +## Core commands |
| 9 | +```bash |
| 10 | +cargo fmt --check |
| 11 | +cargo clippy -- -D warnings |
| 12 | +cargo test # full suite (lib + integration) |
| 13 | +cargo xtask check # vendored artifacts (openapi.json, docs/HELP.md) |
| 14 | +cargo run -q -- <args> # run the CLI directly, e.g. `cargo run -q -- completions` |
| 15 | +``` |
| 16 | + |
| 17 | +## Investigating CI failures |
| 18 | +- `gh run list --limit 20` and `gh run view <run_id> --log-failed` show recent runs and failure logs. |
| 19 | +- A test can pass in PR CI but fail on the push to main — suspect flakiness/races, not the diff itself. |
| 20 | + |
| 21 | +## Flaky tests / env-var races |
| 22 | +- Rust runs tests in parallel threads, and env vars (`std::env::set_var`) are process-global. Tests that mutate env vars (e.g. `SHELL`) can race and fail intermittently with errors like `called unwrap_err() on an Ok value`. |
| 23 | +- Fix pattern: serialize via a `static LOCK: Mutex<()>` and a helper that locks, sets the var, runs the closure, and restores the original value (see `with_shell_var` in `src/commands/completions.rs` tests). |
| 24 | +- To prove a flake exists and that a fix works, stress-run with a control: |
| 25 | +```bash |
| 26 | +# control on a pre-fix worktree, then on the fixed branch |
| 27 | +git worktree add /tmp/cli-main origin/main |
| 28 | +fails=0; for i in $(seq 1 200); do cargo test --lib <module> >/dev/null 2>&1 || fails=$((fails+1)); done; echo $fails |
| 29 | +``` |
| 30 | + Expect failures on the control and 0 on the fix. If the control never fails, the result is inconclusive — increase iterations or thread count. |
| 31 | + |
| 32 | +## Merging |
| 33 | +- Add the `mergequeue` label when a PR is ready; Aviator handles the merge. |
| 34 | + |
| 35 | +## Devin Secrets Needed |
| 36 | +- None for build/lint/test. API-touching commands (auth, bugs, scans) need a `dtl_live_`/`dtl_test_` API token or PKCE login. |
0 commit comments