Skip to content

Open action items: AIRT-0003 follow-ups, blocked decisions, and the verification queue #77

Open

Description

@depalmar

Running tracker for work that is outstanding as of 2026-08-21, after #74 and #76 merged. Complements #8, which tracks the medium/low-confidence queue specifically.


A. Blocked on a decision, not on effort

Neither should be resolved by whoever picks this up without asking. Both are recorded in CLAUDE.md.

  • A1 — AIRT-0007 Devin now overlaps AIRT-0006 Windsurf. Cognition rebranded Windsurf's desktop app as Devin Desktop and docs.windsurf.com redirects to docs.devin.ai, so the two entries describe partly the same binary. Entry IDs are permanent, so how they split is a scoping call. Do not add rows to either until it is made.
    • Note: AIRT-0007 is the only never-verified entry, and verification_worklist.py annotates it "cloud-hosted, no endpoint paths to check" — its three claims are a browser session and two network indicators. So 50/51 is structural, not a backlog, and driving it to 51/51 depends on this decision.
  • A2 — AIRT-0008 has vendor documentation contradicting vendor source. The troubleshooting page gives the Windows CLI log directory as %TEMP%\qlog\; the vendor's own crates/chat-cli/src/util/paths.rs builds %TEMP%\amazon-q\logs on Windows and uses qlog only in the unix branch. Both are vendor sources. Settle on a live host — committing to either on documentation alone is how a responder collects nothing.
    • Amazon Q is not installed on the host used on 2026-08-21, so this needs an install first.

B. AIRT-0003 follow-ups (from #76)

  • B1 — Verify ~/.copilot/mcp-secrets/ and ~/.copilot/mcp-oauth-config/ modes and permissions. Both paths are confirmed present in the shipped package but are created lazily on first MCP server add. Confirming their runtime mode needs an authenticated Copilot session. Existence and permissions only — never contents. These hold live OAuth tokens.
  • B2 — macOS/Linux rows for the computer-use plugin. Every row added in AIRT-0003: confirm the Copilot MCP paths, catalog the computer-use plugin #76 is windows only, because the path embeds copilot-win32-x64 and the binaries are .exe. The binary contains macOS strings (safari.app, accessibility-permission prompts) suggesting the plugin ships there too. That is a lead, not a verified row — do not stamp it without a Mac.
  • B3 — Decide whether the bundled computer-use plugin warrants a detection rule. A vendor-shipped MCP server that screenshots and drives SendInput/UIAutomation is a meaningful execution surface. screen_capture: true is now set on the entry; nothing in artifacts/detections/ covers the plugin's process tree.
  • B4 — Check whether other coding agents bundle computer-use plugins. AIRT-0003 was not previously flagged for this and nobody had looked. If Cursor, Windsurf, or the Copilot IDE extension ship equivalents, the same gap exists there.

C. Verification queue

See #8 for the full list. Live counts: python artifacts/scripts/verification_worklist.py --summary

  • C1 — Unexplained MISSes from the 2026-08-21 host sweep. The query that finds real errors is a HIT and an unexplained MISS on the same host. AIRT-0003 was worked and came back confirmed, not corrected. Remaining candidates:
    • AIRT-0017 Ollama — installed beyond doubt (exe, models, id_ed25519 all HIT) yet the high-rated uninstall registry key MISSES. Unverified; the check was interrupted.
    • AIRT-0002 Cursor — %APPDATA%\Cursor\User\skills\ (medium) misses on a live Cursor install
    • AIRT-0046 Warp — ~/.warp_cli/.mcp.json misses
    • Caveat for all: many of these are created lazily on first use. Trigger the feature, re-check, then correct.
  • C2 — Two medium paths confirmed present, not yet upgraded. Note the tool version, raise confidence, log in docs/VERIFICATION.md:
    • AIRT-0011 → %APPDATA%\Claude
    • AIRT-0002 → ...\Cursor\User\globalStorage\state.vscdb

D. Longer-running

  • D1 — The decisive MSIX experiment. Install Claude Desktop's MSIX on a Windows host with no pre-existing %APPDATA%\Claude, launch once, observe where claude_desktop_config.json is created. Currently reasoned from Microsoft's documented mechanism rather than observed. It has not been run — docs/VERIFICATION.md records one prior instance of this being described back as an accomplished result.
  • D2 — macOS rows across the catalog. Nothing has been host-verified on macOS. Row-level last_verified makes the gap auditable. LM Studio has a known macOS discrepancy on record (bug-tracker #1371).
  • D3 — Apply the rest of docs/RESEARCH-2026-08-14.md. ~30 candidate rows survived adversarial verification and were deliberately not committed, mostly AIRT-0008, AIRT-0033, AIRT-0018. Treat inferred rows as unverified. The Cursor macOS target has research but no adversarial pass — single-source.
  • D4 — Port UI improvements into build_site.py. / search shortcut, active-filter chips, copy-path buttons, sticky table header. The site is generated and never hand-edited, so they go in the generator.
  • D5 — Quarterly re-verification. See docs/REVERIFICATION.md.

E. Housekeeping

  • E1 — Decide whether to keep GitHub Copilot CLI 1.0.80 installed on the verification host. npm uninstall -g @github/copilot reverses it. Kept for now so B1 remains possible.
  • E2 — Consider extending readme_counts.py to gate a second block in the root README.md. Correct stale documentation claims in README and CLAUDE.md #74 deliberately did not hardcode counts there, because the script is hardwired to artifacts/README.md and any number typed into the root file would be ungated. Two of the five bugs Correct stale documentation claims in README and CLAUDE.md #74 fixed were created exactly that way. A second gated block would let the front page carry scale safely.
    • Same applies to the convenience copies in CLAUDE.md, which went stale again within one session of being corrected.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions