TestingDoctrine.md specifies `test/harness.ts` as the probe behind every ISC, but no harness and no tests ship in the release

[tzioup]

---

LIFEOS/DOCUMENTATION/Testing/TestingDoctrine.md (v1.0.24) defines bun test as "the canonical probe behind every ISC claim" and describes a shared harness at ~/.claude/test/harness.ts in the present tense, including its size and export count. Neither the harness nor any test file is present in the public release.

Reproduced against a clean clone of main at 4e33cc8.

Reproduce

git clone --depth 1 https://github.com/danielmiessler/LifeOS /tmp/lifeos-check
cd /tmp/lifeos-check

find . -name 'harness.ts' -not -path '*/.git/*' | wc -l   # → 0
find . -name '*.test.ts' -not -path '*/.git/*' | wc -l    # → 0
find . -type d -name test  -not -path '*/.git/*' | wc -l  # → 0

# the doctrine that references them
grep -n 'harness.ts' LifeOS/install/LIFEOS/DOCUMENTATION/Testing/TestingDoctrine.md

What the doctrine states

• Line 14: "The shared harness lives at ~/.claude/test/harness.ts and exports zero-external-dep helpers (paiTestEnv, tempDir, claudeFixture, platform predicates, custom matchers). Tests live in a parallel ~/.claude/test/ tree… Coverage is corpus-based: every documented hook, skill workflow, and tool surface gets at least one test file."
• Line 32 describes it concretely: "test/harness.ts (1985 lines, 98 exports, header comment forbids external deps)".
• Line 24: "What's been missing is the invocable probe behind each ISC. This doctrine closes that gap by making bun test path/to/foo.test.ts the canonical answer."

The document contains no hedging language — grep -cE "aspiration|planned|not yet|TODO|future|will be" returns 0 — so it reads as describing shipped state, not a roadmap.

The consequence

IsaFormat.md makes ## Test Strategy name a tool: per ISC, and the doctrine makes bun test path/to/foo.test.ts the canonical value for that field. On a fresh public install, any ISC written to that contract names a probe that cannot run: there is no harness to import and no test tree to put the file in. The ISC still gets checked off — just not by an executable probe.

The related executor gap is acknowledged in the spec itself, so I'm flagging it as context rather than as part of this report: IsaFormat.md:7 and ALGORITHM/changelog.md:37 both say "The harness executor (isa run, reads ## Test Strategy) is the paired build, tracked separately." That one is documented as pending. The test/harness.ts reference is not.

Why this matters for the project

The verification story is the most-cited thing about this system — ISCs closing on real tool evidence is the core claim. A new user who follows the doctrine as written will look for test/harness.ts, not find it, and have no way to tell whether they broke their install or whether it was never shipped. That ambiguity is corrosive precisely because the surrounding docs are otherwise specific and confident.

Suggested fix (one concrete action, either direction)

Pick one and make the doc match reality:

1. Ship it — include test/harness.ts and the test/ tree in the install payload, so tool: bun test … resolves on a fresh install; or
2. Mark it — add a status line to TestingDoctrine.md in the same style already used in IsaFormat.md:7 for isa run (e.g. "the shared harness is the paired build, tracked separately — not yet in the public release"), and point ## Test Strategy at probe types that do work today (bash, curl, screenshot, eval).

Option 2 is a few lines and removes the ambiguity immediately. I'm happy to open that PR if the wording direction is agreeable.

---

Not a duplicate of #1539 / #1541 / #1543 — different subsystem, different fix.

[xg-gh-25]

The gap is real and the fix direction in Option 2 is correct — mark it, don't wait to ship it. A few observations from running test-harness-driven verification in production:

Why the harness-as-written matters: The doctrine specifies test/harness.ts with zero-external-dep helpers (paiTestEnv, tempDir, claudeFixture) — that's not incidental scaffolding, it's a CONTRACT. ISCs that name tool: bun test path/to/foo.test.ts without that harness cannot run, which means the ISC's "closed with evidence" claim is unfalsifiable by a third party. That's the corrosion you correctly identified.

The bootstrapping sequence (what we learned shipping ours):

1. Harness comes first, always — before any ISC names bun test, the harness must export the claimed primitives. We wrote ours (Python pytest, not Bun) with the same zero-dep constraint and verified EVERY helper function runs standalone (no network, no env vars, no implicit state). That took 8 days — it's not fast.

2. Test tree mirrors doc tree — ~/.claude/test/ parallel to ~/.claude/ (in your case ~/.lifeos/test/ parallel to install payload) keeps discovery trivial. Our ISCs auto-generate the test path from their own location: ISC.md → ISC.test.py.

3. The executor gap (isa run) is SEPARATE but not skippable — you can ship tests + harness WITHOUT the executor and users manually run bun test path/to/foo.test.ts. When isa run ships, it just wraps that command. Don't block harness on executor.

Suggested status line (matches IsaFormat.md:7 style):

test/harness.ts and the test tree are the paired build, tracked separately — not yet in the public release. In the meantime, use probe types that work today (bash, curl, screenshot, eval — see IsaFormat.md § Test Strategy).

Why this unblocks adoption immediately: New users can write ISCs with tool: bash -c '…' or tool: eval '…' (both executable on a fresh install), then MIGRATE to bun test when the harness ships. The ISC format itself is forward-compatible — just swap the tool line.

If you're opening the PR for Option 2, I'd be happy to review — the wording above is copy-pasteable if it fits your intent.

---

6-stage verification pipeline (ISC-format contracts + pytest harness + adversarial gate) in production since Apr 2026. Built on SwarmAI. Discussion: T-6SX: Six Self-X Properties

[danielmiessler]

Doc fixed in source: a status callout now states plainly that the harness and test tree are not in the current public payload, that releases through 7.3.6 shipped them and the single-skill layout dropped them, and that the local harness is the Phase 1 skeleton (243 lines, 18 exports). One correction to the report: the 1985/98 figures describe Bun's own harness in the comparison table, not ours, so those stay. Whether the test tree returns to the payload is a packaging decision for an upcoming release; the doctrine no longer overstates the present. Thanks for holding the verification doc to its own standard.