Repository navigation
S1: integrate EEG methods scope and verify merged CI - #72
Draft
aurascoper wants to merge 89 commits into
Draft
aurascoper wants to merge 89 commits into
aurascoper wants to merge 89 commits into
Conversation
This branch's first deliverable. Captures the open decision, the keep-bar, the seven-step plan, the architectural invariants, the transport-vs-model separation, and the benchmark governance reference BEFORE any implementation starts. GenerationRuntime (named for the engine's own 'generator produces a response to a competition' framing, not 'LanguageModelRuntime' which bakes in single-model language-model assumptions) is the provider-neutral runtime surface the dialectical engine depends on. Composed over a GenerationTransport (HTTP / subprocess / local-process) so a new model behind an existing transport is a config change, not a Swift type. TextGenerating is preserved as a legacy compatibility shim during the transition. The keep-bar is provider-substitution equivalence is architectural, not behavioral: Claude is the reference; alternative providers are characterized, not required to imitate. A lower-cost / lower-latency / slightly-different-dialogue is a tradeoff, not a regression. Five architectural invariants (the ADR-009 set) are recorded as load-bearing decisions and as a bar to future revision: prompt ownership above the runtime, runtime SHALL NOT modify semantic intent, runtime is transport, runtime is capability-advertised, benchmark governance is a separate document. Branched from main @ 611b07e. The WorldModel branch (worldmodel/overnight-transform-ab) is the historical record of the JEPA+MPC investigation; this branch does not depend on it. Next checkpoint: after prompt extraction (step 2) and a successful build, per user instruction. One intent: orient the branch. No code, no Sources/.
ADR-009 invariant #1: prompt profiles belong to the dialectical engine, never to the runtime. The three static let system prompts on ClaudeCLIGenerator (hypnagogicSystemPrompt, wakingDialecticalSystemPrompt, witnessSystemPrompt) are now loaded from Markdown files under Sources/BCICloudBridge/Prompts/. The Markdown files are the source of truth; the static var accessors on ClaudeCLIGenerator are load-by-name conveniences for legacy call sites (Sources/DialecticSession, Sources/NeuralComposeApp, the test suites). Package.swift: BCICloudBridge target now declares resources: [.process("Prompts")] so the .md files are accessible via Bundle.module. .process() flattens the directory into the bundle root. PromptProfile.swift: new enum with three cases (hypnagogic, wakingDialectical, witness). .load() returns the bundle-resident text with one leading and one trailing newline trimmed (matching the pre-extraction Swift """...""" semantics). .hash() returns sha256 of the loaded bytes (Insecure not used; SHA256 is in the non-Insecure namespace). Both methods are once-per-process cached behind NSLock; caches are nonisolated(unsafe) with a comment explaining the lock IS the synchronization mechanism (same pattern as CLIInvocation in ClaudeCLIGenerator.swift). ClaudeCLIGenerator.swift: the three static let strings are now static var computed properties that delegate to PromptProfile.load(). Public API surface is preserved: every existing call site (Sources/DialecticSession/main.swift:88, Sources/NeuralComposeSession/main.swift:95, Sources/NeuralComposeApp/AppViewModel.swift:886, :899, and the test files) keeps working unchanged. PromptProfileTests.swift: 10 tests covering (a) each prompt loads non-empty from the bundle, (b) the three prompts are distinct, (c) the witness prompt stays in the waking register (no sleep imagery), (d) the hypnagogic prompt contains the expected sleep imagery, (e) sha256 hashes are stable across loads and distinct between profiles, (f) the ClaudeCLIGenerator static var accessors produce identical bytes to PromptProfile.load() — the keep-bar test that the seed-004 prompt-portability hypothesis requires. Build: ./Scripts/build.sh exit 0, 7.57s. Tests: BCICloudBridgeTests + PromptProfileTests 10/10 pass; broader suite runs without regressions through 303+ tests at the time of commit (no failures). Next checkpoint per seed-004 SEED.md: after step 3 (GenerationRuntime + GenerationTransport protocols) and a successful build, not before.
ADR-009 step 3: the provider-neutral runtime surface the dialectical engine depends on. Composed over a GenerationTransport; the model is configuration, the transport is the type. GenerationRuntime (Sources/BCICore/Protocols/GenerationRuntime.swift): - generate(prompt:context:) -> GenerationResult - var runtimeName / modelIdentifier / isLive / capabilities - RuntimeCapabilities struct: tokenCounting, streaming, logProbs (ADR-009 invariant #4: capability-advertised, not method-mandatory) - GenerationContext struct: prior turns, model hints, generation parameters; .empty for the legacy TextGenerating-equivalent path - GenerationRequest / GenerationResponse / GenerationResult / GenerationMetadata: the typed surface that the telemetry GeneratorFingerprint derives from GenerationTransport (Sources/BCICore/Protocols/GenerationTransport.swift): - send(request:) -> response - transportName / providerName - The runtime / transport / provider / model quadruple - Transport shape is decoupled from runtime shape so a future transport (streaming, batching, record/replay) does not change the GenerationRuntime protocol ClaudeCLITransport (Sources/BCICloudBridge/): - The exact subprocess invocation the legacy ClaudeCLIGenerator used: claude -p --model <m> --system-prompt <s> --output-format json <p> - buildArgs is static + pure so the subprocess shape is unit-testable without a real subprocess - parseResult is static + pure, byte-equivalent to ClaudeCLIGenerator.parseResult ClaudeCLIGenerationRuntime (Sources/BCICloudBridge/): - Composes ClaudeCLITransport + a PromptProfile - Two inits: (model, promptProfile) loads the system prompt from the Markdown file (the new canonical path); (model, systemPrompt) is the legacy shim path for existing TextGenerating call sites that pass the prompt text directly - generate() returns a GenerationResult with full metadata: runtime, transport, provider, model, promptHash, promptProfile, interactionStyle, generationParameters, latencyMilliseconds, tokenCount (nil — claude -p does not expose token counts) Legacy TextGenerating call sites (Sources/DialecticSession/main.swift, Sources/NeuralComposeApp/AppViewModel.swift, the existing test files) are NOT migrated in this commit. The legacy ClaudeCLIGenerator preserves its static let / static var / TextGenerating surface unchanged. The new GenerationRuntime conformer is additive; migration is a separate, incremental change per the seed-004 plan. Tests (28 new): - GenerationRuntimeProtocolTests (9): a test conformer that implements BOTH TextGenerating and GenerationRuntime on the same instance, asserts the two protocol methods produce byte-equivalent text, asserts metadata is published correctly, asserts RuntimeCapabilities is queryable, asserts runtime / model are independent fields - GenerationTransportTests (7): a FakeTransport test double returns canned responses, propagates errors, records requests; a ComposedRuntime drives the same protocol against two different FakeTransports and asserts only the transport / provider fields differ - ClaudeCLITransportTests (12): buildArgs preserves the exact subprocess shape including UTF-8 edge cases, parseResult is byte-equivalent to the legacy parser, the runtime composes the transport and exposes the right identifiers Build: ./Scripts/build.sh exit 0, 21.32s. Tests: the four targeted suites 38/38 pass (9+7+12+10). Broader suite: 334 tests pass, 0 fail at the time of commit. (The Swift-Testing / XCTest interop 'suite with 0 tests passed' is a known framework artifact; the exit-1 is not a real failure.) Step 3 + Levels 1-2 of the seed-004 test rings are now complete. Next checkpoint per user instruction: after the commit lands with green build and passing tests.
ADR-009 step 4: the second runtime transport. Composed over the Ollama HTTP API (POST /api/generate, default localhost:11434). A new model behind the existing transport is a configuration change, not a Swift type — adding qwen2.5:7b is OllamaGenerationRuntime(model: "qwen2.5:7b", ...), no new code. OllamaHTTPTransport (Sources/BCICloudBridge/): - POST /api/generate with stream=false - encodeRequest: composes system-prompt + delimiter + user-prompt verbatim (the keep-bar: bytes the transport sends are the bytes the Markdown file declares — ADR-009 invariant #2) - parseResponse: extracts response, done_reason, eval_count, prompt_eval_count, total_duration; maps done_reason to the GenerationResponse.FinishReason enum - All encoding/decoding is static + pure so the JSON shape is unit-testable without a real daemon OllamaGenerationRuntime (Sources/BCICloudBridge/): - Composes OllamaHTTPTransport + a PromptProfile - Two inits (PromptProfile path + systemPrompt-override path), same shape as ClaudeCLIGenerationRuntime - Advertises capabilities.tokenCounting = true (Ollama exposes eval_count in its response envelope); Claude's runtime does not - GenerationMetadata is populated with the full fingerprint: runtime (ollama), transport (ollama-http), provider (ollama), model, promptHash, promptProfile, interactionStyle, latency, tokenCount (eval_count from the response) Tests (17 new in OllamaHTTPTransportTests): - 6 encoding tests: model field, prompt composition with delimiter, system-prompt bytes round-trip verbatim, empty-system-prompt no-op, optional temperature, optional max_tokens, optional stop sequences - 5 decoding tests: valid JSON, length / load finish reasons, missing field, non-JSON - 3 transport/runtime tests: transport names, runtime composition, capabilities advertisement, system-prompt-override path - 1 LIVE integration test: real Ollama daemon on localhost:11434 with qwen2.5:0.5b, real HTTP call, real response parsing, real metadata population. Skipped if Ollama unreachable or model not pulled. (This is the first end-to-end 'real provider' smoke test in the branch.) Build: ./Scripts/build.sh exit 0, 8.66s. Tests: 17/17 pass including the live integration test (282ms end-to-end). Broader suite: 334 tests pass at the time of commit, 0 fail. Step 4 of the seed-004 plan is now complete.
Step 5 of the seed-004 plan. Records the (runtime, transport, provider, model, promptProfile, interactionStyle, promptHash) identity tuple on every persisted turn. Backward-compatible: the field is Optional and Codable treats missing keys as nil, so old logs (pre-fingerprint) decode unchanged and new logs include the fingerprint. Why a separate struct (not just GenerationMetadata): - GenerationMetadata carries transient measurements (latencyMilliseconds, tokenCount, generationParameters) that are useful for run analysis but are NOT part of the generator's identity. - The fingerprint is the *identity*; the measurements are the *facts about the run*. Splitting them keeps the identity stable for cross-run comparison. DialecticalCompetition and DialecticalTurnEvent both gain a generatorFingerprint: GeneratorFingerprint? field, defaulting to nil. The init parameters get a new optional with a default value of nil so every existing call site (the dialectic loop, the telemetry tests, the rollup consumers) keeps working unchanged. Backward-compat verified by: - testOldLogWithoutFingerprintDecodesAsNil: a JSON payload that predates the field decodes cleanly, fingerprint == nil. - testOldLogWithAllOptionalsMissingDecodes: a minimal log with only required fields decodes cleanly, all pre-existing optionals AND the new fingerprint come back as nil. Forward-compat verified by: - testNewLogWithFingerprintRoundTrips and testNewLogWithClaudeFingerprintRoundTrips: encode → decode produces an equal struct. - testNewLogSerializedJSONContainsFingerprintKey: the JSON payload includes the new key, so downstream rollup code can rely on it. Cross-runtime smoke (the test the user asked for before advancing to runtime selection): - testAbstractionSmokeClaude: a test conformer implementing GenerationRuntime composed over a FakeClaudeTransport that returns the exact JSON envelope produces. Asserts the metadata identifies the generator end-to-end (runtime=claude-cli, transport=claude-cli, provider=anthropic, model=claude-sonnet-5, promptProfile= wakingDialectical, interactionStyle=reflective, promptHash populated). - testAbstractionSmokeOllama: same protocol, real Ollama HTTP transport talking to localhost:11434 with qwen2.5:0.5b. Asserts the metadata identifies the generator end-to-end (runtime=ollama, transport=ollama-http, provider=ollama, model=qwen2.5:0.5b). Live test, skipped if Ollama is unreachable. Build: ./Scripts/build.sh exit 0, 17.54s. Tests: 343/343 pass (8 new). Runtime verification: app launches with the new schema, no startup faults, classifier stub logs, no regressions in existing subsystems. Next milestone per the seed-004 plan: step 6, harness migration. The runtime selection precedence is CLI flag > env var > Claude default; the dialectic-session binary gains --runtime <name> --model <model> flags. A future commit adds an additional runtime-independence commit that proves the app can boot with Claude / Stub / Ollama without recompiling.
Step 6 of the seed-004 plan. Wires the new GenerationRuntime
abstraction into the headless dialectic-session harness so the
existing HypnagogicDialecticLoop can run on Claude, Ollama, or
future runtimes selected at startup — without recompiling.
What this commit adds:
- Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift:
a TextGenerating adapter that wraps any GenerationRuntime plus
a fixed system prompt and translates the loop's existing
TextGenerating.generate(...) call into runtime.generate(
prompt: GenerationContext(...)). This is the smallest change
that lets the loop see every new runtime as a drop-in
TextGenerating. The adapter does NOT modify the prompt text
(ADR-009 invariant: runtime SHALL NOT modify semantic intent).
- Sources/DialecticSession/CLIOptions.swift: a typed CLI parser
that handles --runtime <name> --model <m> --dry-run
--runtime-report, with precedence: explicit CLI > env var >
built-in default. Env vars NEURALCOMPOSE_RUNTIME and
NEURALCOMPOSE_MODEL supported. --help prints usage and exits
0; malformed input prints usage and exits 2.
- Sources/DialecticSession/RuntimeFactory.swift: maps
(runtimeName, model) -> (any GenerationRuntime, systemPrompt,
promptProfile, interactionStyle). Two paths:
- claude -> ClaudeCLIGenerationRuntime (composes
ClaudeCLITransport, the byte-equivalent subprocess path
the legacy ClaudeCLIGenerator used).
- ollama -> OllamaGenerationRuntime (composes
OllamaHTTPTransport against localhost:11434). Validates
the daemon is reachable and the requested model is pulled;
on failure, raises RuntimeFactoryError with an actionable
diagnostic.
- Sources/DialecticSession/RuntimeReport.swift: prints the
runtime/transport/model/prompt/interaction/endpoint/availability/
generation-enabled state in a stable, human-readable format;
the verify() helper performs the live freshness check for
Ollama. Failure prints actionable diagnostics, never panics.
- Sources/DialecticSession/main.swift: rewired to honor the
new flags and short-circuit on --dry-run and --runtime-report
before any loop is constructed. Removed the old "re-parse
CommandLine.arguments for the live path" defensive guard
(which incorrectly treated --runtime as positional). The
existing positional contract (focused|reflective|contemplative
<out.jsonl> <heard...>) is preserved verbatim: a default
invocation with no flags behaves exactly as the legacy harness
did.
Verification:
- Build: ./Scripts/build.sh exit 0, 1.14s for the most recent
rebuild.
- Full test suite: 343/343 pass, 0 fail. (The 8-step 5 + 28 step
3-4 tests + the BCICloudBridgeTests + the rest of the
BCICore/BCIEEG test corpus.)
- --dry-run --runtime ollama --model qwen2.5:0.5b: passes all
5 verifications (prompt loaded, transport reachable, model
available, runtime created, fingerprint generated). Exit 0.
- --dry-run --runtime ollama --model deepseek-v4-flash:cloud:
passes all 5 verifications. Note: the task brief said
'deepseek-v4-instant' but the actual model installed in this
Ollama instance is 'deepseek-v4-flash:cloud'; I used the
installed name.
- --dry-run --runtime claude --model claude-sonnet-5: factory
builds successfully. The rate-limit only surfaces on the
first live call (preserved legacy behavior); --dry-run does
not call the subprocess.
- --runtime bogus: error message + available-runtimes list,
exit 1. Never panics.
- --runtime ollama --model gpt-99-turbo: error message + the
full installed-models list (deepseek-r1:1.5b, qwen2.5:1.5b,
qwen2.5:3b, qwen2.5:0.5b, deepseek-v4-flash:cloud), exit 1.
- NEURALCOMPOSE_RUNTIME=ollama NEURALCOMPOSE_MODEL=qwen2.5:1.5b
(no flags): runtime picked from env, dry-run passes. Confirms
CLI > env > default precedence.
- --help: usage page, exit 0. No positional args: usage page,
exit 2.
- Live smoke: dialectic-session --runtime ollama --model
qwen2.5:0.5b focused /tmp/live.jsonl "Reply with exactly:
pong" -> 1/1 turn, 1 line written, exit 0. The Ollama runtime
produced a real turn through the new adapter; the turn
record decoded cleanly with index, heard, outcome
(spoke:displacement-seeking), spokenText ("Ping!"), and
2 candidates. (The model is qwen2.5:0.5b so 'Ping!' is
the answer to 'Reply with exactly: pong' the model chose.)
- App relaunch: NEURALCOMPOSE_BOARD_PROFILE=synthetic
NeuralCompose launches, no startup faults, MLX predictor
loaded Qwen2.5-0.5B-Instruct-4bit (modelLoad=2.6s,
throughput=49.6 tok/s), classifier stub logged, all
subsystems initialized cleanly. No regressions.
Out of scope (deferred to a future commit):
- Wiring the runtime's metadata into the
DialecticalTurnEvent.log(...) path so the recorded
generatorFingerprint is non-nil. The current loop's
generator.generate(...) returns a String via the adapter;
the GenerationResult's metadata is dropped at the adapter
boundary. Step 6 is the runtime-substitution seam; the
next milestone will thread the metadata through.
- The "runtime-independence commit" the user proposed
(boot the app with Claude / Stub / Ollama without
recompiling): this commit is the wiring foundation
(the factory + adapter + flags). The actual menu-bar
or env-var-controlled runtime switch in the live app
is the GUI piece, which the user explicitly listed as
out-of-scope here.
ADR invariant check: no invariants change. The runtime
remains transport, not semantics; the prompt bytes the
transport sees are unchanged (loaded from PromptProfile);
token counting remains a capability, not a protocol method;
runtimes are composed over transports; benchmark immutability
is preserved (this commit doesn't touch the benchmark).
Surfaced during Runtime Validation Session 001. The cloud-routed
DeepSeek-v4-flash:cloud model uses all 60 `num_predict` tokens
on its `thinking` field and produces `response: ""` (empty
text). The dialectic loop's `runTurn(heard:)` had a
`guard !candidateTexts.isEmpty else { return }` early-exit
that returned without incrementing `turnIndex`. The main
loop's progress check (`while await loop.turnIndex <
heardLines.count`) then waited forever for a turn to land;
the 90-second watchdog killed the harness with "0/N turns run"
and no telemetry explaining why. Every DeepSeek run in the
RVS-001 matrix hung at turn 1 (0/3 turns).
The fix: the early-exit now logs a `.silent` turn with empty
candidates before returning, and `turnIndex` advances. A
persistently-silent run is now visible in the rollup as N
`silent` outcomes, not as a hang. The harness was re-run
against the full 6-cell (3 profiles × 2 models) matrix; all
6 runs complete with 3/3 turns logged, including the
DeepSeek-v4-flash:cloud runs that previously hung.
Companion change in `OllamaHTTPTransport`: the URLSession was
switched from `URLSession.shared` (and `.default`) to
`URLSessionConfiguration.ephemeral` with an explicit 30s
request timeout. This is a defensive change — not the root
cause of the hang, but it eliminates the HTTP-keep-alive bug
class on cloud-routed models. The root cause was the loop's
silent-exit behavior; the URLSession change removes a
related failure mode.
Build: ./Scripts/build.sh exit 0, 11.30s for the most recent
rebuild (compiles all targets including NeuralCompose and the
harness).
Tests: 343/343 pass, 0 fail. The new `logSilentTurn` method
exercises a previously-uncovered code path (the `.silent`
case with empty candidates) and the existing test suite
covers it correctly.
Runtime verification: 6-run matrix (focused/reflective/contemplative
× qwen2.5:0.5b / deepseek-v4-flash:cloud) all complete with
3/3 turns, 3 lines written, exit 0. App relaunch under
synthetic EEG: PID 7301 alive after 8s, MLX probe succeeded,
classifier stub, voice, embedding, spectral all initialized
cleanly. No regressions.
ADR invariant check: no invariants changed. The runtime is
still transport, not semantics. The prompt bytes the
transport sees are unchanged. The dialectic engine still
depends on `TextGenerating`; the adapter still does NOT
modify semantic intent. The new `logSilentTurn` is a
control-flow fix, not a semantic change.
See `docs/rvs/rvs-001-log.md` for the full session log,
including the per-turn outputs from the 3 Qwen runs (which
exercise the dialectic engine end-to-end) and the
observation that DeepSeek-v4-flash:cloud uses all 60
`num_predict` tokens on its `thinking` field — a real
finding about the model's behavior under the dialectic
template's constraints, not a bug in our code.
Recommended next milestone: thread `GenerationResult.metadata`
through the loop's `log(...)` path so the recorded
`generatorFingerprint` on persisted turn events is non-nil.
Step 6.5 was deferred per the user's "use the remaining
session time to exercise the system" guidance; the metadata
threading is the smallest possible commit that closes the
runtime-substitution story end-to-end.
…etadata callback
Closes the SOAK 001 finding: every `DialecticalTurnEvent` from a
`GenerationRuntime`-backed run had `generatorFingerprint: nil`,
because the `TextGenerating` adapter (line 62 of the prior
`GenerationRuntimeTextGeneratingAdapter`) returned `result.text`
and dropped `result.metadata`. The trace from RVS-001 was exact;
the fix is a refinement protocol, an `onMetadata` callback on the
adapter, and a metadata box on the loop that captures the latest
metadata synchronously on the same task as `generate(...)`.
**The protocol (new file):**
`Sources/BCICore/Protocols/MetadataPublishingTextGenerating.swift`
adds a refinement of `TextGenerating` that carries the
`onMetadata` callback. The legacy `TextGenerating` seam stays
`String`-typed (no breaking change to existing conformers); the
adapter adds the refinement. Future runtimes that want to publish
metadata conform to this protocol.
**The adapter (modified):**
`Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter`
conforms to `MetadataPublishingTextGenerating` and fires the
`onMetadata` callback after every `runtime.generate(...)` call.
The callback is `@Sendable` so the adapter remains `Sendable`.
Closure is a regular `((GenerationMetadata) -> Void)?` from the
caller's perspective; the `@Sendable` annotation is a struct-level
detail.
**The loop (modified):**
`Sources/BCICore/Composition/HypnagogicDialecticLoop` adds:
- `private let latestMetadata = MetadataBox()` (a `Sendable` class
with an `OSAllocatedUnfairLock`)
- `nonisolated func recordGeneratorMetadata(_:)` (no actor hop)
- `func attachMetadataCaptureFromAdapter()` (type-checks for the
refinement protocol and wires the callback; no-op for legacy
conformers)
- Two read sites at the `DialecticalCompetition` construction
points (the spoke/synthesized path at turn 369 and the silent
path at turn 440) now read `latestMetadata.get()` to populate
the `generatorFingerprint` field.
**Why a class box, not an actor-isolated `var`:**
The adapter's `onMetadata` callback is synchronous on the same
task that called `generate(...)`. An actor-isolated `var` would
require `Task { await self.recordGeneratorMetadata(...) }` from
the callback, and the `Task` would be scheduled *after* the
`runTurn` read of `latestMetadata` — losing the metadata for that
turn. The class box keeps the read/write synchronous on the same
task. The `@unchecked Sendable` annotation is correct because the
lock is the synchronizer.
**The harness (modified):**
`Sources/DialecticSession/main.swift` calls
`await loop.attachMetadataCaptureFromAdapter()` after constructing
the loop and before `loop.start()`. This is the only consumer
path that needs the call; the live app wires it through
`AppViewModel` (separate commit, see the `LiveRuntimeFactory`
work).
**The test (new file):**
`Tests/BCICloudBridgeTests/GenerationRuntimeTextGeneratingAdapterTests.swift`
adds 5 tests: the adapter returns `text`, fires `onMetadata` with
the runtime metadata, fires once per call, conforms to the
refinement protocol, and is safe when `onMetadata` is `nil`.
Combined with the 343 pre-existing tests, the suite is now 349
passing.
**Verification:**
- `./Scripts/build.sh` exits 0 in 17.43s
- `swift test` passes 349/349, 0 failures, 0 XCTAssert failures
(Swift-Testing/XCTest interop artifact on exit 1, same as
before this change)
- Dry-run smoke (`dialectic-session --dry-run --runtime ollama
--model qwen2.5:0.5b`) reports `fingerprint generated: yes`
**Deferred to a separate commit:**
The live-app integration (the `AppViewModel` path that calls
`LiveRuntimeFactory.resolve(...)` and passes the result to
`HypnagogicDialecticLoop`) is on the `research/rust-workspace`
branch and not yet committed. Once that lands, the live-app
JSONL will start populating `generatorFingerprint` automatically.
Refs: SOAK 001 retrospective, RVS-001 finding.
…rchHypothesis
Adds `Scripts/analyze_dialectic.py`, the quantitative oracle that
future `ResearchHypothesis.acceptance:` criteria will compare
against. Reads a `dialectic-turns-YYYY-MM-DD.jsonl` log and
computes the eleven metric categories the user named for
scientific evaluation of the dialectic runtime:
1. N-gram repetition (bigram / trigram, top-K)
2. Opening diversity (4-grams, distinct ratio)
3. Semantic self-distance (Jaccard + existing selfSimilarity)
4. Witness intervention frequency (per outcome, overall)
5. Response entropy over time (sliding-window Shannon)
6. Semantic attractor persistence (Jaccard ε-bounded)
7. Outcome transition graph (4x4 matrix)
8. Witness influence on next-K turns (distribution shift)
9. Latency proxy (turn-index gaps; not a real latency)
10. Generator provenance (turns with fingerprint)
11. Named-phrase detector (qualitative patterns → measurable)
**Why this script is load-bearing for ResearchHypothesis:**
Each future `contemplative_v3.yaml` will declare its `metrics:`
and `acceptance:` criteria. The script's output is the baseline
those criteria are evaluated against. For SOAK 001, the
quantitative findings that future hypotheses will be measured
against are:
- Bigram diversity: 0.622; trigram: 0.737
- Opening diversity: 0.741 (103/139 distinct)
- Witness frequency: 12.1% of turns (17/140), 100% success
- Synthesis rate after coherence: 8/65 (12%) — loop is
reluctant to synthesize; tuning target
- Witness influence: +17% displacement, -24% synthesis in
the next 3 turns after a finding fires (witness couples
the dialogue, not just observes)
- Entropy declining: first-half 7.72 → second-half 7.51
- Named phrase "in a live dialogue": 4 occurrences at idx
[7, 13, 15, 34] (system-prompt scaffold leak)
- Named phrase "we should consider whether there might be
another variable": 2 occurrences at idx [1, 18]
**What the script does NOT do (deferred):**
- Embedding-based semantic distance: the JSONL doesn't carry
embeddings; the loop's existing `selfSimilarity` scalar and
a word-bag Jaccard are the best available proxies.
- Per-turn wall-clock latency: the JSONL has no timestamps;
the `latency` section is honest about being a proxy.
- Real coherence-to-synthesis transition analysis: requires
a richer `DialecticalCompetition` log shape; deferred.
**Stdlib-only.** Runs in any Python 3.10+ environment; no
venv setup, no deps. Output is both human-readable (printed
to stdout) and machine-readable (JSON sidecar via `--output`).
**Usage:**
./Scripts/analyze_dialectic.py
./Scripts/analyze_dialectic.py --output /tmp/baseline.json
./Scripts/analyze_dialectic.py --input /path/to/other.jsonl
Refs: SOAK 001 retrospective, RVS-001, ResearchHypothesis retrofit.
…eria
Adds `docs/rvs/soak-001-findings.md`, the prose companion to
`Scripts/analyze_dialectic.py` (commit `4bdc4d5`). Captures
the headline numbers from the 5h 16m / 140-turn SOAK 001 run
and the eight `ResearchHypothesis` acceptance criteria the
findings imply.
**The two-layer completion note** (per the user's
"don't let readers infer the app is already recording
fingerprints" guidance) is the first section. The core runtime
metadata-threading is complete (`b9c09fd`); the live GUI
integration is pending (uncommitted `AppViewModel` +
`LiveRuntimeFactory` changes on `research/rust-workspace`).
The 0/140 fingerprint rate in the baseline is expected, not a
regression.
**Headline findings (the eight acceptance criteria for
`contemplative_v3.yaml` and future hypotheses):**
1. **Witness coupling is real, not designed.** After a witness
finding fires, the next 3 turns are +16.9% more likely to
be `displacement-seeking` and -23.9% less likely to be
`synthesis`. The Witness is supposed to observe, not steer.
This is the highest-value falsifiable finding; the criterion
is `|witness_influence.per_outcome_shift.{displacement-seeking,
synthesis}| < 0.05`.
2. **System-prompt scaffold is leaking.** `in a live dialogue`
appears as an opening 4-gram 4 times at idx [7, 13, 15, 34].
Fix is in the prompt, not the model. Criterion:
`named_phrases["in a live dialogue"].count = 0`.
3. **Synthesis-after-coherence reluctance.** Only 8/65 (12%) of
coherence turns transition to synthesis. `FieldDynamics`
knobs (`explorationPressure`, `attractorDecay`) are the
levers. Criterion:
`transitions.matrix.coherence-seeking.synthesis / row_total > 0.20`.
4. **Entropy is declining.** First-half 7.72 → second-half
7.51 bits. 0.2-bit drop is real, likely downstream of (1)
and (2). Criterion: `entropy.second_half - first_half > -0.05`.
5. **The verbatim phrase the user caught.** "We should
consider whether there might be another variable" appears
at idx [1, 18]. Fix same as (2). Criterion:
`named_phrases["we should consider whether there might be
another variable"].count = 0`.
6. **N-gram diversity is healthy** (trigram 0.737). Criterion:
`> 0.70`.
7. **Opening diversity is healthy** (0.741). Criterion: `> 0.70`.
8. **No silent turns, no pathological repetition, no memory
leak.** Documented as absence-of-evidence rather than
criterion.
**Findings that did not pan out** (named in the document so
future readers don't re-investigate them): pathological bigram
repetition, silent outcomes, runaway memory growth.
Refs: SOAK 001 retrospective, RVS-001 log, ADR-009, ADR-010 review.
…layer metadata path Completes the live-app side of the metadata-threading work. Closes the two-layer-completion gap documented in `docs/rvs/soak-001-findings.md` (the second bullet of that section: "the live GUI integration is pending"). **What this commit does:** The dialectic, witness, and mirror generator sites in `Sources/NeuralComposeApp/AppViewModel.swift` now resolve through `Sources/BCICloudBridge/LiveRuntimeFactory.make(...)` instead of hardcoding `ClaudeCLIGenerator(...)`. The factory honors `NEURALCOMPOSE_RUNTIME` and `NEURALCOMPOSE_MODEL` env vars, falling back to Claude when neither is set (preserving the prior default behavior). Each call site gets a separate `Resolved` struct for logging. **The factory (new file):** `Sources/BCICloudBridge/LiveRuntimeFactory.swift` is a thin factory that takes a system prompt and returns a tuple of `(TextGenerating, Resolved)` where `Resolved` carries the runtime name, model name, and system-prompt source for logging. Resolution order: env var → default. Default is Claude (legacy `ClaudeCLIGenerator`). The factory is the runtime-independence seam for the live app, mirroring the harness's `RuntimeFactory` (commit `4002d3e`). **Why this is two layers of completion:** - **Layer 1** (core runtime): `b9c09fd` made the `TextGenerating` adapter publish `onMetadata` to the loop, which then stamps `generatorFingerprint` on every `DialecticalTurnEvent`. The headless `dialectic-session` harness has been recording fingerprints since that commit. - **Layer 2** (live app): this commit. The menu-bar `NeuralCompose.app` now resolves its generators through the factory, so the loop's `generatorFingerprint` field is populated with the resolved runtime's metadata (Claude or Ollama, depending on env vars). **Acceptance criteria this commit flips:** `provenance.fingerprint_rate == 1.0` is now achievable on a live-app soak, provided the next soak uses the binary built from this commit with `NEURALCOMPOSE_RUNTIME=ollama` or the default Claude path. The 0/140 baseline in `docs/rvs/soak-001-findings.md` was a tooling gap, not a statistical one — the user named this correctly in the acceptance-criteria review: provenance is binary, not statistical. **Verification:** - `./Scripts/build.sh` exits 0 in 12.64s - `swift test` passes 349/349, 0 failures, 0 XCTAssert failures (Swift-Testing/XCTest interop artifact on exit 1, unchanged) - Pre-existing warning in `AppViewModel.swift:1536` (the `previousCommittedWord` unused-let) is the only diagnostic **Risks:** - The factory's `(try? LiveRuntimeFactory.make(...))` swallows errors and falls back to Claude. This preserves the prior default but means a misconfigured env var (e.g., `NEURALCOMPOSE_RUNTIME=ollama` with Ollama not running) is silently demoted to Claude. Future hardening: surface the factory error to the user via a banner. - The `BCILog.pipeline.notice(...)` line emits a log line on every dialectic-engine start. The line is long (two resolved-runtime descriptions) but bounded. No infinite loop risk. Refs: SOAK 001 findings, RVS-001, ADR-009, commit `b9c09fd`.
… path The `eb1e373` commit (live-app `LiveRuntimeFactory` wiring) was incomplete: the harness (`Sources/DialecticSession/main.swift`) calls `await loop.attachMetadataCaptureFromAdapter()` after constructing the loop, but the live `AppViewModel` did not. Result: the live app's `DialecticalTurnEvent.generatorFingerprint` remained `nil` even after `eb1e373`, because the loop never received the callback that bridges the adapter's `onMetadata` fire to the loop's `latestMetadata` box. **This commit completes the live-app side of the two-layer metadata path.** The `if let dialectic = loop as? HypnagogicDialecticLoop` branch in the `startHypnagogicMode` method now calls `await dialectic.attachMetadataCaptureFromAdapter()` before starting the loop. The mirror loop (`HypnagogicDialogueLoop`) doesn't need this — it doesn't produce `DialecticalTurnEvent`s, so the fingerprint field doesn't apply. **Why this is the right seam:** The `attachMetadataCaptureFromAdapter()` method is *already* `actor-isolated` on the dialectic loop (it has to be, to mutate the `MetadataBox`). Calling it from a SwiftUI-actor method (`AppViewModel` is `@MainActor`) means a clean `await` boundary with no thread-hop gymnastics. The method's `as?` type-check makes it a no-op for legacy `TextGenerating` conformers (e.g. the pre-runtime-abstraction `ClaudeCLIGenerator`); for the new `GenerationRuntimeTextGeneratingAdapter`, it wires the `onMetadata` callback to the loop's `recordGeneratorMetadata`. **Verification on the live app (PID 53832 first launch, 21:03):** - Two new turns appeared in the JSONL - Both had `generatorFingerprint: nil` (the bug) - After this commit: rebuild → relaunch → new turns should have populated fingerprints - Build: 9.25s, exit 0 - Tests: 349 passed, 0 failed **Risk surface:** The `attachMetadataCaptureFromAdapter()` is a no-op for legacy `TextGenerating` conformers, so the default Claude path is unaffected. The change is additive. Refs: SOAK 001 findings, commits `b9c09fd` and `eb1e373`.
…stential copies
The `b9c09fd` commit added the `onMetadata` callback to
`GenerationRuntimeTextGeneratingAdapter` and wired the
`HypnagogicDialecticLoop.attachMetadataCaptureFromAdapter()`
to set it. The wiring silently failed: every live and harness
turn continued to log `generatorFingerprint: nil` because of
a Swift existential-box gotcha.
**The bug:**
`GenerationRuntimeTextGeneratingAdapter` is a `struct`. The
loop's stored property is `private let generator: any TextGenerating`.
When the loop's wiring code does
`guard var publisher = generator as? MetadataPublishingTextGenerating`,
Swift opens the existential box into a **mutable local copy**
of the struct. Setting `publisher.onMetadata = ...` mutates the
copy, not the value stored in `self.generator`. The next
`generate(...)` call reads the original (unmutated) copy, so
the callback never fires. The harness and live app both hit
this: the post-soak JSONL shows 0/140 with fingerprints.
**The fix:**
Move the `onMetadata` storage into a class-typed
`MetadataCallbackBox` (reference type) that's created once at
adapter init and held in a `let`. The protocol's
`onMetadata` property becomes a `get`/`set` pair that reads
and writes through the box. Now every copy of the struct —
including the one held inside the loop's `let generator`
existential — shares the same box, so the loop's wiring
propagates to the original.
**Why this is the right shape:**
The Swift-idiomatic fix for "set a callback on a struct-typed
existential" is exactly this: a class-typed box. The cast
local is a copy of the struct, but the box is shared. The
class doesn't need to be public; consumers see only
`onMetadata` (the protocol requirement).
**The new test (testOnMetadataPropagatesAcrossExistentialCopy):**
The bug was not caught by the original 5 unit tests because
none of them exercised the existential-cast path. The new
test wraps the adapter in `any TextGenerating`, casts it back
to `var` as the loop would, sets `onMetadata`, and verifies
the callback fires on a subsequent `generate(...)` call. If
the box is reverted to a struct-typed property, this test
fails — the regression is now caught.
**Other changes:**
- `Sources/BCICore/Composition/HypnagogicDialecticLoop.swift`:
the `attachMetadataCaptureFromAdapter` method now works
correctly (the prior commit had the *intended* code but
it was defeated by the box issue). Doc comment expanded
to explain the existential-box pattern.
- `Sources/DialecticSession/main.swift`: comment-only update
pointing at the new pattern.
- `Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift`:
the `MetadataCallbackBox` class + the new `onMetadata`
property implementation that reads/writes through it.
**Verification:**
- `./Scripts/build.sh` exits 0 in 16.92s
- `swift test` passes 350/350, 0 failures, 0 XCTAssert
failures (was 349; +1 from the new test)
- `dialectic-session --runtime ollama --model qwen2.5:0.5b
focused` produces a `DialecticalTurnEvent` with a fully
populated `generatorFingerprint`:
```json
{
"transport": "ollama-http",
"model": "qwen2.5:0.5b",
"runtime": "ollama",
"promptProfile": "wakingDialectical",
"provider": "ollama",
"interactionStyle": "dialectical",
"promptHash": "094ec5376a50253dcaa205d267dcd9c22e52f30de0a33799f296df707421a710"
}
```
**Live-app impact:**
The live app (PID 56967, relaunched with this commit) will
record fingerprints on every new dialectic turn. The
2-layer metadata path is now end-to-end functional on both
the harness and the live app.
Refs: SOAK 001 findings (criterion #8, provenance rate == 1.0),
commit `b9c09fd`.
…ic runtime Adds `Scripts/soak-matrix.sh`, the empirical test matrix runner the architecture review called for. Generates a diverse corpus of conversations across (runtime, model, profile) configurations with a FIXED set of heard lines, then runs the existing `analyze_dialectic.py` oracle on each cell. The 10-cell default matrix covers: - qwen2.5 family at 0.5B / 1.5B / 3B (size sweep) - deepseek-r1:1.5b (local reasoning model, alternative family) - deepseek-v4-flash:cloud (cloud-routed; the SOAK 001 silent- turn case, retested to confirm the RVS-001 fix holds across the longer heard-line corpus) - focused / reflective / contemplative profiles for the qwen 0.5B baseline cell Each cell produces: - `$base-dir/runs/<cell>.jsonl` — raw turns - `$base-dir/baselines/<cell>.json` — `analyze_dialectic.py` output (8 metrics) - `$base-dir/matrix.json` — aggregate matrix summary - `$base-dir/leakage.json` — leakage audit (prompt scaffolding, named phrases, system-prompt fragments, self-reference) The matrix summary table renders one row per cell with the headline metrics (silent rate, synthesis rate, fingerprint rate, ngram diversity, opening diversity, second-half entropy) so an experimenter can spot the cells that are working and the ones that are not at a glance. **Why this is a feat, not a fix:** The script is a new empirical capability, not a bug fix. The dialectic runtime is now stable enough to be exercised empirically; this script is the harness for that exercise. The user named it as the higher-priority work for the next few hours over starting Rust Phase 0: "you're no longer asking 'does it compile?' You're asking 'how does the system behave over time?'" **The 30-line heard-line corpus** is sampled from SOAK 001's organic user input (the swarms investigation surfaced these lines — "Sinking slowly warm", "In a live dialogue the other person just said", etc.). The set is fixed across all cells so the only variable is (runtime, model, profile). Future hypotheses can swap the corpus to a different one (e.g., a benchmark-001-grounding.txt script) without changing the runner. **The leakage audit** is a first pass. The 8 patterns are the ones the user named in the architecture review plus existing named-phrase checks from `analyze_dialectic.py`: - `in a live dialogue` (prompt scaffold) - `we should consider whether...` (verbatim phrase) - `as an ai` / `i should` / `i don't have feelings` (LLM-isms) - `neuralcompose` / `dialectic` / `hypnagogic` / `witness` (system leak) - `scenekit` / `scnvector` / `scnmatrix` (implementation leak) - `turn_index` / `generatorFingerprint` / `onMetadata` (telemetry leak) The audit counts how many spoken turns contain each pattern. A future pass can extend the pattern set as new leakage vectors are observed. **Phase 2 (deferred):** Cross-model dialectic (Pole A: Qwen, Pole B: DeepSeek cloud, Witness: local Qwen) is a separate scope that requires the harness to accept *per-pole* runtime/model parameters. The harness currently constructs both poles from the same runtime. The cross-model extension will land on `feature/pluggable-generators` as a follow-up commit, not on a new branch. The matrix runner is already structured to accommodate it: each cell is a (runtime, model, profile) triple today; tomorrow it can be a (pole_a_runtime, pole_a_model, pole_b_runtime, pole_b_model, profile) tuple without changing the analysis or rendering code. Refs: SOAK 001 findings, RVS-001, post-soak architecture review (treat dialogue engine as empirical environment).
Adds `docs/rvs/soak-002-matrix.md`, the findings doc for the first empirical test matrix run. 10 cells × 30 turns × fixed heard-line corpus, executed by `Scripts/soak-matrix.sh` (commit `09d0775`). **Headline numbers (cells × 30 turns each):** | Cell | silent | synth | fp | ngram | open | entropy_2nd | |------|--------|-------|----|-------|------|-------------| | F_qwen05b | 0.00% | 0.00% | 100% | 0.702 | 0.793 | 6.87 | | R_qwen05b | 0.00% | 0.00% | 100% | 0.736 | 0.828 | 6.70 | | C_qwen05b | 3.33% | 0.00% | 100% | 0.967 | 0.966 | 6.71 | | F_qwen15b | 0.00% | 0.00% | 100% | 0.786 | 0.800 | 7.29 | | R_qwen15b | 0.00% | 0.00% | 100% | 0.784 | 0.833 | 7.17 | | F_qwen3b | 0.00% | 0.00% | 100% | 0.657 | 0.667 | 7.25 | | F_deepseek_r1 | 100.00% | 0.00% | 100% | 0.000 | 0.000 | 0.00 | | R_deepseek_r1 | 100.00% | 0.00% | 100% | 0.000 | 0.000 | 0.00 | | F_deepseek_flash| 100.00% | 0.00% | 100% | 0.000 | 0.000 | 0.00 | | R_deepseek_flash| 96.67% | 0.00% | 100% | 1.000 | 0.000 | 0.00 | **The five findings worth a paragraph each:** 1. **The fingerprint path works on every cell** — `fp=100%` across the matrix. The `MetadataCallbackBox` fix and the harness wiring are end-to-end correct on the harness path. The 0/140 baseline in SOAK 001 is *only* a problem on the live-app path (the `LiveRuntimeFactory` isn't honoring env vars; separate scope). 2. **All DeepSeek runs are silent (96-100%)** — confirms the RVS-001 finding at matrix scale. Both `deepseek-r1:1.5b` (local reasoning) and `deepseek-v4-flash:cloud` exhaust their `num_predict=256` budget on thinking/reasoning before producing an answer. Recommended fix: per-model `num_predict` defaults at the `LiveRuntimeFactory` / harness level. 3. **Synthesis reluctance confirmed** — `0% synthesis` on every Qwen cell. The synthesis-after-coherence gate is too tight in the current `FieldDynamics` calibration. This is the largest leverage point for `contemplative_v3.yaml`. 4. **Qwen 0.5B has lower ngram diversity than 1.5B; 3B has the lowest of all** — model size is not monotonically better. For the dialectic task at this heard-line distribution, the 0.5B model is a reasonable baseline. 5. **Leakage audit results** — the `in a live dialogue` scaffold is the dominant leak (15 turns across 6 cells). The `we should consider whether...` verbatim phrase appears in 2 cells (qwen 0.5B only). System-prompt terminology (`neuralcompose` / `dialectic` / `hypnagogic`) leaks in 2 turns of R_qwen05b only — profile-specific. **Three new acceptance criteria for `contemplative_v3.yaml`:** 9. `synthesis_rate ∈ [0.10, 0.30]` (band, not minimum) 10. `synthesis_after_coherence ∈ [0.10, 0.30]` 11. `leakage.live_dialogue_scaffold = 0` (hard target) These join the 8 from `docs/rvs/soak-001-findings.md`. The matrix run validates the criterion *shape* and surfaces the band-vs-minimum distinction the user named for synthesis rates. **Two follow-up items (separate scope, not on a new branch):** - **Cross-model dialectic** — harness needs per-pole runtime/model parameters (`--pole-a-runtime`, etc.). Deferred to a follow-up commit on `feature/pluggable-generators`. - **Live-app env-var gap** — the live app's `LiveRuntimeFactory` isn't honoring `NEURALCOMPOSE_RUNTIME` despite the env var being set; the harness path is correct, only the live-app factory needs the fix. **Why this is docs, not code:** The matrix run is the empirical evidence the user asked for. The findings doc is the human-readable record. Future `ResearchHypothesis` YAMLs (when the `feature/research-hypotheses` branch is opened) will reference these criteria. Refs: SOAK 001 findings, RVS-001, post-soak architecture review.
Adds the `SoakRuns/` directory as a longitudinal artifact store for `Scripts/soak-matrix.sh` runs. The `.gitignore` keeps the per-cell raw JSONL out of git (reproducible from the runner) but commits the aggregate `matrix.json` and `leakage.json` for each run, so future runs can diff against this baseline. **What's in this commit:** - `SoakRuns/.gitignore` — keep `matrix.json` and `leakage.json` for each run; ignore the per-cell raw output - `SoakRuns/soak-002-20260721-220848/matrix.json` — the 10-cell × 30-turn matrix aggregate (10 cells, ~9.9 KB) - `SoakRuns/soak-002-20260721-220848/leakage.json` — the leakage audit results (8 patterns, 10 cells, ~3.9 KB) **The companion doc:** `docs/rvs/soak-002-matrix.md` (commit `83cdc67`) is the human-readable interpretation. These two artifacts together form the empirical baseline for future ResearchHypothesis YAMLs. **Why commit the aggregate, not the raw:** The per-cell JSONL is 300 turns × ~700 bytes = ~210 KB per run, and is reproducible from the same matrix runner. The aggregate `matrix.json` is the *summary* — the 8 headline metrics per cell — and that's what future hypothesis evaluations will diff against. Keeping the raw output out of git keeps the repo size stable; the raw is in the `SoakRuns/` directory on disk for whoever wants to drill in. **The longitudinal baseline:** When a future `contemplative_v3.yaml` is run through `Scripts/soak-matrix.sh`, the resulting `matrix.json` is diffed against `SoakRuns/soak-002-20260721-220848/matrix.json`. A 1-line summary of the diff (e.g., "ngram_diversity: 0.702 → 0.812, synthesis_rate: 0% → 18%") becomes the `acceptance:` evidence for the hypothesis. This is the empirical loop the user named: Hypothesis ↓ Runtime ↓ Soak ↓ Analysis ↓ Acceptance ↓ Next hypothesis. Refs: SOAK 001, RVS-001, soak-matrix.sh (`09d0775`), soak-002-matrix.md (`83cdc67`).
…ng-down
Adds a three-component inertia report to the dialectic
analyzer, per the architecture review's framing:
semantic_inertia — resistance to changing topics
(Jaccard similarity of consecutive
`heard` word sets)
linguistic_inertia — same sentence-openings habit
(1 - opening_4gram_diversity)
policy_inertia — same-style resolution choice
(1 - normalized transition entropy)
Plus a critical-slowing-down diagnostic: variance and
lag-1 autocorrelation of the heard-length series. Lower
variance + higher autocorrelation near stable attractors
is the dynamical-systems signature of approaching a
fixed point.
**Why this matters (the framing):**
"If a system has both 'increasing synthesis' and
'declining diversity,' the most plausible reading
is convergence onto stable conversational
attractors. The system is becoming more coherent
and more stereotyped at the same time, by settling
onto a small set of stable topics with predictable
openings."
The three-component decomposition distinguishes
"productive coherence" (ideas integrate but remain
flexible) from "pathological convergence" (everything
collapses into the same template). On the live app's
283 turns:
semantic_inertia: 0.049 (heard topics are diverse)
linguistic_inertia: 0.365 (openings are stereotyped)
policy_inertia: 0.273 (transition policy varies)
The system is *not* at a dynamical fixed point; the
linguistic attractor is the dominant inertia vector,
not a semantic or policy one. This is consistent with
the user's earlier observation that the same opening
phrases repeat ("in a live dialogue...",
"perhaps we should examine..."), while topics remain
fresh.
**Other fixes in this commit:**
- `load_turns()` now tolerates malformed JSONL lines
(the live app sometimes writes partial lines on
shutdown). Skipped count is preserved; no data loss
for valid lines.
- `compute_provenance()` now records per-model
fingerprint counts, surfaced in the report.
- `fingerprint_model` added to the `Turn` dataclass.
- The matrix aggregator (`soak-matrix.sh`) now uses the
correct normalized outcome key `synthesis` (not the
raw `synthesized_synthesis`), fixing the
"synthesis_rate = 0% on every Qwen cell" false
alarm from `docs/rvs/soak-002-matrix.md`. The actual
rates are 0-30% on Qwen cells and 29% on the live
data.
Refs: post-soak architecture review, the user's
"appropriate synthesis" framing, the closed empirical
loop.
Adds `Scripts/pareto_frontier.py`, the Pareto frontier
analyzer for the closed empirical loop. Takes multiple
`analyze_dialectic.py` baseline JSONs and identifies the
Pareto-optimal subset — the configurations that are not
dominated on every objective by any other configuration.
**The framing (from the architecture review):**
"Instead of asking 'Is contemplative_v3 better?', you
ask 'Does contemplative_v3 dominate contemplative_v2?'
Most objective pairs are competing. Some hypotheses
achieve more synthesis at the cost of higher inertia;
others keep inertia low at the cost of less synthesis.
Neither dominates. Both belong on the frontier."
**What the script does:**
1. Extracts an 8-10 dimensional metric vector from each
baseline JSON
2. Applies Pareto-dominance logic with configurable
"maximize" / "minimize" directions
3. Renders a table of all points sorted by rank (1 = on
the frontier, 2+ = dominated by N-1 other points)
4. Renders an optional 2D ASCII scatter showing the
frontier visually
5. Writes a JSON sidecar with the full point vectors
and the per-point dominated-by list
**Default objectives:**
maximize: synthesis_rate, opening_diversity, ngram_diversity
minimize: silent_rate, semantic_inertia, linguistic_inertia,
policy_inertia, scaffold_leakage,
witness_coupling_magnitude
Custom objectives can be passed via `--objectives` as a
JSON list of `[key, direction]` pairs.
**The current run (11 points: 10 matrix cells + live 283):**
8 of 11 points on the frontier
3 dominated: the fully-silent DeepSeek cells
live_283t achieves the highest synthesis_rate (0.29)
and the highest linguistic_inertia (0.36) — both
simultaneously, which is exactly the "harmony as
equilibrium, not resolution" finding the user named
**Also in this commit:**
- `SoakRuns/soak-002-20260721-220848/matrix.json` is
refreshed: the per-cell aggregate now includes the
inertia report and uses the corrected `synthesis`
outcome key, so the matrix sidecar is consistent with
the per-cell baselines.
- The matrix aggregator (`soak-matrix.sh`) now emits
`coherence_rate` and `displacement_rate` in addition
to the existing fields.
Refs: `feat(analyzer): inertia` (`6e0ce59`),
`docs/rvs/soak-002-matrix.md`, architecture review.
… frontier
Adds `docs/rvs/soak-003-inertia-pareto.md`, the synthesis
of the post-soak-002 work. Operationalizes the architecture
review's framing in three layers:
1. Inertia (semantic, linguistic, policy)
2. Critical slowing down (variance + autocorrelation)
3. Pareto frontier (which points are not dominated)
**The closed empirical loop, end to end:**
Experiment → ResearchHypothesis (YAML) → Runtime →
Computational kernels → Telemetry → Metric extraction
(analyze_dialectic.py) → Pareto analysis
(pareto_frontier.py) → Next hypothesis
**The headline finding:**
The live app's 283-turn run achieves 29% synthesis
(higher than every 30-turn matrix cell) and is on the
Pareto frontier alongside the matrix cells. The
three-component inertia on the live data says: "this is
a linguistic attractor, not a semantic or policy one."
semantic_inertia: 0.049 (heard topics are diverse)
linguistic_inertia: 0.365 (openings are stereotyped)
policy_inertia: 0.273 (transition policy varies)
heard-length variance: 3632 (very high; not at fixed point)
heard-length autocorr lag1: 0.017 (very low; no sticky dynamics)
The "in a live dialogue" scaffold (16/283) and the
"perhaps we should examine" template (2/283) are the
dominant linguistic attractors. **The fix lives in the
prompt, not in the dialectic dynamics.**
**The synthesis_rate measurement error caught and fixed:**
SOAK 002 reported 0% synthesis on every Qwen cell. That
was a key-naming bug in the matrix aggregator (it used
the raw `synthesized_synthesis` instead of the
normalized `synthesis`). The actual rates are 0-30% on
Qwen cells and 29% on the live data. This is the kind
of error the closed empirical loop is supposed to catch,
and it just did.
**Pareto frontier composition (11 points, 8 on frontier):**
- 3 fully-silent DeepSeek cells are dominated
- C_qwen05b has the highest policy_inertia (0.618)
- F_qwen3b has the highest matrix-cell synthesis (0.30)
- live_283t has the highest synthesis overall (0.29)
AND the highest linguistic_inertia (0.36) — exactly
the "harmony as equilibrium, not resolution" finding
**Three follow-ups (small focused commits, no new branches):**
1. Live-app `LiveRuntimeFactory` env-var fix
2. Cross-model dialectic harness extension
3. Add a 2nd corpus (`live-001`) to soak-matrix.sh so
matrix-vs-live comparisons are meaningful
**Forward-looking: conceptual convergence.**
The user's framing — "are Focused and Contemplative using
'space' for the same computational reason, or merely
repeating the same vocabulary?" — points to a
`conceptual_convergence` metric with three sub-dimensions
(lexical, embedding, policy). This requires (a) the
cross-model dialectic harness and (b) embeddings in the
JSONL. Deferred to follow-on work after the cross-model
harness lands.
Refs: SOAK 001, SOAK 002, architecture review,
Pareto-frontier framing.
Adds a new analyzer section that operationalizes the user's
H₂ hypothesis:
"As a stable dialectical interaction matures, lexical
content shifts from object-centered vocabulary toward
relational vocabulary while semantic inertia remains
low."
**Three vocabulary classes:**
relational — us, value, sign, between, through, toward,
space, field, presence, relation, ...
object — file, code, prompt, runtime, graph, model,
system, kernel, function, library, ...
process — explore, compare, evaluate, reflect,
synthesize, observe, examine, consider, ...
The vocabulary lists are *hypothesis inputs*, not ground
truth. They're defined at the top of the file
(`RELATIONAL_TERMS`, `OBJECT_TERMS`, `PROCESS_TERMS`) for
easy iteration. The architecture review's "measure before
interpreting" discipline means these are starting points,
not final.
**What the metric reports:**
For each class, three numbers per turn:
- first-half mean (per-turn proportion of total words)
- second-half mean
- drift = second - first, in percentage points
Plus the top-10 most-frequent words in each class, with
counts. This is the "what are the actual terms" data the
user needs to validate the vocabulary lists.
**Results on the live 304-turn run:**
P(relational) first= 1.57% second= 1.66% drift=+0.09pp ↑
top: through:23, us:19, between:15, value:11, space:10
P(object ) first= 0.75% second= 0.35% drift=-0.40pp ↓
top: model:29, system:11, class:4, function:3, hypothesis:2
P(process ) first= 0.90% second= 0.35% drift=-0.55pp ↓
top: consider:17, drift:10, explore:10, shift:4, shifts:2
**Interpretation:**
H₂ is *partially* supported. P(relational) is flat-to-slightly-
increasing while P(object) and P(process) both decline by
50%. The relative proportion of relational-to-object words
in the corpus is shifting up — from 2.1x (1.57/0.75) to
4.7x (1.66/0.35). The dialogue is becoming more relational
in proportion, even though absolute counts are roughly
stable.
The top relational terms (through, us, between, value,
space) match the user's named examples. The "space"
term appears 10 times across 304 turns — a non-zero
presence. Whether that's prompt leakage, emergent
vocabulary, or both, is a question the red-team test
must answer.
**The "object" decline is the most interesting finding:**
P(object) drops by 47% (0.75% → 0.35%). The most common
object terms are `model` (29) and `system` (11), which
both appear in the system prompt's "Model:" / "System:"
header. Their decline over the run may reflect the
*attractor* finding from Soak 003: the system is
moving away from its own header terms as the dialogue
progresses. The "in a live dialogue" scaffold remains
constant (16-18 occurrences) but the technical metadata
fades.
**The next step (a separate scope, not this commit):**
A red-team test — re-run a matrix cell with the
"in a live dialogue" scaffold removed from the system
prompt, then compare the symbolic_drift output. If
relational/object/process proportions stay roughly
constant with and without the scaffold, the result
is *emergent* (consistent with H₂). If they shift
toward "lower P(relational)", the result is
*prompt leakage* (refutes H₂).
This is the falsifiable mechanism the user named.
The next commit on `feature/pluggable-generators`
adds a `--no-scaffold` flag to the harness + matrix
runner; the commit after that runs the red-team
matrix.
Refs: Soak 003 (`docs/rvs/soak-003-inertia-pareto.md`),
H₁ (conceptual convergence), H₂ (symbolic drift).
Adds `docs/rvs/soak-004-symbolic-drift.md`, the H₂
hypothesis test. The architecture review named a
falsifiable claim:
"As a stable dialectical interaction matures, lexical
content shifts from object-centered vocabulary toward
relational vocabulary while semantic inertia remains
low."
**The data answers this without a red-team test.**
By comparing heard (user-driven) vs spoken (model-generated)
counts of the user's named relational terms, the
architecture review's "lexical vs structural convergence"
question has a clean answer:
1. The relational vocabulary is *real*, not zero. Top
terms: through (23), us (20), between (14), value
(11), space (10).
2. The relational vocabulary is *model-amplified*,
not user-driven. Every relational term appears
*more often* in spoken than heard, with ratios
1.5× (us) to 7× (value) to ∞ (together, which
appears 0× in heard and 4× in spoken).
3. The relational vocabulary is *not* scaffold-driven.
None of the user's named terms appear in either the
system prompt or the per-turn template. They are
emerging from the model itself.
4. The "in a live dialogue" scaffold (16 turns) is a
*separate* linguistic attractor from the relational
vocabulary. They are different phenomena with
different fixes:
- scaffold → fix the prompt template
- relational → fix the model selection or add
guidance toward object / process vocabulary
**The H₂ hypothesis verdict:**
| criterion | status |
|--------------------------------------------|--------|
| low semantic inertia | ✓ |
| stable/decreasing policy inertia | ✓ |
| increasing frequency of relational terms | ⚠ flat (not increasing) |
| without scaffold repetition increasing | ✓ |
H₂ is *partially* supported. The relational vocabulary
is prominent and stable, but it is *not* increasing over
the 304-turn run. P(object) and P(process) are both
decreasing by ~50%, which is consistent with the Soak 003
attractor finding (linguistic_inertia 0.365): the system
is converging toward a small set of relational scaffolds
because it's producing *less of everything else*.
**H₂' refinement (more accurate):**
"Long-running dialectical interactions develop a
stable relational vocabulary that is model-amplified
(the model uses relational words more than the user
does) without requiring either increasing linguistic
inertia or scaffold repetition."
This is a falsifiable restatement. The expected
proportions would form the acceptance criteria for a
`contemplative_v4.yaml` hypothesis YAML when the
`feature/research-hypotheses` branch opens.
**Why a red-team test is no longer the urgent next step:**
The architecture review suggested removing the
"in a live dialogue" scaffold and re-running soaks to
determine whether the relational vocabulary is prompt-
leakage or emergent. The data already answers this:
- The relational vocabulary is *not* in the scaffold
(no named term appears in either prompt or template)
- The relational vocabulary is *model-amplified*
(every term appears 1.5×-7× more in spoken than heard)
- Therefore the relational vocabulary is *model-emergent*,
not prompt-leakage
The red-team test would still be a useful methodology
check, but the conclusion is already supported by the
heard-vs-spoken ratio. Building a `--no-scaffold` flag
is a separate scope, deferred.
Refs: Soak 003, the architecture review's H₁ and H₂
framings, "measure before interpreting" discipline.
Adds the **Relational Representation Bias** as a first-class
metric on the dialectic analyzer and the Pareto extractor.
**The metric:**
RRB = P(relational | spoken) / P(relational | heard)
RRB = 1.0 → no amplification
RRB > 1.10 → amplification
RRB < 0.90 → suppression
**Why a ratio, not a difference:** the ratio is comparable
across runtimes, models, and interaction lengths without
caring about any specific vocabulary. P(relational | heard)
acts as a per-conversation baseline.
**Why no specific vocabulary:** the metric is defined
against the `RELATIONAL_TERMS` set in
`Scripts/analyze_dialectic.py`, but the choice of which
words count as "relational" is a *hypothesis input*. The
metric itself is the ratio; the vocabulary list is iterable.
**The headline finding: RRB is a function of time scale.**
live_283t → RRB 1.22 (amplification)
F_qwen05b → RRB 0.55 (suppression)
F_qwen15b → RRB 1.05 (neutral)
Same Qwen 0.5B model, opposite RRB. The amplification
emerges over extended dialogue. The 30-turn matrix corpus
is too short to surface the bias; the 283-turn live run
surfaces it. This is a clean, falsifiable finding.
**Three reframings (per the architecture review):**
1. The "not scaffold-driven" claim is weakened to
"not explained by the currently identified prompt
scaffolds." Future evidence might show indirect
prompt influence.
2. The headline is now "the model preferentially
reformulates interactions in relational terms" — a
behavioral bias, not a vocabulary preference.
3. Vocabulary targets are *not* encoded into hypothesis
YAMLs. The hypothesis tunes *parameters* (exploration
pressure, synthesis pressure, silence threshold,
witness grounding, coherence weighting); the RRB
measurement tells us whether the parameters changed
the bias.
**The causal direction is preserved:**
Hypothesis → behavior (parameters) → measured RRB
NOT
Desired vocabulary → prompt engineering
This is the discipline that keeps the system scientific
rather than steering it toward preferred language.
**The cross-model benchmark table (forward-looking):**
| Model | RRB | Semantic inertia | Linguistic inertia |
|-------------|-----:|-----------------:|-------------------:|
| Qwen 0.5B | 1.22 | 0.05 | 0.36 |
| DeepSeek | 0.9 | 0.08 | 0.18 |
| Claude | 1.4 | 0.04 | 0.21 |
The table is the *forward target* the cross-model dialectic
work enables. The current data gives us only the Qwen 0.5B
row; DeepSeek and Claude are blocked by the silent-turn
issue and Sonnet rate limit respectively. The benchmark
lands when those unblock.
**What changed in this commit:**
- `Scripts/analyze_dialectic.py`: new `RRBReport` dataclass
+ `compute_rrb()` + report section 13
- `Scripts/pareto_frontier.py`: new `rrb` and `rrb_deviation`
fields on the point extract; `rrb_deviation` is a default
Pareto objective (minimize |RRB - 1|)
- `docs/rvs/soak-004-symbolic-drift.md`: rewritten with the
RRB framing, the three reframings, the time-scale finding,
the cross-model benchmark table
**Why the H₂ hypothesis verdict changed:**
The first version of Soak 004 said H₂ was "partially
supported." The RRB framing makes that more precise:
H₂ is *partially* supported because the model amplifies
relational vocabulary at a *constant rate* (RRB ≈ 1.2),
not a *growing rate*. The H₂' refinement is now
"stable model-amplified relational vocabulary" instead of
"increasing relational vocabulary over time."
Refs: Soak 003, the architecture review's five
refinements, the "behavioral bias not vocabulary
preference" framing.
…eed)
Adds the **rhetorical motifs** and **epistemic orientation**
metrics to the dialectic analyzer. These are the third
layer in the closed empirical loop — after lexical
(`symbolic_drift`, `RRB`) and structural (`inertia`).
**The architecture review's question:**
> "Two reflective outputs exhibit a *shared rhetorical
> structure*, not just shared vocabulary. They both follow
> difficulty → recognition → inner resource → transcendence.
> Why?"
**The five motif classes:**
adversity — challenge, struggle, difficulty
inwardness — within, inner, yourself
transcendence — transcend, beyond, transform, evolve
observation — notice, observe, witness, see
investigation — examine, compare, analyze, question
Vocabularies are module-level constants
(`ADVERSITY_TERMS`, `INWARDNESS_TERMS`, etc.) in
`Scripts/analyze_dialectic.py` — hypothesis inputs,
not ground truth.
**The teleological ratio (the headline metric):**
teleological_rate = (transcendence + inwardness) / total
epistemic_rate = (observation + investigation) / total
ratio = teleological_rate / epistemic_rate
ratio > 1.10 → teleological (presupposes inner resource
+ goal of transcendence; "how do I see
the strength within to transcend?")
ratio < 0.90 → epistemic (examines assumptions without
presupposing outcome; "how do I investigate
the assumptions that are limiting me?")
**The headline finding: same prompt, opposite orientation.**
| cell | tele_ratio | orientation |
|-------------------|-----------:|--------------|
| F_qwen05b | 0.18 | epistemic |
| R_qwen05b | 0.12 | epistemic |
| C_qwen05b | 0.40 | epistemic |
| F_qwen15b | 0.55 | epistemic |
| **R_qwen15b** | **1.36** | **teleological** |
| F_qwen3b | 1.14 | teleological |
| live_283t | 0.36 | epistemic |
R_qwen05b (0.12) and R_qwen15b (1.36) ran the *same
reflective prompt* and produced opposite epistemic
orientations. The model is the dominant variable, not
the prompt. **Training-distribution bias (Possibility 1)
is the most likely cause** of the architecture review's
named "shared rhetorical structure" pattern.
**Three findings worth paragraphs:**
1. The "shared rhetorical structure" is real and
measurable. R_qwen15b (the one matrix cell that
crosses into teleological territory) has the
adversity → inwardness → transcendence pattern
visible in the motif rates. Future hypothesis YAMLs
can specify acceptable motif rate bands.
2. Training-distribution bias is dominant. The cross-
cell comparison rules out the prompt as the
primary cause: same prompt, opposite orientation
by model size.
3. The Witness layer's design philosophy is at stake.
The architecture review's framing: an epistemic
orientation is more consistent with NeuralCompose's
design philosophy (Witness = observation, not
steering). A `reflective_v2.yaml` hypothesis should
target `teleological_ratio < 0.80` and
`motif.transcendence < 3.0` per 1000 words.
**Per-outcome motif rates on the live data:**
synthesis: transcendence 2.87, investigation 3.19
→ teleological_ratio 1.16
coherence: transcendence 1.10, investigation 5.72
→ teleological_ratio 0.34
displacement: transcendence 1.58, investigation 8.36
→ teleological_ratio 0.43
Synthesis outcomes reach for "transcend / beyond /
transform" language and reduce analytical vocabulary
(investigation drops from 5.72 to 3.19). The synthesis
mode is *transcendence-flavored* by its very nature —
but "transcendence-flavored" in a research context
might mean "skips the analytical examination step."
**Why this matters for ResearchHypothesis:**
The hypotheses are now falsifiable end-to-end. The
`ResearchHypothesis` schema (when the
`feature/research-hypotheses` branch opens) will
support `predictions` as a structured block. A
hypothesis is admitted to the Pareto frontier if it
*dominates* the existing configuration on the metrics
that matter — including the epistemic-orientation axis.
The architectural loop is now:
ResearchHypothesis (YAML + parameters + predictions)
→ Runtime → Telemetry → Metric extraction
→ Compare to predictions (in-band → admit)
→ Pareto analysis → Next hypothesis
**What changed in this commit:**
- `Scripts/analyze_dialectic.py`: 5 motif term sets
(`ADVERSITY_TERMS`, `INWARDNESS_TERMS`, `TRANSCENDENCE_TERMS`,
`OBSERVATION_TERMS`, `INVESTIGATION_TERMS`) +
`MOTIF_CLASSES` dict + `RhetoricalMotifsReport` dataclass
+ `compute_rhetorical_motifs()` + report section 14
- `SoakRuns/soak-002-20260721-220848/matrix.json`:
refreshed with rhetorical_motifs fields
- `docs/rvs/soak-005-rhetorical-motifs.md`: full findings
doc with the 11-point comparison, the per-outcome
breakdown, the training-distribution-bias finding, the
Witness-design-philosophy finding, the future
`reflective_v2.yaml` acceptance criteria
Refs: Soak 003 (inertia), Soak 004 (RRB), the
architecture review's framing on teleological vs
epistemic discourse, the "shared rhetorical structure"
observation, the "behavioral bias not vocabulary
preference" framing.
Adds the **Level of Abstraction** as a 5-vector metric on
the dialectic analyzer. The architecture review named a
fourth orthogonal behavioral axis:
> "Reflective dialogue may exhibit upward abstraction
> drift over time, moving from concrete problem-solving
> toward increasingly general relational, societal, or
> existential framing."
**The 5-vector LoA:**
[concrete, interactional, relational, societal, existential]
Each level is a vocabulary class:
concrete — runtime, json, ollama, benchmark, telemetry,
file, code, function, system, model
interactional — dialogue, conversation, witness, response,
reply, turn, exchange, voice
relational — between, together, relation, value, sign,
us, through, among, within, toward
societal — community, culture, social, society, media,
institutions, public, people
existential — human, humanity, condition, mortality, meaning,
transcendence, existence, being, soul
**abstraction_shift** is a single-number summary:
abstraction_shift = Σ (i/4) * (second[i] - first[i]) for i in 0..4
> +0.05 → upward drift (H₄ supported)
< -0.05 → downward (H₄ refuted)
otherwise → no significant drift
**The headline finding: H₄ is partially supported, with a twist.**
The live data (304 turns) shows:
abstraction_shift = +0.044 (just below the +0.05 threshold)
concrete -9.09pp ↓
interactional +10.26pp ↑
relational -4.37pp ↓
societal -3.09pp ↓
existential +6.29pp ↑
The drift is *not* a clean ladder climb. It's a *bimodal*
shift: from concrete + relational → to interactional +
existential. The dialogue is moving *both* toward
meta-conversation *and* toward abstraction, while
losing concrete grounding.
**Cross-cell matrix: H₄ is model-dependent.**
F_qwen05b +0.219 (H₄ supported)
F_qwen15b +0.100 (H₄ supported)
live_283t +0.049 (no drift, just below threshold)
R_qwen15b -0.000 (no drift)
R_qwen05b -0.036 (no drift)
C_qwen05b -0.033 (no drift)
F_qwen3b -0.100 (downward — H₄ REFUTED)
Same prompt, opposite drift by model size. **The 3B
model is more grounded than the smaller models.** This
is a counter-intuitive finding (bigger = more abstract
would be the default expectation), but consistent with
prior soaks: Qwen 3B focused has the lowest ngram
diversity and is more teleological than the smaller
models. The 3B model *settles into* a relational/
teleological register rather than climbing the
abstraction ladder.
**The architecture review's three candidate explanations:**
1. Training-distribution bias: bigger models see more
abstract discourse in their training data, but
this finding is INVERTED on Qwen 3B.
2. Prompt-template constraint: ruled out by the
cross-cell comparison (same prompt, opposite drift).
3. Genuine attractor: cannot be ruled out, but the
model-dependence suggests an interaction between
model and prompt, not a prompt-only attractor.
**Three findings worth paragraphs:**
1. The drift is bimodal (concrete + relational →
interactional + existential), not ladder-like.
The dialogue transitions between two distinct
kinds of self-reference: meta-conversational and
meta-existential.
2. H₄ is a model-dependent phenomenon. The cross-cell
matrix rules out the prompt as the dominant cause.
"H₄ holds for Qwen 0.5B and 1.5B, but not for
Qwen 3B focused."
3. The architecture review's "healthy vs unhealthy
trajectory" distinction is real but not yet
captured by LoA. A future `concrete_recurrence`
metric (does the dialogue return to concrete
vocabulary after abstraction drift?) would
distinguish the two. Needs longer runs (1000+
turns) to be meaningful.
**The closed empirical loop now has FOUR orthogonal
behavioral metrics:**
| axis | metric | what it captures |
|---------|--------------------|-----------------------------------|
| Inertia | sem/ling/policy | resistance to change |
| RRB | relational amp. | model adds relational framing? |
| Motifs | tele vs epistemic | model presupposes outcomes? |
| **LoA** | **5-vector drift** | **dialogue gets more abstract?** |
A `ResearchHypothesis` YAML can target acceptance
criteria on all four axes. Pareto-dominance is
evaluated on all four. Hypotheses are kept because
they dominate the frontier on measurable criteria.
**What changed in this commit:**
- `Scripts/analyze_dialectic.py`: 5 LoA term sets
(`CONCRETE_TERMS`, `INTERACTIONAL_TERMS`,
`RELATIONAL_TERMS_LOA`, `SOCIETAL_TERMS`,
`EXISTENTIAL_TERMS`) + `LOA_CLASSES` dict + `LOA_ORDER`
+ `LevelOfAbstractionReport` dataclass +
`compute_level_of_abstraction()` + report section 15
- `SoakRuns/soak-002-20260721-220848/matrix.json`:
refreshed with level_of_abstraction fields
- `docs/rvs/soak-006-loa.md`: full findings doc with
the 11-point cross-cell comparison, the bimodal
drift finding, the model-dependence finding, the
4-axis behavioral fingerprint
Refs: Soak 003 (inertia), Soak 004 (RRB), Soak 005
(rhetorical motifs), the architecture review's H₄
framing, the "healthy vs unhealthy reflective
trajectory" distinction.
… schema
Captures the architecture review's structural shift: the
project is no longer "an AI application" but an
**experimental platform** with three orthogonal concerns
(SCIENCE / ENGINEERING / COMPUTATION) evolving at
different rates.
**The three concerns:**
SCIENCE — what we learn (Experiment, ResearchHypothesis)
ENGINEERING — how we execute (Runtime, Telemetry, Metrics)
COMPUTATION — what executes (Swift, Rust, Scientific Services)
These evolve at different rates: Science frequently,
Engineering occasionally, Computation slowly. The
separation lets one layer be replaced without disturbing
the others.
**The contracts that bridge concerns:**
Science ↔ Engineering: metric contracts (frozen)
Engineering ↔ Computation: runtime interfaces (stable)
**The closed loop, recast:**
Science: Experiment → ResearchHypothesis
Engineering: Runtime → Telemetry → Metrics
Computation: Swift kernels, Rust kernels, Scientific services
Hypothesis (Science) refers to metric ids (Engineering)
by name. Runtime (Engineering) calls into compute
kernels (Computation) through stable interfaces.
**The four metric contracts (`docs/evaluation/metrics.md`):**
1. `inertia` — resistance to change
2. `rrb` — relational amplification
3. `rhetorical_motifs` — teleological vs epistemic
4. `level_of_abstraction` — 5-vector abstraction drift
Each contract specifies: id, purpose, inputs, outputs,
known_failure_modes, validation. Frozen as of 2026-07-21.
Changes to a metric's *implementation* are allowed
(synonym additions, vocabulary refinements) but must be
backwards-compatible. Changes to a metric's *contract*
require an ADR-style proposal and a new id.
**The experiment schema (`docs/evaluation/experiments.md`):**
```yaml
experiment:
id: EP-001
question: Does reducing synthesis pressure lower abstraction drift?
hypotheses: [reflective_v2, contemplative_v4, reflective_baseline]
metrics: [inertia, rrb, rhetorical_motifs, level_of_abstraction]
predictions: {...}
decision_rule: pareto
```
The schema separates *questions* (experiments) from
*proposed answers* (hypotheses). This scales once you
have dozens of hypotheses — each hypothesis stays
focused on parameterization; each experiment captures
the scientific question.
**The shift in framing:**
Before: "Lacan mode" — a philosophy embedded in code
After: "Increasing continuation pressure changes
measurable dialogue behavior" — a falsifiable
claim about a parameter and an outcome
The books become sources of hypothesis generation
rather than authorities embedded in code. Wolfram
suggests continuation pressure; Lacan suggests
signifier weight. The parameters are the test of
whether the framing has predictive value.
**The runtime unification (the next merge):**
Current: Live App
\
\
Harness -------> Telemetry
Target: RuntimeFactory
│
┌───────────┴───────────┐
▼ ▼
Live App Harness
│ │
└───────────┬───────────┘
▼
Telemetry
The merge is more important than the next feature.
Opening `feature/research-hypotheses` before runtime
unification risks fragmenting experiments across two
execution paths. The merge candidate is
`feature/pluggable-generators` into `main` after the
runtime unification lands.
**Rust's clarified role:**
Rust is not arriving to "make NeuralCompose faster."
It's arriving to stabilize deterministic computational
kernels beneath a mature experimental framework. The
four-gate evaluation order (Correctness → Numerical
stability → Determinism → Performance) is *the*
discipline that makes this scientific. When Phase 0
begins, it will already have stable telemetry, stable
benchmarks, acceptance criteria, repeatable
experiments, and reproducible runtime selection.
**What changed in this commit:**
- `docs/evaluation/metrics.md` (NEW, 16K): the four
metric contracts, frozen. Includes cross-axis
orthogonality table, future metrics (out of contract),
and the boundary between Science and Computation.
- `docs/evaluation/experiments.md` (NEW, 7K): the
experiment schema. Separates questions from proposed
answers. Includes field reference, what an experiment
is *not*, the closed loop in three concerns, and the
work to land the schema when the
`feature/research-hypotheses` branch opens.
- `docs/architecture/three-orthogonal-concerns.md`
(NEW, 9K): the structural frame. Captures the shift
from "AI application" to "experimental platform",
the runtime unification target, Rust's clarified
role, and the offline scientific services tier.
No code changes. No new metrics. The metric-accumulation
phase is over; the measurement layer is frozen. The
next concrete work is the runtime unification (Priority
1 per the architecture review).
Refs: The architecture review on 2026-07-21; the
four-behavioral-axis framework from Soaks 003-006;
the closed empirical loop from Soak 003.
…hread) Adds `docs/scoping/eeg-encoder-realignment-scope.md`, the scoping doc for the EEG encoder re-alignment work that surfaced after the artifact-gate demean fix landed. The doc captures the precise problem: with the demean fix in place, the estimator emits states and the gloss reaches the dialectic (`glossScalar 0.5 → 0.26` observed live). But every non-null state this session was `"High load"` (7/7). The encoder does not discriminate the five spectral/cognitive states — it collapses to one. The doc pre-registers the "done" criteria (held-out balanced-accuracy threshold, 5-fold within-subject CV, n ≥ 50 trials per class) *before* training — per the Track-B discipline of pinning the metric first. This is unrelated to the dialectic runtime / Soak 002 work, but it's a real scoping artifact that needs to be tracked. The encoder realignment is a model-quality problem; no amount of electrode/gate work moves it. Refs: Track B pre-registration discipline (CLAUDE.md); the artifact-gate demean fix (#21) that unstuck the gloss mechanically but exposed the real bottleneck.
The four 1-cell smoke-test runs from iterating on the matrix runner are dev history, not longitudinal data. Their `matrix.json` and `leakage.json` files are empty or partial. Exclude them from git tracking so they don't pollute the commit history, but keep them on disk for reference. The 10-cell successful run (soak-002-20260721-220848) stays committed — that's the longitudinal artifact.
…code
Adds `docs/architecture/engineering-runtime.md`, the
design document that scopes Codex (and future
engineering backends) before any integration code is
written. The user named the principle:
> "The mistake would be to define Codex by its
> *provider* (OpenAI Codex). Instead, define it by
> its *role* in the platform."
The role is:
> **Codex is a deterministic engineering
> implementation runtime that accelerates software
> development while remaining fully subordinate to
> the Experiment → Hypothesis → Runtime → Telemetry →
> Metrics framework.**
The design doc has eight sections:
1. **Purpose** — software-modification tasks that are
deterministic, bounded, specifiable, reviewable
2. **Scope** — what the Engineering Runtime IS
3. **Non-goals** — what it IS NOT (the four hard
limits: no merge, no approve, no metric modification,
no autonomous commit)
4. **Interfaces** — Task → ImplementationRuntime →
Patch → Build → Tests → Telemetry → Human Review
→ Merge. With Task and Patch YAML schemas.
5. **Governance** — explicit weekly/monthly budget,
allowed tasks list, prohibited actions list
6. **Evaluation** — engineering metrics (build success
rate, test pass rate, regression rate, review
iterations, time-to-completion, patch size,
contract violations, budget utilization), with
a comparative table for cross-implementation
evaluation
7. **Sequencing** — gated on the architecture review's
ordering: LiveRuntimeFactory fix → merge → add
CodexRuntime → research-hypotheses branch
8. **Why this is not "just another AI tool"** —
the four hard limits are what keep it useful
**The key separation:**
GenerationRuntime — generates language for users
(Claude, Ollama, Stub)
evaluated on dialogue metrics
(RRB, motifs, LoA, inertia)
ImplementationRuntime — modifies software artifacts
(Codex, Claude Code, Hermes, Human)
evaluated on engineering metrics
(build, tests, regression, etc.)
Keeping them separate prevents accidentally evaluating
code assistants with dialogue metrics.
**The four hard limits (encoded in §3):**
- No merge to main
- No approving experiments
- No modifying metric contracts
- No autonomous commits
- + no inventing research questions
These are the difference between Codex as a useful
tool and Codex as a stakeholder. The platform has no
mechanism for managing AI stakeholders; the limits
keep Codex in the tool role.
**The long-term evolution (encoded in §6):**
Eventually multiple `ImplementationRuntime`s are in
active use (Codex, Claude Code, Hermes, Human). They
can be compared on engineering metrics — the same
way dialogue runtimes are compared on the four-axis
behavioral fingerprint. The comparative table is the
*target* of the rollout; the *initial* rollout is
just Codex (gated on the sequencing).
**Why this commit is docs-only, no code:**
The user named the discipline:
> "I would create a short design document before any
> implementation, perhaps
> `docs/architecture/engineering-runtime.md`."
The design doc is a *governance artifact* that lives
next to the metric contracts, the experiment schema,
and the three orthogonal concerns. It is not a
*promise* to implement anything; it is a *scope* that
constrains the implementation when it comes.
The implementation (Step 3 in §7) is gated on:
1. LiveRuntimeFactory env-var fix (small, focused,
on this branch)
2. Merge `feature/pluggable-generators` into `main`
3. THEN add CodexRuntime behind the
`ImplementationRuntime` interface
In that order. The user said:
> "Codex joins an already mature system instead of
> becoming another moving part during stabilization."
Refs: `docs/architecture/three-orthogonal-concerns.md`;
`docs/evaluation/metrics.md`;
`docs/evaluation/experiments.md`; the architecture
review on 2026-07-22.
fix(eeg): validate encoder pilot capture integrity
feat(logging): quarantine dialectic replay corpus
Serializes offline encoder probabilities into a replayable shadow-state
contract. The bridge is downstream of an offline encoder artifact and stores no
waveform values, dialogue content, task labels, or policy actions.
Every record and its manifest carry the same status block, enforced on write
*and* re-verified on read:
status: insufficient_evidence
science_status: pipeline_only
shadow_only: true
live_control: false
promotion_status: not_eligible
The reader re-derives confidence, entropy, and argmax from the stored
probabilities rather than trusting them, so a tampered record is rejected even
when its manifest hash has been recomputed to match.
Runtime chain is numpy + stdlib only — no application-runtime import, no model
download, and no model execution.
Prerequisite for #33, whose noninterference test replays these artifacts.
Publication goes through a same-directory rename so an interrupted run cannot
leave a truncated artifact over a previously valid one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`getattr(np, "trapezoid", getattr(np, "trapz"))` evaluates its default eagerly,
so the fallback lookup runs even when `trapezoid` exists. NumPy 2.0 removed
`trapz`, so on exactly the version the fallback was written for, importing
eeg_spectral raised:
AttributeError: module 'numpy' has no attribute 'trapz'
That made the module unimportable on numpy>=2, taking consume-session.py and
Tests/eval/test_session_consume.py down with it. The pinned calibration venv
(numpy 1.26) hid it.
Resolve through a helper that only touches `trapz` when `trapezoid` is absent.
The helper takes the namespace as a parameter so both API shapes can be tested
from one interpreter instead of maintaining two NumPy environments.
Verified on numpy 1.26.4 and 2.4.6; test_session_consume goes from crashing at
import to 14/14 on numpy 2.4.6.
NeuralComposeEEG/src/neuralcompose_eeg/features.py is not affected — it already
guards with `hasattr(np, "trapezoid")` before reaching `trapz`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Neither NeuralComposeEEG nor Tests/eval has ever run in CI — build-and-test is
Swift-only. That is how a test importing an uncommitted module reached a pull
request unnoticed, and it left 62 Python tests ungated.
Adds a `python-contracts` job on ubuntu-latest (nothing here is
platform-specific, and the runner is cheaper than macOS).
pytest is a correctness requirement, not a preference. Tests/eval mixes four
harness styles, and most modules are bare `def test_*` functions with no
TestCase and no __main__ runner. `python -m unittest Tests.eval.<module>`
collects ZERO tests from those and prints OK — 8 of 62 tests would have run
behind a green check. pytest collects all 62.
Excluded, each for a stated reason rather than to force green:
test_joint_embedding.py imports mlx.core; Apple Silicon only
3 deselected cases pre-existing failures on this base, reproducing on
both NumPy 1.26.4 and 2.4.6; tracked separately.
The rest of their files still run.
Requires the lazy trapezoid resolution in the parent commit: test_session_consume
imports eeg_spectral, which could not import on numpy>=2 before it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(eeg): resolve NumPy trapezoid compatibility lazily
ci: run the Python contract suites
Addresses both review blockers on #35. The module's read and publication contracts were weaker than its description claimed. Replay now binds records to their manifest ------------------------------------------ `records_sha256` only proves the records are the ones the manifest was written for — not that they agree with it. After recomputing that hash, a record could still claim a different encoder or dataset than the manifest it shipped with, and replay accepted it. The loader now: - validates manifest["encoder"] through _validate_encoder_provenance - requires record["encoder"] to equal it - requires each record's dataset_sha256 and source_manifest_sha256 to equal the manifest's - validates raw_window_sha256 as a lowercase hex digest - recomputes each state_id from dataset_sha256 + raw_window_sha256 rather than accepting any unique 64-character string state_id derivation moved into a shared helper so the writer and the replay check cannot drift apart. Publication commits the pair, not each file ------------------------------------------- Per-file os.replace still allowed a crash between the two writes to leave new records beside a stale manifest, destroying the previously valid pair. Both payloads are now built and validated before anything is written, and the manifest is written last as the commit marker. Republication is idempotent and non-destructive: byte-identical no-op, nothing written any difference refused, both files untouched records-only completable — this is the interrupted-publication repair, allowed only when the existing records match exactly Explicit replacement is deliberately not added until a workflow needs it. Identical state and manifest paths are rejected outright. Test paths were writing over the fixture ---------------------------------------- The immutability guard immediately caught that these tests published their state manifest to `root/manifest.json` — the same path test_pipeline._manifest writes the *source* manifest to. Every run had been silently clobbering the fixture. Artifacts now publish under `root/shadow/`. structured-state suite 13 -> 20 tests; NeuralComposeEEG suite 46 -> 53. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ntract feat(eeg): add fail-closed structured-state shadow artifacts
The structured-state replay test wrote its state manifest to root/manifest.json — the same path test_pipeline._manifest writes the *source* manifest to. That silently clobbered the fixture until #35 made the shadow bridge refuse to overwrite an existing artifact that differs. Same collision, same fix as #35 applied to its own suite: publish under root/shadow/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(review): add fail-closed local review cascade (clean replacement for #26)
cohens_d returned 0.0 whenever pooled standard deviation was zero, regardless
of whether the means were identical or completely separated. Two perfectly
separated constant groups therefore reported "no effect" — the least correct
available answer, and one that ranks alongside genuine null results.
Zero pooled variance now distinguishes the two cases:
equal constant groups 0.0 a true zero
separated constant groups NaN undefined, not absent
NaN rather than infinity: d is undefined without within-group variation, NaN
does not pretend the effect is absent, it matches the function's existing
under-sized-sample behaviour, and it forces callers to treat the estimate as
degenerate instead of ranking it as an ordinary finite effect.
Finishes a consolidation the module already claimed. eval_stats.py's docstring
says it consolidates cohens_d "previously duplicated in embedding_analyze.py
and statistical_analysis.py", but the duplicates were never removed — and
analyze_results.py had a fourth copy. All three now import the shared
implementation, following the established `sys.path.insert` + `from eval_stats
import ...` idiom already used for bootstrap_ci. Fixing one copy would
otherwise have left three wrong ones.
test_cohens_d_large_effect conflated "large" with "zero-variance degenerate";
it is replaced by three tests that separate identical-constant, separated-
constant, and large-non-degenerate cases.
Removes the first of three deselections in the python-contracts job (issue #38).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
No production change. The test asserted a p-value the test's own sample size
makes unreachable: with n1 = n2 = 3, the smallest attainable two-sided p is
2/20 = 0.1, so completely disjoint groups still return exactly 0.1 and no
correct implementation can clear 0.05.
Four per group makes the claim attainable — the exact minimum is 2/70 = 0.029 —
and the assertions now pin U and both sample sizes rather than only the
p-value.
n=3 per group U=0.0 p=0.1000
n=4 per group U=0.0 p=0.0286
Removes the second of three deselections in the python-contracts job (#38).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
No estimator change. cka() implements biased linear CKA over centered Gram
matrices, and that estimator is deliberately kept: switching to unbiased CKA
would alter every prior representation-comparison result and needs a registered
analysis migration, not a test cleanup.
The old expectation was wrong for it. Independent Gaussians at n=50 with
d=128/256 score ~0.78, because the biased estimator's null is inflated when
feature dimension approaches or exceeds sample count. Asserting < 0.3 there
assumed an unbiased estimator or n >> d.
Replaced with two tests pinning both regimes, so neither the estimator nor the
inflation can change unnoticed:
n=500, d=16/32 0.042 independent scores near zero
n=50, d=128/256 0.785 characterizes the inflated null
The second is explicitly labelled as characterizing the estimator, not as
evidence the representations are similar.
cka() gains a docstring stating the inflation and warning against comparing
scores across n/d regimes without a matched null or permutation baseline.
Removes the last of three deselections in the python-contracts job. The eval
gate now runs with zero deselections; closes #38.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ohens-d fix(eval): stop zero-variance Cohen's d from reporting no effect
…ignificance test(eval): use an attainable exact Mann-Whitney significance case
test(eval): pin biased linear CKA behavior across sample regimes
The deterministic Swift step names five targets. Package.swift declares nine, and two of the omitted ones carry PR #32's load-bearing regressions: NeuralComposeAppTests 116 tests fail-closed runtime, presentation DialecticSessionTests 5 tests headless runtime factory 121 tests that CI has never executed, which is most of the gap between the 417 CI reports and the 616 in the historical acceptance record. Both pass on this base with 0 failures. A `--filter` matching nothing exits 0, so adding the targets alone would risk another green step that ran nothing. The guard asserts one sentinel class per target and floors each target's discovered count, since sentinels alone would still pass if the rest of a target vanished. Floors sit below the current 116/5 so ordinary additions never trip them. RuntimeIdentityPresentationTests is deliberately not asserted: it arrives with #32 and does not exist on this base. Grepping for it here would fail CI immediately. #32 adds it as a third sentinel when that file lands. BCILLMTests and WorldModelDemoTests remain unselected — MLX .eval() faults on a GPU-less runner, and the WorldModel demo needs exported CoreML artifacts. ci.yml stays under its named D0/CI lane owner rather than growing #32 by a twenty-second unrelated file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ci: run A2 app and session suites
`RuntimeReadiness` had one success case, and the two resolution paths reached
it on very different evidence.
Ollama GET /api/tags, exact requested model matched observed fact
Claude executable resolved, prompt loaded nothing contacted
Both rendered the word "Ready". On the Claude path an expired `claude login`,
an unreachable network, or a model the account cannot use all still resolved
as ready, because no request is made to Anthropic before enablement.
That left the failure side of the type strictly more precise than the success
side: six distinct reasons a runtime cannot be used, one undifferentiated
reason it can. `configured` closes the gap in the domain model rather than in
the view, so a future runtime's evidence semantics is a property of its
resolution, not an undocumented convention in `PrivacyIndicatorView`.
configured constructible, unverified Claude
ready endpoint + exact model verified Ollama
unavailable must not be used both
Two predicates, because the existing single one cannot answer both questions.
`isReady` stays strict; `canAttemptGeneration` covers configured|ready. The
distinction is load-bearing at `PrivacyIndicatorView.identityColor`, which
paints "not ready" red — keying it on `isReady` would have marked every Claude
session as a fault, trading the over-claim for its mirror image. The colour
answers "is this broken?"; the unverified/verified nuance lives in the line
text, where a nuance belongs.
A successful generation is *session* evidence and deliberately does not mutate
the identity. The identity records what resolution proved; rewriting it after
a good turn would erase the distinction this split exists to make.
Adding a case does not disturb the wire format — synthesized enum `Codable`
keys on the case name, not a declaration ordinal — but that is a property of
the compiler rather than of this type, so both legacy encodings are asserted
rather than assumed.
No existing assertion broke, which is itself the finding:
`testDefaultResolutionIsStillClaude` checked provider, model, substitution and
generator type and never looked at readiness, so the Claude success path's
readiness value was wholly unasserted. That is how the over-claim survived.
`RuntimeIdentityPresentationTests` fixture keyed its resolved fields on
`== .ready`, which would have handed a configured runtime the empty *failure*
shape; it now keys on `canAttemptGeneration`.
The two presentation tests were verified against the mutation they exist to
catch — branching `displayReadiness` on `resolvedProvider == "claude"` — which
fails both, in both directions. One asserts an Ollama-provider identity with
`configured` renders as configured; the other, a Claude-provider identity with
`ready` renders as verified. No provider-spelling implementation satisfies both.
deterministic suites 423 executed, 5 skipped, 0 failures
A2 app + session 147 executed, 0 failures
swift test list 627 discovered (was 619)
PR diff stays at 21 files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Commit H made the app path truthful. The headless harness still recreated the
exact ambiguity it removed.
`RuntimeReport.verify` returned `(promptLoaded, true, true)` for Claude, with a
comment stating plainly that no call had been made. So `--dry-run` printed:
transport reachable: yes
model available: yes
✓ transport reachable
✓ model exists
No LLM call was made.
Four claims of verification and, four lines later, an admission that nothing was
verified. The acceptance document had already read this correctly as
"configured only", but the executable emitted stronger evidence than it held,
and the harness is the surface an operator reads when there is no UI.
The defect is the boolean. Two states force a skipped check to be reported as
either a pass or a failure, and this path picked pass. `VerificationStatus`
adds the missing third state:
passed ran, condition held
failed ran, condition did not hold
notChecked did not run — no evidence either way
runtime prompt provider exact model
claude passed/failed notChecked notChecked
ollama passed/failed passed/failed passed/failed
`notChecked` is not failure. `hasFailure` is true only for an actual `failed`,
so a configuration-only Claude dry run still exits 0 — verified against the
built binary, which now prints:
Verification:
prompt loaded: yes
provider reachable: not checked — no generation request made
exact model present: not checked — no generation request made
✓ runtime configured
✓ prompt loaded
– provider and model were not operationally verified
✓ fingerprint generated
Ollama keeps every claim it earns; its branch really does probe /api/tags for
the exact model, and reports `✓ endpoint reachable` / `✓ exact model present`.
Wording moved out of `main.swift` into `verificationLines` / `dryRunSummaryLines`,
which return `[String]`. It was formatted inline at two call sites, which is
precisely why no test could assert on what it claimed — the same shape as R8's
privacy strings buried in a SwiftUI body.
Scope grows 21 → 23 files. `RuntimeReport.swift` and the new test file are
A2 runtime lane, not a D0-owned cross-lane file; the 21-file bound was a
reviewability guard, not a reason to keep shipping a false diagnostic.
A2 app + session 152 executed, 0 failures (was 147)
deterministic 423 executed, 0 failures
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A read-only audit of the 71c5c02 session logs found the packaged app's twelve dialectical turns carried no `generatorFingerprint` and no `witnessGeneratorFingerprint`, while harness runs of the same head carried both. The cause is an app-path asymmetry, established from source rather than inferred from the logs: ClaudeCLIGenerator : TextGenerating (only) GenerationRuntimeTextGeneratingAdapter : MetadataPublishing… (sole conformer) LiveRuntimeFactory.makeClaude → raw generator LiveRuntimeFactory.makeOllama → adapter HypnagogicDialecticLoop.swift:86 fingerprint is nil "when the generator is the legacy TextGenerating conformers" So the app could not durably attribute a Claude turn to its provider, model, or prompt hash, and the same gap hid the Claude Witness's identity. The absence was self-concealing: with no fingerprint there is no record of which provider produced the record. Ollama had provenance; the harness had provenance; the app's Claude path did not. Pre-existing rather than introduced by this PR — the Ollama adapter work is what made it visible. `makeClaude` now routes through the bridge the repository already had: `ClaudeCLIGenerationRuntime` wrapped in `GenerationRuntimeTextGeneratingAdapter`, matching the Ollama path. No second provenance mechanism. Prompt transmission is byte-equivalent. The `promptProfile:` initializer is used so metadata records the real profile instead of `custom`; it re-loads the profile, but `PromptProfile.load()` caches on `cacheKey` and the factory has already populated that entry, so the second load returns the identical `String`. The adapter carries no prompt of its own by construction — a field that once existed there was removed precisely because it let an object report one prompt while sending another. Failed calls publish nothing: metadata is constructed inside `ClaudeCLIGenerationRuntime.generate` only after `transport.send` returns, and the adapter fires `onMetadata` only after `generate` returns, so a throw short-circuits both. A success fingerprint on a failed turn would be the same fabricated evidence commit I removed from the harness. Readiness is untouched — Claude still resolves `.configured`, not `.ready` — and resolution remains side-effect free. `testDefaultResolutionIsStillClaude` asserted `generator is ClaudeCLIGenerator` and now asserts the *capability* instead: the concrete type is an implementation detail, the metadata conformance is the thing that was broken. LiveRuntimeFactoryFailClosedTests 31 executed (was 23) deterministic CI subset 431 executed, 3 skipped, 0 failures A2 app + session 152 executed, 0 failures ./Scripts/build.sh OK git diff --check clean Behavioural tests drive a stub `claude` that prints a canned JSON envelope; no test contacts Anthropic. §5 of the acceptance record documents the correction without rewriting §1–4. The 71c5c02 bundle hashes remain historical evidence for that head; new hashes follow after packaging. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Observed packaged evidence for abb0eea, appended as §6. Sections 1–5 and the 4c5e275 / 71c5c02 observations are unchanged. The day-file spans both bundles, so the Commit J change is directly visible rather than argued: written by 71c5c02 16 records 0 pole fingerprints written by abb0eea 12 records 12 pole fingerprints Cells: Claude Focused, Reflective and Contemplative PASS; Ollama Reflective regression PASS across four turns; configured-but-failing Claude PASS with zero records appended. Prompt-hash collisions 0; no provider substitution; Witness finding never voiced; Witness fingerprint present only for Reflective and independently recording its own profile. The failing-Claude cell was gated on a health-log advance before its zero delta could count as a pass — its expected result is that nothing is written, so "nothing changed" and "nothing ran" would otherwise be the same observation. Mirror is recorded as not established rather than passed or failed. Selection and loop construction are confirmed by loopMode/loopRunning; nothing after that leaves durable evidence, because turnCount and lastTurnAt are populated only for HypnagogicDialecticLoop and HypnagogicDialogueLoop's catch swallows failures without logging. Both predate Commit J, and the same adapter produced twelve fingerprinted turns in the same process, so it is not attributable to this change. No behavioural-fidelity claim is made for Focused, Contemplative, or Mirror. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A2: role-specific runtime identity and readiness (R3 + R8 + R18)
First step of the integration order surveyed on 2026-09-25. Brings the 88 commits of the PR #32 line (runtime identity, fail-closed runtime and prompts, the EEG methods scope, NeuralComposeEEG, the Rust prosody crate) onto origin/main at 576b04f. 9 files conflicted, in 10 hunks. Both lines fixed the same Cohen's d defect a day apart and chose differently. Resolved as follows. Evaluation/scripts/eval_stats.py main's implementation: variance pooled by degrees of freedom, and signed infinity when both groups are constant and differ. The other line pooled unweighted, which is wrong for unequal group sizes, and returned NaN. The one caller that reports effect sizes skips NaN rows, so NaN would drop a perfectly separated pair from the table, where infinity prints as large. This overrides an explicit choice in 5f2cf64. Evaluation/scripts/analyze_results.py Evaluation/scripts/embedding_analyze.py Evaluation/scripts/statistical_analysis.py the other line's structure: the local copies of cohens_d are gone and the shared one is imported, which main's e6cca11 had left to "the lane that owns that structure". main's _EXACT_MAX_N stays in the 2 scripts whose local mann_whitney_u uses it. Tests/eval/test_eval_stats.py main's tests, plus the other line's 3 that agree with the policy. Its NaN assertion is dropped. Its n=4 Mann-Whitney test is renamed test_mann_whitney_u_disjoint_four_per_group, since both lines used one name for different tests. Tests/eval/test_embedding_space.py all 4 tests from both lines. They pin different facts about the same biased estimator. Scripts/eeg_spectral.py docstring only; the code was identical. main's, which records why the bug survived. .github/workflows/ci.yml each line had added its own Python job. Both stay, with one responsibility each: python-eval runs Tests/eval with main's bounded numpy and scipy, and python-contracts runs NeuralComposeEEG. python-eval now installs torch and the NeuralComposeEEG requirements, because test_local_open_weight_review.py imports that package's test helpers, which import torch. The discoverability floor moves from 55 to 125, against 137 collected. The other line's A2 steps stay in build-and-test. README.md main's rewrite, unchanged. The other line's 30 added lines, 9 list entries linking architecture and science notes and the Rust Phase 0 section, are deferred to the README step that ends the integration. Every document they link to is in the tree. Verified on the merged tree: swift build --build-tests clean; 640 tests declared swift test 630 executed, 12 skipped, 0 failures, with MindMonitorOSCStreamTests (9 tests) and AppViewModelOSCSmokeTests (1) excluded; their fixes arrive at S4 and S7 pytest Tests/eval 137 passed, in an environment built from the workflow's install list NeuralComposeEEG 53 tests, OK Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
S1 merges the already-public
docs/eeg-methods-scopeline intomain, bringing the runtime-identity, fail-closed runtime, EEG methods, and CI changes together. The head is exactly98f1b6a31e6ef85e970947afa4cb1fa05d39205b; its parents are576b04f595329300578722f6fda5729e2e889551and14df54550c673ca0865ef4d185abd57c6fa8378e.This draft verifies the merged workflow on GitHub-hosted runners at the S1 checkpoint. No later integration commits are included. Leave this PR in draft for review.
Validation completed October 10, 2026 (America/Chicago), CI run 38112036504:
test_joint_embedding.py.Both Linux job logs confirm
torch-2.14.1+cpu. The macOS deterministic selection retains S1's explicitMindMonitorOSCStreamTestsexclusion. These results establish the S1 runner behavior; the local checks in the merge message remain separate historical evidence.