SRT — Semiotic-Reflexive Transformer (Adapter Architecture)

Meaning forks. SRT sees it.

SRT-Adapter is a lightweight module that bolts semiotic awareness onto any frozen causal language model. The backbone runs natively — its own embeddings, its own LM head, its own attention. SRT modules are small taps that read divergence from hidden states, track reflexive awareness, and optionally inject semiotic corrections back into the stream.

30-second TL;DR

• What: a ~12 M-parameter adapter that observes a frozen LLM at 3 layers and injects a FiLM correction at 2 of them, exposing per-token semiotic signals (divergence, reflexivity r̂, regime) plus a discourse/embedding vector.
• Why: lightweight, portable instrumentation for a frozen backbone — no base-model weight updates, zero cross-entropy degradation, trains in hours at ≈0.17 % of backbone params. The released v1.0 checkpoint targets semantic embeddings (MTEB-STS).
• How (one line): read divergence → integrate in a GRU → emit γ, β → h ← h·(1+γ) + β.
• Reading order (5 min): Architecture → Visual grammar → One-token trace.

Architecture

The base LLM stays fully frozen. SRT taps the residual stream at three layers (L7 / L14 / L21) into Metapragmatic Attention Heads, the GRU-based Reflexive Reasoning Module integrates the divergence stream into a meta-state, and a FiLM correction h ← h·(1+γ) + β is injected back into L14 and L21. A community head taps L2 for discourse basin, and BEN reads the RRM meta-state to emit per-token reflexivity r̂ and a regime label.

Full visual walkthrough with captions: docs/EXPLAINERS.md.

Key Ideas

1. Zero CE degradation — The backbone's native embeddings and LM head are untouched. Cross-entropy starts at pretrained quality (~3.5), not 200+.

2. ~14.6M trainable params — Only the semiotic modules train. The 7B backbone is fully frozen. Trains in hours, not weeks.

3. Unsupervised community discovery — A small encoder discovers discourse-trajectory structure from hidden state patterns. No hardcoded labels. As of v8a the encoder output is the community vector directly (continuous trajectory mode); earlier checkpoints used a 32-prototype soft-argmax readout that turned out to be a discriminability bottleneck (see arxiv/paper.md §5.8–§5.9).

4. Backbone-agnostic — Works with any HuggingFace AutoModelForCausalLM: Qwen, LLaMA, Mistral, Phi, Gemma, etc.

5. Portable — Save/load just the 44MB adapter weights. Attach to any compatible backbone at inference time.

Frozen-backbone probe — TruthfulQA-MC2 hidden-state detector

Scope note. This result is a separate diagnostic probe, not a capability of the SRT adapter. It trains LightGBM on features extracted from a single forward pass of the frozen backbone (no adapter, no fine-tuning) and is reported here only to characterize the backbone's hidden-state geometry. The SRT adapter's own side-channels (r̂, regime, divergence) are observational signals; they are not a validated hallucination detector, and on free-form generation they do not, on their own, separate hallucinated from faithful answers above chance. Treat hallucination detection as out of scope for the adapter.

Using frozen-backbone features plus LightGBM, this repo reaches the top of the published hidden-state-detector band on TruthfulQA-MC2, group-CV by question (n=817, 5882 paired choices):

backbone	params	LightGBM AUC
Gemma-2-2B	2B	0.8563 ± 0.016
Llama-3.2-3B	3B	0.8475 ± 0.013
Qwen-2.5-7B	7B	0.8656 ± 0.011

Reference band: SAPLMA ≈ 0.72, SAR ≈ 0.75–0.83, INSIDE ≈ 0.78–0.85, EigenScore ≈ 0.80–0.85.

Full protocol, ablations, and reproduction command in docs/TRUTHFULQA_RESULTS.md. Evaluator: scripts/evals/truthfulqa_v3.py. Artifacts: Qwen, Llama, Gemma.

Explainer series

A 10-figure visual walkthrough of SRT — pipeline, modules, anisotropy, losses, TQA ladder, cross-architecture results, and the demo map — with captions kept separate from the visuals so they can be repositioned.

See docs/EXPLAINERS.md. PNGs live in artifacts/explainers/ and re-render via python scripts/explainers/make_all.py.

Overview	MAH	TQA ladder	Cross-arch

Modules

Module	Purpose	Parameters
MAH (Metapragmatic Attention Head)	Detects where meaning diverges across positions	~2.7M × 3 layers
RRM (Reflexive Recurrent Module)	Tracks semiotic meta-state, injects corrections	~2.2M
BEN (Bifurcation Estimation Network)	Estimates reflexivity coefficient r̂ and regime	~0.2M
Community Head	Discovers discourse-trajectory structure unsupervised	~0.2M

Quick Start

# install
git clone https://github.com/space-bacon/SRT.git
cd SRT
pip install -e .

Run inference (frozen Qwen-7B + released adapter)

from srt.adapter import SRTAdapter
from srt.config import build_config_from_json
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer
import torch

repo = "RiverRider/srt-adapter-v1.0"          # or RiverRider/srt-adapter-v8a
cfg  = build_config_from_json(hf_hub_download(repo, "config.json"))
adap = SRTAdapter(cfg).cuda().eval()
adap.load_state_dict(load_file(hf_hub_download(repo, "adapter.safetensors")), strict=False)
tok  = AutoTokenizer.from_pretrained(cfg.backbone_id)

enc = tok("meaning forks here", return_tensors="pt").to("cuda")
with torch.no_grad():
    out = adap(input_ids=enc.input_ids, attention_mask=enc.attention_mask)
print(out.r_hat.mean().item(), out.community_output.encoded.shape)

See examples/ for end-to-end loading, scoring, and sentence-encoding scripts.

Live demos

• v1.0 demo: https://huggingface.co/spaces/RiverRider/srt-adapter-v1.0-demo
• v8a demo: https://huggingface.co/spaces/RiverRider/srt-adapter-v8a-demo

Train from scratch

python scripts/train.py \
    --backbone Qwen/Qwen2.5-7B \
    --train-data data/all_train.jsonl \
    --val-data   data/all_val.jsonl \
    --output-dir checkpoints/adapter_v1 \
    --batch-size 16 --epochs 3 --lr 3e-4 --max-val-samples 5000

Resume from a saved training_checkpoint.pt with --resume <path> (restores optimizer, scheduler, step, epoch).

Training Diagnostics

Every --log-every steps, the training script logs standard loss metrics plus semiotic diagnostics:

Diagnostic	What It Shows	Healthy Range
div_norms	MAH divergence vector L2 norms per hook layer	> 0.1 (not collapsed)
inj_norms	RRM injection magnitudes at each injection point	~1.0 (target norm)
r_hat_mean±std	BEN reflexivity predictions — distribution spread	std > 0.1 (not saturated)
r_hat_min/max	Range of r̂ across the batch	Should span [-1, 1]

Red flags to watch for:

• div_norms → 0: divergence vectors collapsed, MAH not learning
• r_hat_std < 0.05: BEN stuck in trivial constant prediction
• inj_norms > 5: injection regularization not constraining norms (fixed in v3)
• CE climbing steadily: injections corrupting backbone representations
• Chain loss exactly 0.0: divergence collapsed to a constant

Checkpointing

The training script saves:

• training_checkpoint.pt — full state (adapter weights + optimizer + scheduler + step + epoch) at every validation step, for seamless resumption
• best_adapter.pt — adapter weights only, at best validation loss
• adapter_epoch{N}.pt — adapter weights at end of each epoch
• final_adapter.pt — adapter weights at end of training
• train_log.jsonl — all metrics + diagnostics in structured format

SRT-NLA — Stage 4 of the SRT program

Activation verbalization — read any hidden state of a frozen backbone as a sentence.

SRT-NLA is the fourth stage of the SRT program:

1. Stages 1–2 — semiotic theory and pretraining-time architecture (Lancaster, 2025 [SSRN 5987495]; Lancaster, 2026a [SSRN 6349978]).
2. Stage 3 — frozen-backbone bolt-on adapter (the SRT-Adapter manuscript under arxiv/; repository-hosted, not yet on arXiv).
3. Stage 4 — NLA: a small (~12.7M-param) Activation Verbalizer (AV) is trained so that given a target mid-layer hidden vector v from a fully frozen backbone, it generates text whose own re-encoded activation h matches v under an anisotropy-corrected metric fve_nrm_cen = ½(1 + cos(h−μ, v−μ)), normalised against a random-text floor and a same-source paraphrase ceiling to give a backbone-agnostic ρ_norm ∈ [0, 1].

The full Stage-4 framing — Peircean interpretant completion, Kockelman sieving, Silverstein metapragmatic awareness as decoding capacity, and the substrate-asymmetry hypothesis (text port strong / hidden-state port weak) — lives in paper_nla.md §1.5 and §12.

• Paper (Stage 4): paper_nla.md
• Release notes (Qwen v1): RELEASE_NOTES_NLA_v1.md
• Forward plan: FORWARD_PLAN.md
• Mission & stakes (historical, superseded units): docs/nla_mission.md
• Architecture & phased plan: docs/SRT_NLA_PLAN.md

Headline numbers

ρ_norm is anchored at 0 = random unrelated text, 1 = a same-source paraphrase. Best-of-K is oracle rerank against the target v, which is available at deploy time.

Backbone	Layer	Greedy ρ_norm	Best-of-64 ρ_norm
Qwen2.5-7B	L20	0.26	0.92
Llama-3.2-3B	L19 (73% depth)	reported in paper_nla.md §10	reported in §10
Gemma-2-2B	L19 (73% depth)	0.30	1.33 (overshoots paraphrase ceiling)

The greedy-vs-rerank gap is the central artifact of the program: the verbalizer can express paraphrase-quality outputs, but argmax does not surface them on the first try. The bag-of-K self-distillation attempt to close that gap (Lever B) returned a clean negative result (paper_nla.md §6); deploy-time best-of-K oracle rerank (Lever A) remains the only mechanism that closes the gap on this backbone.

HF artifacts

Backbone	Model	Targets dataset
Qwen2.5-7B	RiverRider/srt-nla-av-v1	RiverRider/srt-nla-targets-v1
Llama-3.2-3B	RiverRider/srt-nla-av-llama32-3b	RiverRider/srt-nla-targets-llama32-3b-v1
Gemma-2-2B	RiverRider/srt-nla-av-gemma2-2b-v1 (release pending)	RiverRider/srt-nla-targets-gemma2-2b-v1 (release pending)

Reproducibility caveat

A bug in scripts/sample_targets.py (Qwen2.5 sets bos_token_id == eos_token_id == 151643, which caused the BOS prompt to register as the first EOS and collapsed every target activation into one constant vector) was fixed on 2026-05-16 (commit 902b746). All NLA-branch results before that date are invalidated. The Llama-3.2-3B and Gemma-2-2B sample paths do not have this trap. The released SRT-Adapter checkpoints (v1.0 / v8a / v18 / v21a / v22c_a050) are on a separate codepath and are unaffected.

srt_introspect — adaptive-density reasoning trace (Stage 3 + 4 sidecar)

srt_introspect is a read-only product wrapper that turns the trained adapter (Stage 3) and the activation verbalizer (Stage 4) into a single generate-with-trace call. It produces text alongside a non-uniform trace: dense narration where the model's internal state is moving fast (high MAH divergence) and sparse where it's coasting.

from srt_introspect import Trace

t = Trace.load()  # defaults: RiverRider/srt-adapter-v1.0 + RiverRider/srt-nla-av-v1
result = t.generate(
    "Q: What killed the dinosaurs?\nA:",
    max_new_tokens=200,
    budget=12,   # adaptive verbalization slots
    k=8,         # AV samples per slot (consensus)
)
print(result.text)
for s in result.selected():
    print(s.token_idx, repr(s.token), s.divergence, s.regime, "→", s.verbalization)

Each Step carries token_idx, token, divergence, regime, r_hat, verbalization (the last populated only for scheduler-selected sites). The scheduler (srt_introspect.scheduler.quantile_by_density) places verbalizations at equal-mass quantiles of the per-token divergence series so the trace density tracks where the model's metapragmatic state is changing.

Throughput on an RTX Pro 6000 Blackwell:

op	latency
Trace.load (cold)	~10s
adapter forward (10-tok prompt)	21 ms
AV verbalize, K=8 (32 new tok)	770 ms
120-token generation + 8 verbalizations	~7 s

CLI demo, JSON dump, and self-contained HTML viewer:

PYTHONPATH=. python scripts/demos/trace_demo.py \
    --max-new-tokens 200 --budget 12 --k 8 \
    --out-json artifacts/trace.json
PYTHONPATH=. python scripts/demos/render_trace_html.py \
    artifacts/trace.json --out artifacts/trace.html

Mini benchmark over a small prompt set:

PYTHONPATH=. python scripts/evals/trace_bench.py \
    --out-dir artifacts/trace_bench

Theoretical Foundation

SRT is grounded in C.S. Peirce's semiotics. Language models process signs (representamens) but are blind to when meaning forks — when the same word means different things to different communities. SRT makes the model reflexively aware of its own semiotic processing:

• MAH implements metapragmatic awareness: detecting that "freedom" carries different interpretive weight in libertarian vs. socialist discourse.
• RRM implements reflexive recursion: the model's awareness of its own awareness, tracking how divergence propagates through the interpretant chain.
• BEN estimates the bifurcation point: where a sign tips from stable (subcritical) to contested (supercritical) interpretation.

See Lancaster (2025) — the full SRT-Adapter manuscript and its LaTeX source live under arxiv/ (paper.md, paper.tex, paper.pdf). The folder name is forward-looking: this manuscript is not yet on arXiv. The only currently posted Lancaster preprints are the two SSRN entries cited from paper_nla.md §14 (SSRN 5987495, SSRN 6349978); the SRT-Adapter and SRT-NLA manuscripts are repository-hosted at the time of writing.

Versioning policy

Two tiers exist on Hugging Face:

• Stable product release. RiverRider/srt-adapter-v1.0 is the only checkpoint we recommend pinning from external code, papers, or downstream products. Semver applies to this lineage going forward (v1.0, v1.1, v2.0, ...).
• Research checkpoints. Every other repo of the form RiverRider/srt-adapter-vNNx* (e.g. v8a, v18, v21b_a070, v22c_a050, v23*) is an internal research-iteration release. Weights are open under Apache-2.0 for reproducibility of paper results, but the labels are research generations, not versions in the semver sense — mentally, these are v0.8a, v0.18, v0.22c_a050, etc. They may be moved, retired, or renamed without notice.

If you are integrating SRT into a product (including RiverRider/zooL4nD3r-v0.1), pin srt-adapter-v1.0.

Released checkpoints

Repo	Tier	Notes
RiverRider/srt-adapter-v1.0	Stable release	First semver release. Use this for downstream pinning. (Internal lineage: v15a.)
RiverRider/srt-adapter-v8a	Research checkpoint	Encoder-as-community headline result (Reddit recall@1 0.484). Paper §5.9.
RiverRider/srt-adapter-v18	Research checkpoint	CoSENT supervised STS, English-purist tier. Paper §5.14.
RiverRider/srt-adapter-v21a	Research checkpoint	mxbai-distilled CoSENT, multilingual-leaning. Paper §5.14.
RiverRider/srt-adapter-v22c_a050	Research checkpoint	Souping v18 + v21a at α=0.5; MTEB-STS SOTA (mean 0.3744). Paper §5.14.

Citation

@misc{lancaster2025srtadapter,
  title  = {The Semiotic-Reflexive Transformer Adapter: Lightweight Semiotic Awareness for Frozen Causal Language Models},
  author = {Lancaster, Burton},
  year   = {2025},
  url    = {https://github.com/space-bacon/SRT},
}

See CITATION.cff for machine-readable metadata.

License

Apache-2.0 — see LICENSE. The released adapter weights on Hugging Face are also Apache-2.0; the underlying Qwen/Qwen2.5-7B backbone is released under its own Qwen license, which applies whenever the backbone is loaded.