ARCHITECTURE.md

Hematite Architecture

This is the contributor map of Hematite's current runtime boundaries.

The goal is simple:

• conversation.rs should orchestrate turns
• specialized modules should own their own policy or explanation logic
• the TUI should render runtime truth, not invent it
• the local model runtime should remain the model runtime, not the product brain
• recurring workflow structure should live in the harness when local models are not dependable enough to infer it every turn

Runtime State Anchoring

Hematite resolves a runtime-state directory instead of assuming every artifact lives directly under the launch directory.

• normal project workspaces use workspace_root()/.hematite/
• OS shortcut directories such as Desktop, Downloads, Documents, Pictures, Videos, and Music use the global ~/.hematite/ directory instead
• this keeps broad user folders clean while preserving docs-only memory, settings, reports, scratch output, and teleport/session state across launches

If you are changing where session files, Vein artifacts, reports, logs, or workspace metadata live, start from src/tools/file_ops.rs and its hematite_dir() / is_os_shortcut_directory() helpers.

Top-Level Entry Points

src/main.rs

CLI entry point. Parses flags (--no-splash, --yolo, --rusty, --brief, --stats, --teleported-from), then delegates to src/runtime.rs.

src/runtime.rs

Owns runtime assembly and startup.

• builds the typed runtime bundle
• wires agent channels, watcher channels, voice, swarm, and runtime profile sync
• resolves the workspace root, loads config, launches the Vein, and spawns the agent loop
• handles the CWD guard (relocates to home if launched from an inaccessible system path)
• Teleportation Handshake: Initializes the session with a context-aware greeting when arriving from a previous workspace.

If you are changing startup ownership, channel plumbing, or steady-state runtime boot, start here.

src/lib.rs

Public crate surface for integration testing and diagnostics.

---

Agent Layer — src/agent/

src/agent/conversation.rs

Owns turn orchestration. The main loop of Hematite.

• handles user turns and slash-command flow
• assembles prompts via prompt.rs
• applies workflow mode policy (/ask, /code, /architect, /read-only, /teach, /chat)
• runs harness pre-run orchestration (multi-topic inspect_host before the model turn)
• injects loop_intervention for computation routing, shell-block recovery, Deno parse error recovery, and repeat-tool guards
• coordinates tool execution, verification, compaction, recovery, and final output
• Heuristic Command Sanitizer: Enforces a mandatory gate that blocks AI-generated natural language sentences from being passed as shell command arguments.
• Turn-Loop Circuit Breaker: Tracks tool calls within a single turn to terminate repeated (duplicate) calls and prevent infinite reasoning loops.
• drives session persistence via save_session() / load_checkpoint()

This file should not drift back into being a tool registry, product-truth catalog, or giant policy dump.

src/agent/routing.rs

Owns query intent classification.

• classifies stable product-truth questions (identity, capability, mode questions)
• identifies routing classes: architecture, runtime diagnosis, computation sandbox, toolchain questions
• Diagnostic Interception Matrix: coordinates the 'Workstation Doctor' redirection logic, mapping raw shell commands to internal high-precision inspect_host topics
• Intent-Based Tool Pruning: Dynamically hides risky workflow tools (shell, swarm, etc.) when a user's query is classified as a surgical filesystem or diagnostic mutation, forcing the model to use deterministic tools.
• needs_computation_sandbox() — detects math/hash/financial/statistical/date queries and triggers pre-turn nudge toward run_code
• keeps prompt-shaped routing logic out of the main turn loop

src/agent/direct_answers.rs

Owns stable product-truth responses (zero model tokens).

• identity and authorship answers
• workflow mode explanations
• Gemma-native settings explanations
• session memory policy
• recovery-recipe explanations
• MCP lifecycle explanations
• tool-class and tool-registry explanations

If a behavior is stable product truth and should answer with Tokens: 0, it belongs here.

src/agent/prompt.rs

Owns system prompt assembly.

• builds the full system prompt per turn from workspace mode, Vein results, repo map, hot files, session memory, and workspace profile
• injects per-project rules from instruction files
• assembles the L1 hot-files context block and PageRank repo map injection
• manages workspace mode detection (Coding vs. Document)

src/agent/inference.rs

Owns the model and tool protocol surfaces.

• InferenceEngine — HTTP client to the active OpenAI-compatible runtime, streaming, tool calls
• chat message types (ChatMessage, role handling)
• InferenceEvent — the enum flowing from agent to TUI over mpsc
• tool definitions and tool metadata (ToolMeta)
• provider/runtime event flow (typed provider states: live, recovering, degraded, context-ceiling)
• prompt preflight and runtime profile sync (loaded_context_length when the provider exposes it)
• Gemma 4 native markup wrapping (controlled by gemma_native_auto)

Tool metadata should continue to live here or in adjacent registry-owned code, not leak back into ad hoc name lists.

src/agent/tool_registry.rs

Owns the built-in tool catalog and dispatch.

• built-in tool definitions
• builtin dispatch path

conversation.rs should consume the registry, not act like a second registry.

src/agent/architecture_summary.rs

Owns architecture-overview shaping.

• project-map and runtime-trace summary shaping
• architecture-overview assembly for grounded architecture questions
• read-only architecture batch pruning (prevents redundant tool calls on repeated architecture questions)

src/agent/parser.rs

Owns Swarm workload parsing.

• resilient XML-ish parser for LLM output (handles trailing commas, broken escapes)
• maps model output to typed WorkerTask and Hunk structs for swarm dispatch

src/agent/swarm.rs

Owns parallel worker agent coordination.

• SwarmCoordinator — spawns parallel worker agents for multi-file or multi-step tasks
• dispatches worker tasks from parser.rs output
• collects results and routes them to the diff review modal
• triggered via /swarm

src/agent/specular.rs

Owns the SPECULAR panel event source.

• filesystem watcher via notify — fires events when workspace files change
• emits watcher events and shell-line events to the TUI SPECULAR panel
• provides side-panel content for reasoning traces and live activity

src/agent/compaction.rs

Owns compaction and session carry-forward.

• recursive summary compression when context pressure mounts
• compaction thresholds and deduplicated summary normalization
• typed session ledger: checkpoint, blocker, recovery step, verification result, compaction metadata carry-forward
• budgeted recursive summaries clamped to real line/character limits

src/agent/recovery_recipes.rs

Owns typed runtime recovery planning.

• named recovery scenarios: provider degraded, context window, prompt-budget pressure, history pressure, MCP workspace read blocked, proof-before-edit blockers
• recovery plans are explicit runtime policy, not buried in ad hoc branches

src/agent/policy.rs

Owns tool-policy helper logic.

• destructive-tool classification
• path normalization (resolving Path Alias shortcuts like @DESKTOP)
• MCP mutation/read helper checks
• target-path extraction

Keeps low-level policy helpers out of conversation.rs.

src/agent/permission_enforcer.rs

Owns typed authorization decisions.

• outcomes: Allow, Ask, Deny
• inputs: workflow mode, workspace trust, shell rules, trust sensitivity, tool metadata

src/agent/trust_resolver.rs

Owns workspace trust state.

• states: trusted, require-approval, denied
• trust affects destructive or external actions, not normal repo reads

src/agent/mcp.rs

Owns MCP transport and framing.

• stdio MCP transport
• newline-delimited and Content-Length-framed protocol support
• TUI-safe process handling (MCP stderr captured in memory)

src/agent/mcp_manager.rs

Owns MCP server lifecycle and discovery.

• loads mcp_servers.json from workspace and global scope
• typed MCP lifecycle states: unconfigured, healthy, degraded, failed
• resolves Windows launcher wrappers (npx, .cmd, .bat)

src/agent/lsp/

Owns LSP server integration.

• lsp/client.rs — LSP protocol client
• lsp/manager.rs — language server process lifecycle and diagnostics collection
• surfaced via /lsp in the TUI

src/agent/config.rs

Owns runtime config loading.

• loads and merges workspace .hematite/settings.json and global ~/.hematite/settings.json
• workspace values win; global fills missing fields
• covers api_url, model, voice, gemma_native_auto, context settings, and verify-build profiles

src/agent/economics.rs

Owns session economics tracking.

• tracks token usage and tool calls per session
• feeds the session report written on exit/cancel

src/agent/pricing.rs

Owns model pricing tiers.

• USD-per-million-token cost table for known models
• used by economics to estimate session cost

src/agent/git.rs

Git helpers used by the agent layer (branch, status, short log).

src/agent/git_context.rs

Reads a short git status summary (branch + changes) injected into context at turn start.

src/agent/git_monitor.rs

Background git state monitor — tracks uncommitted changes via atomic flags so the TUI badge stays current without blocking the turn loop.

src/agent/hooks.rs

Owns hook configuration and dispatch.

• loads hook definitions from .hematite/hooks.json
• fires pre/post tool hooks as shell commands

src/agent/instructions.rs

Owns project instruction discovery.

• walks up from the workspace root looking for instruction files (.hematite/AGENTS.md, CLAUDE.md, etc.)
• deduplicates and injects discovered rules into the system prompt

src/agent/transcript.rs

Owns session transcript persistence.

• persistent transcript logger for the DeepReflect engine
• writes transcript logs under the resolved runtime-state logs/ directory

src/agent/workspace_profile.rs

Owns the auto-generated workspace profile.

• detects stack, package managers, important folders, noise folders, and build/test suggestions
• uses robust project-marker detection (e.g., index.html, style.css, .hematite management files) to distinguish Managed Projects from Docs-only folders
• emits typed runtime contracts for stacks that need more rigid control loops, such as websites with local URL hints, route hints, and preferred website_* workflows
• written to workspace_profile.json under the resolved runtime-state directory on startup
• injected into the prompt as lightweight repo grounding
• inspectable via /workspace-profile

src/agent/utils.rs

Shared agent utilities (ANSI stripping, text normalization helpers).

---

Memory Layer — src/memory/

src/memory/vein.rs

Owns The Vein — local RAG memory engine.

• SQLite FTS5 BM25 full-text retrieval (always available, zero GPU cost)
• semantic embedding retrieval via /v1/embeddings (optional, requires embedding model in LM Studio)
• hybrid ranking: semantic hits score 1.0–2.0, BM25 fills to 0.0–1.0
• indexes project source files plus the resolved runtime-state docs/, reports/, and imports/
• database at vein.db under the resolved runtime-state directory
• incremental indexing (only re-indexes changed files by mtime)
• active-room bias: tracks file edit heat and boosts retrieval toward the hot subsystem
• memory-type tagging: chunks tagged as decision, problem, milestone, preference for intent-matched retrieval
• status badge: VN:SEM (semantic active), VN:FTS (BM25 only), VN:DOC (docs-only outside a project)

src/memory/repo_map.rs

Owns PageRank-powered repo maps.

• tree-sitter AST indexing across all source files
• petgraph PageRank to rank files by structural importance
• heat-weighted personalization: hottest files get score boosts so actively edited central files float to the top
• injected into the system prompt each turn so the model knows architecture without burning tool calls

src/memory/deep_reflect.rs

Owns idle-triggered session memory synthesis.

• fires after 5 minutes of TUI inactivity
• reads the day's transcript and calls the local model to extract structured memories: files changed, decisions made, patterns observed, next steps
• outputs written to .hematite/memories/<YYYY-MM-DD>.md
• injected into the system prompt at startup as persistent session context

---

Tools Layer — src/tools/

src/tools/mod.rs

Tool registry and dispatch. Routes tool calls from the agent to the correct implementation.

src/tools/file_ops.rs

File listing, reading, writing, and project mapping. Core file inspection tools.

src/tools/file_edit.rs

Targeted editing helpers — edit_file, patch_hunk, multi_search_replace.

• CRLF→LF normalization before matching
• fuzzy match escalation: rstrip-only → full-strip → cross-file hint
• delta-corrected indentation on fuzzy matches

src/tools/shell.rs

Shell execution.

• execute_streaming — streams each stdout/stderr line to the SPECULAR panel via InferenceEvent::ShellLine
• execute_blocking — blocking execution for background tasks
• blocked for computation tasks when needs_computation_sandbox fires (redirects to run_code)

src/tools/code_sandbox.rs

Sandboxed code execution (run_code tool).

• Deno sandbox (JS/TS): --deny-net --deny-env --deny-sys --deny-run --deny-ffi --allow-read/write=.
• Python sandbox: blocked socket/subprocess/dangerous imports, clean environment
• Hard timeout: 10s default, up to 60s
• Deno detection order: LM Studio bundled copy → system PATH
• 16 KB output cap; scratch file overflow for large results

src/tools/host_inspect.rs

SysAdmin and Network Admin inspection (inspect_host tool). 115+ read-only topics covering the full OS stack. Recent additions include hyperv (Hyper-V role, VM inventory, switches, checkpoints, RAM overcommit), event_query (targeted Windows Event Log filtering by Event ID, log, level, provider, and time window), identity_auth (Microsoft 365 token-broker, WAM, device-registration, and account-mismatch triage), windows_backup (File History, wbadmin, System Restore, OneDrive KFM), teams (Teams health, cache, WebView2, device binding), outlook (Outlook health, OST/PST, add-ins, auth), browser_health (Edge/Chrome/Firefox/WebView2 health), latency (ping RTT/packet loss), network_adapter (NIC offload/link-speed/errors), credentials, tpm, and installer_health on top of the earlier camera, sign_in, display_config, ntp, cpu_power, docker_filesystems, wsl_filesystems, lan_discovery, audio, and bluetooth work. See CLAUDE.md for the complete topic reference.

src/tools/guard.rs

Safety checks for risky shell actions.

• blocks destructive commands
• whitelists safe read-only diagnostics (get-counter, Get-Item, Test-Path, Select-Object, arp -a, etc.)
• redirects structured diagnostic commands to inspect_host topics

src/tools/git.rs

Git tool implementations — commit, push, branch, diff, log, worktree.

src/tools/git_onboarding.rs

Git remote configuration helper — configures or updates a Git remote and optionally performs an initial push.

src/tools/verify_build.rs

Build validation tool.

• runs per-project build, test, lint, and fix profiles from .hematite/settings.json
• falls back to stack autodetection
• exempt from the repeat-tool guard (fix-verify loops are legitimate)

src/tools/research.rs

Web research tools.

• research_web — Jina Reader/Search for technical information
• fetch_docs — pulls external docs into readable form
• rate-limited; gracefully handles missing JINA_API_KEY

src/tools/runtime_trace.rs

trace_runtime_flow — grounded read-only runtime/control-flow inspection. Gives the model a verified path for exact architecture questions instead of confident guessing.

src/tools/toolchain.rs

describe_toolchain — verified read-only map of Hematite's built-in tools, when to use them, and what investigation order makes sense.

src/tools/vision.rs

vision_analyze — base64-encodes images and passes them to the model via the multimodal vision path. Used for screenshot/diagram analysis.

src/tools/lsp.rs / src/tools/lsp_tools.rs

LSP startup and language-aware tooling. Surfaces diagnostics from language servers.

src/tools/plan.rs

PlanHandoff — persists architect session plans to .hematite/PLAN.md for /architect → /implement-plan handoff and mirrors durable project plans into docs/exec-plans/active/ with archive/carry-forward support.

src/tools/tasks.rs

manage_tasks — persistent TODO list for the agent in .hematite/TASK.md. Actions: list, add, update, remove.

src/tools/health.rs

Quick workspace health check — file count, source structure summary. Surfaced via /health.

src/tools/repo_script.rs

Workspace script runner — executes project-local build/test/lint/clean scripts with a hard timeout and output cap. Used by the workspace workflow lane.

src/tools/workspace_workflow.rs

Structured workspace workflow invocations — rooted to the locked workspace root. Separate from Hematite's own maintainer scripts.

• generic lane: build/test/lint/fix/package scripts/task/just/make/script path/exact command
• website lane: website_start, website_probe, website_validate, website_status, website_stop
• website validation consumes the workspace runtime contract when present so local URL hints and route hints can come from the repo instead of the prompt

src/tools/scoping_tools.rs

Context scoping helpers — auto_pin_context lets the model lock 1–3 core files into prioritized memory for the current turn.

src/tools/risk_evaluator.rs

Swarm risk triage — classifies worker actions as LOW (auto-approve), MODERATE (log warning), or HIGH (Red Modal). Eliminates approval prompts for safe file reads while enforcing hard gates on destructive operations.

src/tools/tool.rs

RiskLevel and shared tool types.

src/tools/tool_schema_cache.rs

Static tool schema cache (OnceLock) — avoids reserializing tool definitions on every turn.

---

UI Layer — src/ui/

src/ui/tui.rs

Owns the operator interface.

• main transcript rendering (chat surface)
• SPECULAR panel: live reasoning traces, shell output lines, watcher events
• bottom status bar: provider badge, VN badge, BUD/CTX meters, session error count
• runtime badges: provider health, compaction pressure, context ceiling
• approval prompts and diff preview modal (Y/N)
• voice toggle state
• Directory-Aware Autocomplete: @ file autocomplete in the input field, prioritized by type and optimized with Smart Splicing for Path Aliases
• Mouse Interaction: Autocomplete suggestions and status badges are clickable
• all slash-command UI flows
• Self-Destruct Protocol: coordinates clean workspace transitions by spawning a new terminal instance and breaking the original event loop.

The TUI renders typed runtime truth from the agent/runtime layer. It is not the source of truth for provider health, recovery, or policy state.

src/ui/voice.rs

Owns the self-contained TTS pipeline.

• Kokoro ONNX model (311 MB) and voice styles (27 MB) embedded via include_bytes!
• ONNX Runtime 1.24.2 statically linked — no system DLL dependency
• 54 voices across 7 languages
• Ctrl+T to toggle, /voice to switch, speed/volume configurable in settings.json
• 1024-token speech buffer so audio is never lost during ONNX graph optimization on first load

src/ui/gpu_monitor.rs

Background VRAM polling.

• watches GPU memory usage and emits pressure signals
• can force brief mode or reduce swarm fanout under memory pressure

src/ui/modal_review.rs

Swarm diff review modal — presents per-worker diffs for operator approval before applying.

src/ui/hatch.rs

Companion soul/personality generation — /reroll hatches a new personality mid-session.

---

Supporting Files

src/telemetry.rs

Global session error counter (lightweight Mutex<i32>) — feeds the session error count badge in the status bar.

---

Practical Rules For Contributors

• If you are adding a stable explanation, prefer direct_answers.rs.
• If you are adding a new routing class or computation-routing pattern, prefer routing.rs.
• If you are adding a low-level approval/path/MCP helper, prefer policy.rs.
• If you are changing the built-in tool list or builtin dispatch, prefer tool_registry.rs.
• If you are changing typed authorization behavior, prefer permission_enforcer.rs and trust_resolver.rs.
• If you are changing architecture-overview shaping, prefer architecture_summary.rs.
• If you are changing the live turn loop itself, use conversation.rs.
• If you are adding a new inspect_host topic, add it to host_inspect.rs and document it in CLAUDE.md.
• If you are adding a new slash command, wire it in tui.rs and document it in CLAUDE.md.

What Not To Reintroduce

Avoid letting conversation.rs grow back into:

• a second tool registry
• a second policy registry
• a pile of direct-answer strings
• a second architecture summary formatter
• a home for dead compatibility wrappers

The product is strongest when each boundary owns one thing clearly.