eoc-biomes-compute

Unity GPU-compute biomes + a memory architecture spanning installations.

Successor to the biomes/Metaesthetica work in edge-of-chaos-unity-compute, split off so the workshop legacy code stays separate. Built around a TouchDesigner-orchestrated installation pipeline (sensors + Unity output mixed in TD), with a Python memory daemon that gives consecutive installations a persistent, queryable history.

Layout

eoc-biomes-compute/
├── Assets/
│   ├── Workspace/
│   │   ├── 10.0 Metaesthetica/   earlier Unity scenes + sims
│   │   ├── 11.0 Biomes/          shared biome engine (src/, docs/, TestScene)
│   │   ├── 11.1 CURRENTS Scene/  CURRENTS show — scene + curated assets/snapshots
│   │   ├── 11.2 SIGGRAPH Scene/  SIGGRAPH show — scene + leaner assets
│   │   └── Includes/             shared compute helpers / shaders
│   ├── Settings/                 Unity 6 Build Profiles (macOS)
│   └── {EasyButtons, HDRPDefaultResources, TextMesh Pro, ...}  Unity infra
├── Packages/                     UPM (klak.spout)
├── ProjectSettings/
├── memory/
│   ├── daemon/                   Python: folder-watch → sqlite + OSC
│   ├── td/                       TouchDesigner files (placeholder)
│   └── docs/                     OSC contract, schema notes
└── docs/
    ├── ARCHITECTURE.md           Unity runtime + memory system reference
    ├── migration.md              memory architecture plan
    ├── adr/                      architecture decision records
    └── sessions/                 per-session handoff logs

Getting started

Unity: open the project in Unity Hub (HDRP). First import regenerates Library/. One folder per show: 11.1 CURRENTS Scene/Scene_CURRENTS.unity (the active build scene) and 11.2 SIGGRAPH Scene/Scene_SIGGRAPH.unity; DAC minimal-test takes _DAC_interior.unity / _DAC_exterior.unity / _DAC_4k.unity (3840×2160 record target, biome rez 1024×576) under 11.2 SIGGRAPH Scene/assets/DAC_params/0-minimal-test/ (SimTimeline-driven 120 s cue list, per-scene record RTs); the TouchDesigner side of the DAC loop (StreamDiffusion attention-map guide, resolution-agnostic crop rect) lives in tools/touchdesigner/; quick validation in 11.0 Biomes/TestScene.unity. The shared sim engine (src/, docs/) lives in 11.0 Biomes/; per-show scenes carry only their curated assets/materials.

Local media (not in git): 11.2 SIGGRAPH Scene/data/ is git-ignored (~2.8 GB). Copy these in from your media backup before playing the DAC takes; without them the debug video and audio stay empty. The firing blob (StreamingAssets/biomes11/organoid_firing.f16) and neuron positions CSV are tracked.

• data/videos/TesMovie_1920x1152_25.0_PRORES_PRORES422LT_ALAC_0045.mov (1.2 GB, debug video in the DAC takes)
• data/videos/TesMovie_3100x1000_25.0_PRORES_PRORES422LT_ALAC_0046.mov (1.6 GB)
• data/SimulacraNaturaeNeuroEcologies125000_131000.wav (22 MB, audio for the 125000–131000 firing cut)

Memory daemon:

cd memory/daemon
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
memory-daemon --snapshot-dir "../../Assets/Workspace/11.1 CURRENTS Scene/assets/Snapshots" -v

OSC listens on :9100. /memory/ping and /memory/count are the v0 endpoints.

Concepts

• Memory across installations. Each exhibition run writes snapshots/events; future installations read from accumulated memory, creating a slow dialogue across time and venues.

• Folder as event log. Unity's SaveParams writes .asset files; daemon watches and indexes. Folder is source of truth, DB is a derived index — replay-from-folder always works.

• TD as orchestration hub. Sensors, Unity Spout/Syphon output, and termite/biome sims composite in TouchDesigner. Daemon talks OSC to TD; both decoupled from each other.

• Biological substrates as peer modality. Brain organoid spike data (CSV) and live plant biopotential (planned) feed into memory alongside Unity params and viewer-derived signals.

• Slow parameter crossfades. ParameterInterpolator (11.0 src/components/utils/) eases a sim's live params from current state through a queue of preset .asset waypoints, sim-step driven, per-param enable toggles, shortest-arc hue — for long-running installations. It also carries an optional umweltWaypoints queue, index-paired with the param waypoints (an umwelt-only queue is valid), crossfading the sim's live UmweltMapping clone toward each target over the union of source/target keys — reads/writes unique to one side fade in or out rather than popping — with its own umwelt.* toggles. ParameterInterpolatorGroup fans one Play/Stop/Skip across several interpolators (playOnStart, keepRunningOnSimReset rebases the leg clock on sim reset); SimTimeline.interpolatorGroup cues it via PlayInterpolators/StopInterpolators/SkipInterpolators (each timeline Play stops the group, or restarts it from waypoint 0 when it has playOnStart), and the timer runs without started sims — a sim started mid-leg gets the reached waypoint re-imposed. Design: docs/superpowers/specs/2026-06-07-parameter-interpolator-design.md.

• Timeline video cues. SimTimeline.externalReceiver gives the Timeline the same Play/Stop + cue ownership over an ExternalTextureReceiver's debug video clip that it has over firing playback: StartDebugVideo/PauseDebugVideo/StopDebugVideo cues drive the clip, Stop clears the output to transparent (the composite overlay reads straight through), and a recorded take can bring a video coda in at an authored sim second. The clip itself runs on wall-clock time, not the sim's fixed step — a caveat for anyone matching it to sim-driven visuals. Design: docs/superpowers/specs/2026-09-09-umwelt-interpolation-and-timeline-video-design.md.

• Termite simulation. TermiteSim (11.0 src/components/Sim/) — a neuron-coupled pheromone-stigmergy swarm ported from PDE_Nefeli_Termites. Builds persistent permeability mounds via a dedicated firing-gated build kernel (agent-authored topography — see Habitat partitioning below). Runs at 131 agents (1:1 with neurons): each termite keeps a coherent heading and its own fixed per-neuron-group turn-angle (turnAngleSpread), so the 131 streams each curve with a distinct character. Designs: docs/superpowers/specs/2026-06-07-termite-sim-design.md, docs/superpowers/specs/2026-06-11-termite-biome-features-design.md.

• Shared neuron firing (externally driven). All three sims seed agent i → neuron i%131 at the same labels_positions.csv positions and read one shared firing signal each step (faster movement + brighter trails). The firing values are a preprocessed float16 blob (131 neurons × 180000 frames; tools/firing_csv_to_f16.py converts the 729 MB source CSV); the playhead is external — another patch sends OSC /index <int> to scrub which frame fires (NeuronFiringSource), holding + decaying to quiet when silent. Design: docs/superpowers/specs/2026-06-08-osc-neuron-firing-design.md; decision: docs/adr/0006-osc-neuron-firing.md.

• Raster → biome channels (video as ecology). TextureChannelSeeder routes a whole raster — an ExternalTextureReceiver stream/debug video clip, or any texture — into biome channels every step, one route per R/G/B/A/luma component with its own gain and blend mode (Biome.SeedChannelFromTexture gained a swizzle). A colour video becomes up to three independent fields that agents perceive through their Umwelt mappings and the PDE then erodes/advects; the 10.0 "external influence" steer-off-a-texture path is not wired in 11.0. A receiver with blank output (debug clip stopped, or debug input off with nothing received) is skipped, so SetToward/MinToward routes never write zeros.

• Local firing playback. NeuronFiringPlayback plays an inclusive blob frame range over a wall-clock duration (e.g. Shanghai cut 125000–131000 over 120 s, ≈50 fps) through the same SetFrame entry OSC /index uses — rehearse a segment without TouchDesigner. Pure timing in FramePlayhead (Biomes.Core, unit-tested).

• Shared chemical field (Biome). A 15-channel GPU field (nutrient, three species pheromones, oxygen, temperature, waste, permeability, flow x/y, dispersal, humidity, humidity-gradient, plus two CA-owned substrate channels) that every sim reads and writes through its per-species UmweltMapping. The field runs a live PDE each step — temperature gradients generate flow, flow advects the chemicals, waste decomposes into nutrient, temperature evaporates humidity, then everything diffuses and decays. Architecture: docs/ARCHITECTURE.md §3.3.

• Habitat partitioning (permeability mounds). Permeability (ch7) is no longer static terrain — it starts uniform-open and is authored entirely by termites, who lower it probabilistically (pulsing with neuron firing) into persistent walls that heal back only very slowly. Each species is then confined to its own preferredPermeabilityMin/Max band — Boids to open ground, Physarum to the mid transition at wall edges, Termites to the solid interiors — via a perception habitat gate (out-of-band → steer-back + a floored speed penalty). Because termites build the very structure that confines every species, habitat is emergent and successional; ResetTermites melts the walls and frees everyone. A post-composite overlay paints the mounds — from the coarse biome grid, or with SimulationManager.moundDetail from a sim-resolution mirror of the walls that draws the termites' actual paths instead of grid-sized blocks. Design: docs/adr/0010-permeability-agent-built-topography.md, docs/adr/0016-mound-detail-render-mirror.md.

• Humidity channel. A high-diffusion, flow-advected Humidity field that relaxes toward an ambient baseline (renewable) and is driven off by heat — humidity -= temperatureToEvaporation·max(0, temp−0.5) each step. Hot zones leave a drying wake; the steep |∇Humidity| edge they carve is the termite build cue, and the renewable moisture is a depletable resource sims can compete for via their UmweltMapping reads. Design: Assets/Workspace/11.0 Biomes/docs/INTEGRATION_DESIGN.md (Tier 2).

• Dispersal channel + firing scatter. A transient, fast-decay Dispersal field scatters all sims: they flee its gradient (negative chemotaxis) and accelerate out of it (a SpeedBoost Umwelt effect → dispersalSpeedMode, constant or multiplier). Neuron firing injects expanding Dispersal pulses via BiomeInjector — at fixed neuron positions or at the live termite positions — so a firing neuron blows its swarm outward; an expanding shockwave ring marks each pulse. The live-position path reads agent positions back via non-blocking AsyncGPUReadback (useAsyncReadback, default on) — removes the per-frame CPU↔GPU sync stall (the serialization barrier, not just the copy), a large win on both discrete GPUs and Apple Silicon; positions lag 1-2 frames. Design: docs/superpowers/specs/2026-06-11-termite-biome-features-design.md.

• Stream-safe resets. Sim/biome resets clear GPU state in place instead of reallocating, so the composite (and per-sim) Syphon/NDI/Spout streams keep a stable texture and aren't torn down mid-show — no downstream reconnect/flash. OSC reset commands are marshalled to the main thread. Decision: docs/adr/0008-clear-in-place-reset.md.

• Show blackout. OSC /sim_off [fadeSeconds] fades the whole composite to black (default 5 s, perceptual curve) and, with pauseWhenDark, stops stepping once dark; /sim_on [fadeSeconds] resumes and fades back in. Also as Output Off/On buttons on SimulationManager. organoid-analysis's show mode uses it for a 2 min blackout after each 50 min /index pass. Session: docs/sessions/2026-10-05-sim-off-blackout.md.

• Diurnal sun (environmental pump). A procedural BiomeInjector source sweeps a warm zone across Temperature L→R, phased off the neuron playhead so one firing-blob playthrough (/index 0→last) is one "day" and the index=0 loop reset lands on sunrise. Kept indirect: no sim reads Temperature directly — the moving heat re-aims flow, drives the humidity drying-wake, sharpens the Q10 fertility front, and bends permeability, so the whole ecosystem breathes on a diurnal rhythm through the channels sims already read. Tuning is about headroom (agent metabolic heat vs Temperature relax) and gradient (Temperature diffuseRate), not a direct chemotaxis leash. Design: Assets/Workspace/11.0 Biomes/docs/INTEGRATION_DESIGN.md (Tier 1, row 6).

• Cellular automata as field-native sims. The CA lineage from the predecessor repo arrives as agent-less grid sims on a shared FieldSimulationBase: CyclicCASim (Griffeath cyclic CA — smooth spiral waves, an excitable medium) and LookupCASim (multi-state lookup-table CA — a crisp standing lattice, with Langton's lambda as the edge-of-chaos dial). Both are double-buffered at the base, so the in-place read/write race in the ported source is inexpressible rather than merely avoided. They composite as ordinary layers and can publish their state into a biome channel (Excitability / Substrate), so an agent species responds to an automaton with no shader change — only a UmweltMapping entry. Pointing one CA's couplingSource at the other gates the waves on the lattice, making structure and flow one coupled system. Design: docs/superpowers/specs/2026-07-23-cellular-automata-sims-design.md; decision: docs/adr/0011-field-native-sims-derive-simulationbase.md.

• Event-driven CA bursts. Field sims are idle by default and dispatch nothing at all between bursts — no rule, no render, no publish. A burst is triggered by a rising edge of the neuron firing level, by a playback-frame advance whose aggregate activity clears burstFiringThreshold (burstOnFrameAdvance; activity is the frame's mean firing normalized to the loaded recording's peak synchrony, so 0.6 means "top-tier bursting events only" and a dense /index stream produces sporadic automata instead of one starved edge), or the Inspector's Trigger burst. On ignition the grid seeds — optionally from a biome channel by threshold (seedFromChannel), so the automaton grows out of what the ecosystem is actually doing — then holds burstSustainSteps (sim steps; retrigger extends the one evolving lattice, never restarts it), fades, and goes idle. Publishing stops on idle, so the deposited lattice stays in Excitability/Substrate and the PDE erodes, spreads and advects it from there — the picture fades, the trace lives on, and species keep reacting to it. Grid size is authored as an absolute cell height (cellRezHeight, width from the master's aspect), so the automaton's on-screen cell size survives output-resolution changes. Design: docs/superpowers/specs/2026-08-10-event-driven-ca-design.md.

• Non-uniform diffusion. Biome diffusion is a shaped weighted 3×3 average per channel: box→gaussian kernelShape, flowAnisotropy stretching spread along the local flow (turbulent eddy diffusivity — plus a global ambientWind prevailing drift), and permeabilityInfluence gating transport through termite walls, so mounds become scent containers (chambers pool pheromone, corridors duct it). Agent trail blurs gain coherence-enhancing diffusion (trailAnisotropy per sim): each cell's blur elongates along the trail's own structure-tensor ridge, so moving agents leave crisp comet tails instead of round blooms. All defaults reproduce the legacy uniform box blur. Decisions: docs/adr/0012-weighted-diffusion-kernels.md, docs/adr/0013-coherence-enhancing-trail-diffusion.md.

• FPS-independent sim. Step() runs on a fixed clock in FixedUpdate (Time.fixedDeltaTime = 1/simRate, default 60 Hz) so sim speed is identical across installs regardless of render FPS; the composite Render() is decoupled to LateUpdate (one composite per rendered frame). maxAllowedTimestep (Unity's Time.maximumDeltaTime) caps catch-up — weak installs run long, never fast (sim slows uniformly under load, no burst/stutter). Shader RNG seeds from a monotonic per-sim step, not Time.frameCount, so agents advance deterministically when multiple steps run per frame. 60 Hz = legacy per-frame feel, so no content re-tuning. Frames above simRate repeat the last step unless frameBlend (default off) blends the last two composites for smooth motion on 120/240 Hz displays; vSyncDivisor paces presents to every Nth refresh. Architecture: docs/ARCHITECTURE.md §3.1; design: docs/superpowers/specs/2026-07-11-fps-independent-sim-design.md.

• MIDI piano composition mixer. MidiPianoMixer turns a USB MIDI piano (Clavinova) into a live layer mixer over the composite. It opens its own Minis device connection (self-contained, like MidiFighterTwister — no MIDIMapping needed): white keys from A0 up auto-assign to the sims in SimulationManager.simulations order, and note velocity sets each sim's compositeWeight (chords crossfade; weights ease via smoothingSeconds; weightMax allows boost past 1, note-off holds the level). Command zone (top octave): C8 → full Reset(), B7 → ResetSimsOnly() (both require the sustain/damper pedal held), C7 → pause (stepsPerTick 0↔1). Coexists with the Midi Fighter Twister — disjoint controls, the piano owns compositeWeight. The custom Inspector shows the live key→sim map. Spec/plan: docs/superpowers/specs/2026-07-11-clavinova-composition-mixer-design.md, docs/superpowers/plans/2026-07-11-clavinova-composition-mixer.md.

• MFT LED legibility. MidiFighterTwister LEDs encode which type and which bank at a glance: SimParam knobs keep a flat family hue (physarum blue, boid orange, termite yellow) with a per-type brightness ramp (mid→full across type index, typeBrightnessMid tunable; hue gradients tried first but read as confusing on device), non-sim knobs run full brightness, and every soft/HW bank switch triggers a ~0.7s bank flash (top row counts the soft bank in its identity color, bottom row counts the HW bank in white). Spec: docs/superpowers/specs/2026-07-19-mft-led-feedback-design.md.

• Temporal Composer (11.2 SIGGRAPH show). A Unity Timeline (PlayableDirector)-driven show sequencer: track mixers push per-frame draw state into CompositeSequencer, which owns its own composerOutTex (rez independent of sim rez, stable RT per docs/adr/0008-clear-in-place-reset.md) and composites sim output + cells + patches each LateUpdate. Five track types: Biome Cell (2–4 self-paced BiomeCellRig sub-sims, Overlay/Replace onto a dstRect), Patch Scatter (Anadol-style scattered patches crossfading raw sim → diffusion return, deterministic per seed — pure logic lives in the engine-free Biomes.Sequencer.Core assembly, unit-tested outside play mode), Param Snapshot (eased live-param morph to a snapshot asset), Routing (overrides a sim's external-influence source), and reset SignalEmitters. ScreenLayout and the outbound sender read composerOutTex once wired per the manual-setup doc (scene wiring pending) — ExternalTextureSender gained SendSource.ComposerOutput (default stream EoC/Composer); a second ExternalTextureReceiver (TD_Diffusion stream) closes the loop with TouchDesigner's StreamDiffusion, Spout both directions on the show machine. Cap SimulationManager.targetFPS/BiomeCellRig.cellRate so Unity leaves GPU headroom for the diffusion step. Author clips via Biomes → Biome Palette (thumbnail grid over snapshot assets, capture-from-composite, drag/insert-at-playhead). Design: docs/superpowers/specs/2026-07-19-temporal-composer-design.md; manual setup/perf checklist: docs/superpowers/2026-07-19-temporal-composer-manual-setup.md.

• Agent colors (HSB + palettes). Every agent type has hue, saturation and brightness (defaults reproduce the old hardcoded look, so existing scenes render unchanged). The params inspector shows each type as an exact color field (picker/eyedropper/hex — what you pick is what shows on screen) with H/S/B sliders, and assigns colors from AgentColorPalette assets: 11.0 Biomes/assets/Palettes/Shows/ reproduce the exhibited scenes sim by sim (Metaesthetica VISAP, CURRENTS, SIGGRAPH, SIGGRAPH DAC, Brave New Work), Curated/ adds Bioluminescent, Ember, Lichen, Monochrome, Viridis; any sim's colors can be saved back as a palette. Live: SimulationManager → Agent Palettes cycles a palette list (plus the authored Preset) with fades, from its buttons, MFT NextPalette/PreviousPalette side actions, or OSC /palette_next · /palette_prev · /palette <i>; the active palette survives resets. MFT bank 2 is now hue/saturation/brightness/diffuse. Spec: docs/superpowers/specs/2026-10-04-agent-color-palettes-design.md.

• Measured performance changes. Perf work ships only with numbers: A/B ms/frame at one sim step per frame in _DAC_4k, kernel timings from a fixed GPU-state snapshot, and bit-compared readbacks. Latest pass: boid Move in cell-sorted order (−54 % hash+Move), divide-free torus wraps in the diffuse/trail kernels, a 64-lane inject cull, allocation-free ParameterRecorder, and frame export without per-frame managed PNG arrays. Log: Assets/Workspace/11.0 Biomes/docs/PERFORMANCE.md §4c.

See docs/ARCHITECTURE.md for the system reference and docs/INDEX.md for the doc map (ADRs, sessions, specs).

Roadmap

• Agent life/death + respawn — UmweltMapping exposes the lifecycle params (oxygen/permeability death thresholds, corpse→waste amount/decay) but mortality is not yet executed; parked for this version. Planned as an aesthetic bloom/collapse mechanism rather than homeostasis.
• Parameter literature grounding — map the exposed sim/biome parameters onto real slime-mold / termite / flocking biology. Brief: Assets/Workspace/11.0 Biomes/docs/RESEARCH_BRIEF.md.
• Layer & external-input integration — Q10 decomposition fronts, homeostatic channel equilibria (O₂/Temperature relax to baseline, docs/adr/0007), and injector raw-sensor calibration are live in the show scene. Humidity (12th channel, temperature-driven evaporation) is now live. Still planned: diurnal forcing, topographic stigmergy, texture-valued injector sources. Next-step coupling ideas (outside-signal routing, generalizing field→agent beyond the 4 perception slots, lifecycle death→succession) are sketched in Assets/Workspace/11.0 Biomes/docs/INTERACTION_DESIGN_II.md. Design: Assets/Workspace/11.0 Biomes/docs/INTEGRATION_DESIGN.md.