Skip to content

feat(storage): add unified persistence layer for agent checkpoints and thread journals#110

Merged
mikehostetler merged 6 commits intomainfrom
feature/agent-storage
Feb 2, 2026
Merged

feat(storage): add unified persistence layer for agent checkpoints and thread journals#110
mikehostetler merged 6 commits intomainfrom
feature/agent-storage

Conversation

@mikehostetler
Copy link
Contributor

Summary

Implements the Jido Storage unified persistence layer based on the JIDO_STORAGE.md specification.

Changes

New Modules

  • Jido.Storage - Unified storage behaviour with 6 callbacks for checkpoints and journals
  • Jido.Storage.ETS - ETS adapter (ephemeral, for dev/test)
  • Jido.Storage.File - File-based adapter (durable)
  • Jido.Persist - Hibernate/thaw coordinator, invariant enforcer

Modified Modules

  • Jido - Added storage: option to use Jido macro, hibernate/2 and thaw/3 functions
  • Jido.Agent - Added checkpoint/2 and restore/2 callbacks with default implementations

Documentation

  • Added comprehensive guides/storage.md guide (~800 lines)
  • Added Storage modules to mix.exs docs configuration

Tests

  • 87 new tests for storage functionality
  • All 1479 tests pass
  • 82.5% overall coverage (above 80% threshold)
  • mix quality passes

Key Features

  • Unified Jido.Storage behaviour - One adapter handles both checkpoints (overwrite) and journals (append-only)
  • Thread pointer invariant - Checkpoint stores only %{id, rev} pointer, never full Thread
  • Optimistic concurrency - expected_rev: option for conflict detection
  • Agent callbacks - Optional checkpoint/2 and restore/2 for custom serialization

Usage

defmodule MyApp.Jido do
  use Jido,
    otp_app: :my_app,
    storage: {Jido.Storage.ETS, table: :my_storage}
end

# Hibernate an agent
:ok = MyApp.Jido.hibernate(agent)

# Thaw an agent
{:ok, agent} = MyApp.Jido.thaw(MyAgent, "user-123")

mikehostetler and others added 6 commits February 2, 2026 07:23
…s and thread journals

This commit introduces a comprehensive storage and persistence mechanism for the Jido framework, enabling agents to hibernate and thaw their state. Key changes include:

- Added a new `Jido.Persist` module to manage hibernation and restoration of agents, ensuring that only thread pointers are stored in checkpoints.
- Implemented `Jido.Storage` behavior with an ETS-based storage adapter for fast in-memory operations.
- Introduced a `Jido.Thread` module to manage an append-only log of interaction entries, enhancing the tracking of agent activities.
- Updated agent strategies to support optional thread tracking during command execution, allowing for better observability of instruction flows.

These enhancements improve the overall functionality and robustness of the Jido AI framework, facilitating richer interactions and better state management for agents.
- Merge persistence.md content into storage.md as unified guide
- Add 'Choosing Your Persistence Model' decision table
- Add 'Automatic Lifecycle with InstanceManager' section
- Add 'When NOT to Persist' section
- Add 'Migration from Legacy API' section
- Fix API naming: Jido.AgentPool -> Jido.Agent.WorkerPool
- Update cross-references in configuration.md, runtime.md,
  worker-pools.md, phoenix-integration.md
- Delete redundant persistence.md

Amp-Thread-ID: https://ampcode.com/threads/T-019c1f12-5bc6-77bd-9a00-513d1f8af4ee
Co-authored-by: Amp <[email protected]>
- Add all ungrouped modules to appropriate groups
- Add new groups: State Operations, Sensors, Thread, Exceptions
- Rename 'Agent Components' to 'Agent Internals'
- Include all sub-modules for Skills, Actions, Storage, Observability
- Remove persistence.md from docs config (consolidated into storage.md)

Amp-Thread-ID: https://ampcode.com/threads/T-019c1f12-5bc6-77bd-9a00-513d1f8af4ee
Co-authored-by: Amp <[email protected]>
@mikehostetler mikehostetler merged commit 01af01b into main Feb 2, 2026
6 checks passed
@mikehostetler mikehostetler deleted the feature/agent-storage branch February 2, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant