AI integration layer for the Jido ecosystem - LLM orchestration, accuracy improvement techniques, and reasoning strategies for building intelligent agents in Elixir.
Jido.AI provides a comprehensive toolkit for improving LLM output quality through proven accuracy enhancement techniques. It implements research-backed algorithms for self-consistency, search, verification, reflection, and more - all designed to get better results from language models.
# Quick example: ReAct agent with tool use
defmodule MyApp.Agent do
use Jido.AI.ReActAgent,
name: "my_agent",
tools: [MyApp.Actions.Calculator, MyApp.Actions.Search],
model: :fast
end
{:ok, agent} = MyApp.Agent.start_link()
{:ok, response} = MyApp.Agent.chat(agent, "What is 15 * 23?")def deps do
[
{:jido, "~> 2.0"},
{:jido_ai, "~> 2.0"}
]
endConfigure your LLM provider (see Configuration Guide):
# config/config.exs
config :jido_ai, :models,
anthropic: [
api_key: System.get_env("ANTHROPIC_API_KEY")
],
openai: [
api_key: System.get_env("OPENAI_API_KEY")
]Strategies are agent patterns that determine how an LLM approaches a problem. They are the foundation of building intelligent agents with Jido.AI.
| Strategy | Pattern | Best For | Guide |
|---|---|---|---|
| ReAct | Reason-Act loop | Tool-using agents | Guide |
| Chain-of-Thought | Sequential reasoning | Multi-step problems | Guide |
| Tree-of-Thoughts | Explore multiple paths | Complex planning | Guide |
| Graph-of-Thoughts | Networked reasoning | Interconnected concepts | Guide |
| Adaptive | Strategy selection | Variable problem types | Guide |
When to use which strategy:
- ReAct - When your agent needs to use tools or APIs
- Chain-of-Thought - For multi-step reasoning and math problems
- Tree-of-Thoughts - When exploring multiple solution paths is beneficial
- Graph-of-Thoughts - For problems with interconnected concepts
- Adaptive - When you need dynamic strategy selection based on the problem
# ReAct agent with tools
defmodule MyApp.Agent do
use Jido.AI.ReActAgent,
name: "my_agent",
tools: [MyApp.Actions.Calculator, MyApp.Actions.Search],
model: :fast
end
# Chain-of-Thought for step-by-step reasoning
{:ok, result} = Jido.AI.Strategies.ChainOfThought.run(
"If 3 cats catch 3 mice in 3 minutes, how many cats are needed to catch 100 mice in 100 minutes?",
model: :fast
)Beyond strategies, Jido.AI provides research-backed techniques to improve LLM output quality. These are organized by how they enhance results:
Generate multiple candidates and aggregate results for more reliable answers.
| Technique | Best For | Guide |
|---|---|---|
| Self-Consistency | Multi-step reasoning, math problems | Guide |
| Adaptive Self-Consistency | Dynamic resource allocation | Guide |
When to use consensus methods:
- Problems with definite answers (math, logic, factual)
- When you can afford multiple LLM calls
- When majority voting improves reliability
# Generate 5 candidates, use majority vote
{:ok, best, _meta} = Jido.AI.Accuracy.SelfConsistency.run(
"If 3 cats catch 3 mice in 3 minutes, how long for 100 cats?",
num_candidates: 5,
aggregator: :majority_vote
)Systematically explore the reasoning space to find optimal solutions.
| Algorithm | Best For | Guide |
|---|---|---|
| Beam Search | Focused exploration, limited depth | Guide |
| MCTS | Complex reasoning, game-like scenarios | Guide |
| Diverse Decoding | Creative brainstorming | Guide |
When to use search algorithms:
- Problems with clear branching structure
- When systematic exploration beats single-shot
- Game-like or planning scenarios
# MCTS for complex reasoning
{:ok, best} = Jido.AI.Accuracy.Search.MCTS.search(
"Solve: x^2 + 5x + 6 = 0 for x",
llm_generator,
llm_verifier,
simulations: 100
)Validate outputs before accepting them, catching hallucinations and errors.
| Verifier Type | Best For | Guide |
|---|---|---|
| LLM Verifier | General purpose checking | Guide |
| Code Execution | Code generation, math | Guide |
| Deterministic | Known answers, test cases | Guide |
| Static Analysis | Code quality checks | Guide |
| Unit Test | Test-driven validation | Guide |
When to use verification:
- When hallucinations are costly
- For code generation or mathematical outputs
- When you have reference answers or tests
# Create a code execution verifier
verifier = Jido.AI.Accuracy.Verifiers.CodeExecutionVerifier.new!(%{
language: :elixir,
timeout: 5000
})
# Verify code outputs
{:ok, result} = Jido.AI.Accuracy.Verifiers.CodeExecutionVerifier.verify(
verifier,
candidate,
%{}
)Iteratively refine outputs through self-critique and revision.
| Technique | Best For | Guide |
|---|---|---|
| Self-Refine | Improving draft outputs | Guide |
| Reflection Stages | Multi-stage refinement | Guide |
| Critique & Revision | Structured improvement cycles | Guide |
When to use reflection:
- When initial drafts need refinement
- For writing, code, or complex explanations
- When you have time for iteration
# Self-refine for better outputs
strategy = Jido.AI.Accuracy.SelfRefine.new!(%{})
{:ok, result} = Jido.AI.Accuracy.SelfRefine.run(strategy, "Write a function to sort a list")
result.refined_candidate # The improved responseEstimate confidence and difficulty to allocate resources appropriately.
| Technique | Best For | Guide |
|---|---|---|
| Confidence Calibration | Reliability scoring | Guide |
| Difficulty Estimation | Resource allocation | Guide |
| Process Reward Models | Step-by-step quality | Guide |
When to use quality estimation:
- Variable-difficulty workloads
- Cost-sensitive applications
- When you need confidence scores
# Estimate difficulty to allocate resources
estimator = Jido.AI.Accuracy.Estimators.HeuristicDifficulty.new!(%{})
{:ok, estimate} = Jido.AI.Accuracy.Estimators.HeuristicDifficulty.estimate(
estimator,
"What is the square root of 144 multiplied by the sum of the first 10 primes?",
%{}
)
case estimate.level do
:easy -> use_fast_model()
:hard -> use_full_pipeline()
endCombine multiple techniques into powerful pipelines that adapt to your needs.
# Build a pipeline that adapts based on difficulty
{:ok, pipeline} = Jido.AI.Accuracy.Pipeline.new(%{})
generator = fn query, _context ->
# Your LLM generation logic here
{:ok, "Answer to: #{query}"}
end
{:ok, result} = Jido.AI.Accuracy.Pipeline.run(pipeline, "Solve this complex problem...", generator: generator)
result.answer # The final answer
result.confidence # Confidence score [0-1]When to use pipelines:
- Complex problems requiring multiple techniques
- When you need adaptive processing
- Production workflows with quality requirements
- Overview - Library introduction and concepts
- Strategies - Reasoning strategies
- Self-Consistency - Consensus-based improvement
- Adaptive Self-Consistency - Dynamic resource allocation
- Search Algorithms - Beam search, MCTS, diverse decoding
- Verification - Output validation techniques
- Reflection - Self-refine and reflection stages
- Critique & Revision - Structured improvement cycles
- Process Reward Models - Step-by-step quality scoring
- Confidence Calibration - Reliability estimation
- Difficulty Estimation - Resource-aware processing
- Pipeline - Combining techniques into workflows
- Architecture Overview - System design
- Strategies - Strategy implementation
- State Machines - Pure state machine pattern
- Directives - Declarative side effects
- Signals - Event-driven communication
- Tool System - Tool registry and execution
- Skills - Modular agent capabilities
- Configuration - Model aliases and providers
See the examples/ directory for runnable code:
examples/accuracy/- Accuracy improvement examplesexamples/strategies/- Reasoning strategy examples
Not sure which technique to use? Start here:
Building an agent?
├─ Need to use tools/APIs?
│ └─ Use ReAct Strategy
├─ Multi-step reasoning?
│ └─ Use Chain-of-Thought
└─ Complex planning?
└─ Use Tree-of-Thoughts
Improving accuracy?
├─ Problem has definite answer?
│ └─ Use Self-Consistency
│
├─ Requires exploration/planning?
│ ├─ Shallow depth → Beam Search
│ └─ Deep/complex → MCTS
│
├─ Output needs validation?
│ └─ Add Verification
│
├─ Initial draft acceptable?
│ └─ Use Self-Refine
│
└─ Variable difficulty?
└─ Use Pipeline with difficulty estimation
Contributions are welcome! Please see CONTRIBUTING.md for guidelines.
Apache-2.0 - See LICENSE for details.