Skip to content

INTEGRATION_GUIDE.md

Latest commit

 

History

History
428 lines (341 loc) · 14.4 KB

INTEGRATION_GUIDE.md

File metadata and controls

428 lines (341 loc) · 14.4 KB

AI Optimization & AgentDB Integration Guide

Overview

This guide covers the implementation of Phases 3-4 from /workspaces/lean-agentic/plans/lean-agentic.md:

  • Phase 3: AI-Driven Optimization (LLM Compiler, 4-Tier JIT, Multi-Lane Routing)
  • Phase 4: AgentDB Integration (Vector Memory, ReasoningBank, Causal Graphs)

Architecture

┌─────────────────────────────────────────────────────────┐
│                 LEAN-AGENTIC RUNTIME                    │
├─────────────────────────────────────────────────────────┤
│  ┌──────────────────┐  ┌──────────────────────────┐   │
│  │ LLM COMPILER     │  │ 4-TIER JIT RUNTIME       │   │
│  │ - XLA AOT        │  │ - Tier 0: Interpreter    │   │
│  │ - Vectorization  │  │ - Tier 1: Baseline       │   │
│  │ - Test Synthesis │  │ - Tier 2: Optimizing     │   │
│  │ - SMT Validation │  │ - Tier 3: Max-Opt        │   │
│  └──────────────────┘  └──────────────────────────┘   │
│                                                          │
│  ┌──────────────────────────────────────────────────┐  │
│  │ MULTI-LANE ROUTING                               │  │
│  │ - onnx_local (0 cost, variable latency)         │  │
│  │ - anthropic ($0.10/1K, low latency)             │  │
│  │ - openrouter ($0.05/1K, medium latency)         │  │
│  │ - Real-time cost tracking & quota enforcement   │  │
│  │ - Adaptive routing with RL                       │  │
│  └──────────────────────────────────────────────────┘  │
│                                                          │
│  ┌──────────────────────────────────────────────────┐  │
│  │ AGENTDB (Vector Memory)                          │  │
│  │ ┌────────────────┐  ┌─────────────────────────┐ │  │
│  │ │ Qdrant/HNSW    │  │ ReasoningBank           │ │  │
│  │ │ - Semantic     │  │ - Trajectory Tracking   │ │  │
│  │ │ - Episodic     │  │ - Verdict Judgment      │ │  │
│  │ │ - <10ms P99    │  │ - Pattern Distillation  │ │  │
│  │ └────────────────┘  └─────────────────────────┘ │  │
│  │ ┌────────────────────────────────────────────┐  │  │
│  │ │ Causal Graph + Memory Consolidation       │  │  │
│  │ └────────────────────────────────────────────┘  │  │
│  └──────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────┘

Component Details

1. AgentDB - Vector Storage

Location: /workspaces/lean-agentic/src/agentdb/

Key Files:

  • mod.rs - Main AgentDB interface
  • vector_store.rs - Qdrant/HNSW integration
  • reasoning_bank.rs - Trajectory tracking and pattern learning
  • episode_store.rs - Episodic memory storage
  • causal_graph.rs - Causal relationship tracking
  • memory_consolidation.rs - Background memory updates

Performance Targets:

  • Vector search: <10ms P99 latency
  • Throughput: 1K+ QPS per node
  • Recall@10: 95%+ with HNSW
  • Memory efficiency: 1.2-1.4x base vector size

Example Usage:

use agentdb::{AgentDb, AgentDbConfig, Episode};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize AgentDB
    let config = AgentDbConfig::default();
    let db = AgentDb::new(config).await?;

    // Store episode
    let episode = Episode::new(
        "episode_001".to_string(),
        "User asked about API optimization".to_string(),
        "Suggested caching strategy".to_string(),
        "20% latency reduction".to_string(),
        vec![0.1; 1536], // Embedding vector
        vec!["api".to_string(), "optimization".to_string()],
    );

    db.store_episode(episode).await?;

    // Retrieve similar episodes
    let query_embedding = vec![0.1; 1536];
    let recall = db.recall(
        "optimization strategies",
        query_embedding,
        10, // limit
    ).await?;

    println!("Retrieved {} episodes in {}ms",
        recall.episodes.len(),
        recall.total_time_ms
    );

    for (episode, explanation) in recall.episodes.iter()
        .zip(recall.explanations.iter()) {
        println!("  - {} (score: {:.3})",
            episode.context,
            explanation.similarity_score
        );
        println!("    Reasoning: {}", explanation.reasoning);
    }

    Ok(())
}

2. ReasoningBank - Pattern Learning

Location: /workspaces/lean-agentic/src/agentdb/reasoning_bank.rs

Purpose: Track optimization attempts, judge outcomes, and distill successful patterns.

Example Usage:

use agentdb::reasoning_bank::{
    ReasoningBank, Trajectory, TrajectoryStep, StepType,
    Verdict, VerdictMetrics
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let rb = ReasoningBank::new();

    // Track optimization trajectory
    let trajectory = Trajectory {
        id: "opt_001".to_string(),
        task_type: "lane_selection".to_string(),
        steps: vec![
            TrajectoryStep {
                step_id: "step_1".to_string(),
                step_type: StepType::LaneSelection,
                input_state: "high load, tight budget".to_string(),
                action: "selected onnx_local".to_string(),
                output_state: "completed in 50ms".to_string(),
                cost: 0.0,
                latency_ms: 50,
                timestamp: 0,
            },
        ],
        verdict: None,
        metadata: HashMap::new(),
        created_at: 0,
    };

    rb.track(trajectory).await?;

    // Judge outcome
    let verdict = Verdict {
        success: true,
        score: 0.95,
        metrics: VerdictMetrics {
            cost_savings_pct: 100.0,
            latency_improvement_pct: 50.0,
            correctness_preserved: true,
            test_coverage_pct: 90.0,
        },
        reasoning: "Excellent cost savings with acceptable latency".to_string(),
    };

    rb.judge("opt_001", verdict).await?;

    // Get statistics
    let stats = rb.stats().await;
    println!("ReasoningBank Stats:");
    println!("  Total trajectories: {}", stats.total_trajectories);
    println!("  Successful: {}", stats.successful_trajectories);
    println!("  Patterns learned: {}", stats.distilled_patterns);
    println!("  Avg cost savings: {:.1}%", stats.avg_cost_savings_pct);

    Ok(())
}

3. LLM Compiler

Location: /workspaces/lean-agentic/src/llm-compiler/

Key Components:

  • XLA Runtime: AOT compiled models (no runtime deps)
  • Auto-Vectorization: GNN + DRL for loop optimization
  • Test Synthesis: MuTAP mutation-guided approach
  • SMT Validation: Z3 for semantic equivalence

Performance Targets:

  • Inference: <100ms in batch mode
  • Vectorization speedup: 10-30% on vectorizable code
  • Test mutation score: 90%+

Example Usage:

use llm_compiler::{LlmCompiler, LlmCompilerConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = LlmCompilerConfig::default();
    let compiler = LlmCompiler::new(config)?;

    // Analyze code for optimizations
    let code = r#"
        fn compute(data: &[f32]) -> Vec<f32> {
            let mut result = Vec::new();
            for &x in data {
                result.push(x * 2.0 + 1.0);
            }
            result
        }
    "#;

    let suggestions = compiler.analyze_code(code, "").await?;

    for suggestion in suggestions {
        println!("Optimization: {:?}", suggestion.suggestion_type);
        println!("  Expected speedup: {:.1}x", suggestion.expected_speedup);
        println!("  Confidence: {:.2}", suggestion.confidence);
        println!("  Reasoning: {}", suggestion.reasoning);
    }

    // Auto-vectorization
    let loop_code = r#"
        for i in 0..n {
            result[i] = a[i] + b[i];
        }
    "#;

    let vec_config = compiler.auto_vectorize(loop_code).await?;

    if vec_config.should_vectorize {
        println!("Vectorization recommended:");
        println!("  VF: {}", vec_config.vectorization_factor);
        println!("  IF: {}", vec_config.interleave_factor);
        println!("  Expected speedup: {:.1}x", vec_config.cost_estimate);
    }

    Ok(())
}

4. 4-Tier JIT Runtime

Location: /workspaces/lean-agentic/src/jit-runtime/

Tiers:

  1. Tier 0 (Interpreter): 0ms startup, 1x speed
  2. Tier 1 (Baseline): 1-5ms compile, 5-15x speed
  3. Tier 2 (Optimizing): 10-50ms compile, 20-50x speed
  4. Tier 3 (Max-Opt): 100-500ms compile, 50-200x speed

Features:

  • Profile-guided optimization
  • On-stack replacement (OSR) for hot loops
  • Type feedback for speculative optimization
  • Automatic deoptimization

Example Usage:

use jit_runtime::{JitRuntime, JitRuntimeConfig};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = JitRuntimeConfig::default();
    let runtime = JitRuntime::new(config);

    // Execute function - will tier up automatically
    for i in 0..1100 {
        let result = runtime.execute(
            "my_function",
            &[serde_json::json!(i)],
        )?;

        if i % 100 == 0 {
            println!("Iteration {}: {:?}", i, result);
        }
    }

    // After 1100 iterations, function should be at Tier 3 (MaxOpt)

    Ok(())
}

5. Multi-Lane Routing

Location: /workspaces/lean-agentic/src/multi-lane/

Providers:

  • onnx_local: Free, variable latency (50-200ms)
  • anthropic: $0.10/1K tokens, low latency (100-300ms)
  • openrouter: $0.05/1K tokens, medium latency (150-500ms)

Features:

  • Real-time cost tracking
  • Quota enforcement
  • Adaptive routing with learning
  • Cost variance <5%

Example Usage:

use multi_lane::{
    LaneRouter, LaneRouterConfig,
    InferenceRequest, RequestPriority
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = LaneRouterConfig::default();
    let router = LaneRouter::new(config);

    // Create inference request
    let request = InferenceRequest {
        prompt: "Explain quantum computing".to_string(),
        estimated_tokens: 500,
        max_tokens: 1000,
        latency_requirement: Some(Duration::from_millis(200)),
        priority: RequestPriority::High,
    };

    // Route to optimal lane
    let provider = router.route(&request).await?;
    println!("Selected provider: {:?}", provider);

    // Execute inference
    let response = router.execute_inference(provider, &request).await?;
    println!("Response: {}", response.content);
    println!("Tokens used: {}", response.tokens_used);
    println!("Cost: ${:.4}", response.cost);

    // Check cost statistics
    let stats = router.cost_stats().await;
    println!("\nCost Statistics:");
    println!("  Total cost: ${:.2}", stats.total_cost);
    println!("  Cost variance: {:.2}%", stats.cost_variance * 100.0);
    println!("  Savings vs Anthropic only: {:.1}%",
        stats.savings_vs_anthropic_only
    );
    println!("  Predicted monthly: ${:.2}", stats.predicted_monthly_cost);

    Ok(())
}

Integration with Hooks

Use hooks for coordination across agents:

# Before starting optimization work
npx claude-flow@alpha hooks pre-task \
  --description "AI Optimization & AgentDB Integration"

# After editing files
npx claude-flow@alpha hooks post-edit \
  --file "/workspaces/lean-agentic/src/agentdb/mod.rs" \
  --memory-key "swarm/ai-opt/agentdb"

# After completing task
npx claude-flow@alpha hooks post-task \
  --task-id "ai-optimization"

Performance Benchmarks

Create benchmarks to verify targets:

// benchmarks/ai_optimization.rs

use criterion::{black_box, criterion_group, criterion_main, Criterion};
use agentdb::{AgentDb, AgentDbConfig};

fn bench_vector_search(c: &mut Criterion) {
    let rt = tokio::runtime::Runtime::new().unwrap();
    let db = rt.block_on(async {
        AgentDb::new(AgentDbConfig::default()).await.unwrap()
    });

    c.bench_function("vector_search_10_results", |b| {
        b.to_async(&rt).iter(|| async {
            let query = vec![0.1; 1536];
            db.recall("test", query, 10).await.unwrap()
        })
    });
}

criterion_group!(benches, bench_vector_search);
criterion_main!(benches);

Expected Outcomes

After full implementation:

  1. Cost Savings: 30-50% vs single provider (Anthropic-only)
  2. Cost Variance: <5% deviation from predicted
  3. Vector Search: <10ms P99 latency
  4. JIT Speedup: 50-200x for hot functions
  5. Vectorization: 10-30% speedup on vectorizable code
  6. Pattern Learning: 90%+ success rate on learned optimizations

Next Steps

  1. Implement remaining stub functions (TODOs)
  2. Integrate actual Qdrant client (replace in-memory HNSW)
  3. Add XLA AOT model loading
  4. Implement actual JIT compilation backends
  5. Connect to real provider APIs
  6. Create comprehensive benchmark suite
  7. Add example demonstrating 30%+ cost savings

See Also

  • /workspaces/lean-agentic/plans/lean-agentic.md - Full roadmap
  • /workspaces/lean-agentic/src/agentdb/ - Vector storage implementation
  • /workspaces/lean-agentic/src/llm-compiler/ - AI optimization layer
  • /workspaces/lean-agentic/src/jit-runtime/ - Tiered JIT runtime
  • /workspaces/lean-agentic/src/multi-lane/ - Multi-provider routing