@@ -36,6 +36,45 @@ The caching system includes several key features for efficient management:
36
36
3 . ** Size-Limited Cache** : The cache has a maximum size per project to prevent memory issues.
37
37
4 . ** Automatic Cleanup** : A background process periodically removes expired entries.
38
38
39
+ ## Sequence Diagram: Plan Caching Workflow
40
+
41
+ The following diagram illustrates the key aspects of how plan caching works in Orra:
42
+
43
+ ``` mermaid
44
+ sequenceDiagram
45
+ participant Client
46
+ participant Orra as Orra Orchestration
47
+ participant Cache as Plan Cache
48
+ participant LLM as LLM API
49
+
50
+ Client->>Orra: Submit Action + Parameters
51
+ Note over Orra: Process Action & Parameters
52
+
53
+ Orra->>Cache: Check for Similar Actions
54
+
55
+ alt Cache Miss - First Time Action
56
+ Cache-->>Orra: No Similar Plan Found
57
+ Orra->>LLM: Generate Plan (API Call)
58
+ LLM-->>Orra: Execution Plan
59
+ Orra->>Orra: Process & Validate Plan
60
+ Orra->>Cache: Store Plan with Action Signature
61
+ Orra->>Orra: Execute Plan
62
+ else Cache Hit - Similar Action
63
+ Cache-->>Orra: Return Cached Plan
64
+ Note over Orra: Adapt Plan with New Parameters
65
+ Orra->>Orra: Execute Adapted Plan
66
+ end
67
+
68
+ Orra-->>Client: Return Orchestration Results
69
+
70
+ rect rgb(240, 248, 255)
71
+ Note over Client,LLM: Benefits
72
+ Note over Client,LLM: ✓ Reduced API Costs
73
+ Note over Client,LLM: ✓ Faster Response Times
74
+ Note over Client,LLM: ✓ Consistent Behavior
75
+ end
76
+ ```
77
+
39
78
## Cost Savings with Plan Caching
40
79
41
80
### LLM Cost Reduction
0 commit comments