fix(flows): extract embedded JSON from prose text, preventing output_parser hard-fails (#5151)#5153
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 50 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAgent structured-output parsing now extracts JSON from whole responses, fenced blocks, or prose spans before falling back to text. Tests cover precedence and fallback behavior. A NodeExecTool test receives new constructor arguments, and the tinyflows vendor pointer advances. ChangesStructured output recovery
Node execution test alignment
Tinyflows vendor revision
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
Comment |
…lt, preventing output_parser hard-fails (tinyhumansai#5151)
coderabbitai
Bot
added
agent
8e59081 to
419b4e3
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8e5908121e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Find the *last* closing ``` (preferring the outermost fence, which | ||
| // matches how Markdown renderers treat nested fences — the last ``` is | ||
| // the one that closes the block the LLM opened). | ||
| let close = content.rfind("```")?; |
There was a problem hiding this comment.
When the model includes any earlier fenced example/log before the final JSON answer, this pairs the first opening fence with the last closing fence, so the fenced parse fails over a span containing multiple fences. The code then falls through to balanced-brace extraction, which can return an unrelated earlier object such as an example {"example": true} instead of the later ```json answer, silently feeding the wrong value to the flow/output_parser.
Useful? React with 👍 / 👎.
| let b = bytes[end]; | ||
| if b == open { | ||
| depth = depth.checked_add(1)?; | ||
| } else if b == close { | ||
| depth = depth.checked_sub(1)?; | ||
| if depth == 0 { | ||
| // Found a balanced span — try to parse it. | ||
| let candidate = &text[start..=end]; |
There was a problem hiding this comment.
When the valid embedded JSON contains a structural character inside a string, for example prose followed by {"text":"use } as a delimiter"}, this byte scanner decrements depth for the quoted } and tries to parse an incomplete object, then breaks without reconsidering the real closing brace. Those structured agent outputs still fall back to {text, agent_ref} and can fail the downstream output_parser even though the JSON object was valid.
Useful? React with 👍 / 👎.
|
| Filename | Overview |
|---|---|
| src/openhuman/tinyflows/caps.rs | Adds two new JSON-extraction fallbacks (extract_fenced_json_block, extract_balanced_json) and wires them into build_agent_result; byte-level brace scanning in extract_balanced_json fails silently for JSON containing braces or brackets inside string values, and rfind in extract_fenced_json_block merges multiple fence blocks incorrectly |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["build_agent_result(agent_ref, final_text, request)"] --> B{structured_output_requested?}
B -- No --> Z["return {text, agent_ref}"]
B -- Yes --> C["Try 1: parse_llm_json(final_text)\n(bare JSON or whole-text fence block)"]
C -- Some(parsed) --> R1["return parsed"]
C -- None --> D["Try 2: extract_fenced_json_block(final_text)\n(fence block anywhere in prose)"]
D -- Some(parsed) --> R2["return parsed"]
D -- None --> E["Try 3: extract_balanced_json(final_text)\n(first balanced {…} or […] span)"]
E -- Some(parsed) --> R3["return parsed"]
E -- None --> W["warn: no extraction strategy succeeded"]
W --> Z
Reviews (1): Last reviewed commit: "fix(flows): extract embedded JSON from p..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| matches!(parsed, Value::Object(_) | Value::Array(_)).then_some(parsed) | ||
| } | ||
|
|
||
| /// Find and parse the first balanced `{…}` or `[…}` span in `text`. Walks |
There was a problem hiding this comment.
[…} should be […]
The closing delimiter in the doc comment reads […} (a mix of bracket and brace), but the matching pair for [ is ]. The same typo appears in the inline comment on line 920 inside build_agent_result ({…} / […}). Neither affects runtime behavior, but it's misleading for anyone reading the docs.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| fn extract_balanced_json(text: &str) -> Option<Value> { | ||
| let text = text.trim(); | ||
| let bytes = text.as_bytes(); | ||
| let len = bytes.len(); | ||
| for start in 0..len { | ||
| let open_byte = bytes[start]; | ||
| let (open, close) = match open_byte { | ||
| b'{' => (b'{', b'}'), | ||
| b'[' => (b'[', b']'), | ||
| _ => continue, | ||
| }; | ||
| let mut depth = 0u32; | ||
| for end in start..len { | ||
| let b = bytes[end]; | ||
| if b == open { | ||
| depth = depth.checked_add(1)?; | ||
| } else if b == close { | ||
| depth = depth.checked_sub(1)?; | ||
| if depth == 0 { | ||
| // Found a balanced span — try to parse it. | ||
| let candidate = &text[start..=end]; | ||
| if let Ok(parsed) = serde_json::from_str::<Value>(candidate) { | ||
| if matches!(parsed, Value::Object(_) | Value::Array(_)) { | ||
| return Some(parsed); | ||
| } | ||
| } | ||
| // Span didn't parse; continue scanning from the position | ||
| // after this false-positive open byte. | ||
| break; | ||
| } | ||
| } | ||
| } | ||
| } | ||
| None |
There was a problem hiding this comment.
extract_balanced_json scans bytes and increments/decrements depth on every {/} or [/], regardless of whether the byte is inside a quoted string. When a JSON string value itself contains a brace or bracket — e.g. {"message": "Use {name} as a placeholder"} or {"sql": "SELECT {col} FROM t"} — the { inside the string increments depth to 2, so the outer closing } only decrements it to 1 and the balanced span is never found. The function returns None, falling through to the {text, agent_ref} shape and still breaking the downstream output_parser for those inputs. Adding a test case for this pattern would expose the gap — e.g. The result is: {"message": "Use {name}"} should extract the object but currently will not.
| fn extract_fenced_json_block(text: &str) -> Option<Value> { | ||
| let text = text.trim(); | ||
| // Look for the first opening ``` fence | ||
| let fence_start = text.find("```")?; | ||
| let after_fence = text[fence_start + 3..].trim(); | ||
| // Skip optional "json" after the opening fence | ||
| let content = after_fence.strip_prefix("json").unwrap_or(after_fence).trim(); | ||
| // Find the *last* closing ``` (preferring the outermost fence, which | ||
| // matches how Markdown renderers treat nested fences — the last ``` is | ||
| // the one that closes the block the LLM opened). | ||
| let close = content.rfind("```")?; | ||
| let inner = content[..close].trim(); | ||
| let parsed = serde_json::from_str::<Value>(inner).ok()?; | ||
| matches!(parsed, Value::Object(_) | Value::Array(_)).then_some(parsed) | ||
| } |
There was a problem hiding this comment.
rfind("```") incorrectly merges two independent fenced blocks
When the LLM response contains more than one fenced code block — e.g. a first block containing a wrong attempt and a second block with the corrected JSON — extract_fenced_json_block opens at the first ``` and closes at the last ``` via rfind. Everything between those two positions becomes inner, which includes the prose and the second opening fence, so serde_json::from_str fails on it and the whole function returns None. The balanced-brace scan would then rescue a valid object anyway, but the "fenced block wins" guarantee documented in the test build_agent_result_prefers_fenced_json_over_balanced_brace_extraction breaks when two blocks are present. Using find instead of rfind for the closing fence (i.e. the first ``` after the opening) would correctly delimit the first block.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/openhuman/tinyflows/caps.rs`:
- Around line 5620-5692: Extract parse_llm_json, extract_fenced_json_block, and
extract_balanced_json from caps.rs into a focused sibling module such as
json_extract.rs, moving their dedicated tests there as well. Expose the required
functions to build_agent_result through the module boundary, preserving fenced
JSON precedence, balanced extraction, and existing fallback behavior.
- Around line 458-492: Update extract_balanced_json to track JSON string-literal
and escape state while counting delimiters, so brackets or braces inside quoted
values do not affect balancing and valid spans are still parsed. Also bound or
otherwise avoid rescanning the full remaining text for every unmatched opening
delimiter to prevent quadratic behavior, and add regression coverage for
embedded stray delimiters in string values.
- Around line 439-453: Update extract_fenced_json_block to locate the closing
fence with content.find("```") rather than rfind, so it closes the first fenced
block after the opening fence and handles multiple separate fenced blocks
correctly. Preserve the existing JSON parsing and object/array validation
behavior, and add a regression test covering an unrelated fenced block followed
by a JSON fenced block.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 9c481440-a988-4971-b577-4ec5a62a88b6
📒 Files selected for processing (3)
src/openhuman/tinyflows/caps.rssrc/openhuman/tools/impl/system/node_exec.rsvendor/tinyflows
| fn extract_fenced_json_block(text: &str) -> Option<Value> { | ||
| let text = text.trim(); | ||
| // Look for the first opening ``` fence | ||
| let fence_start = text.find("```")?; | ||
| let after_fence = text[fence_start + 3..].trim(); | ||
| // Skip optional "json" after the opening fence | ||
| let content = after_fence.strip_prefix("json").unwrap_or(after_fence).trim(); | ||
| // Find the *last* closing ``` (preferring the outermost fence, which | ||
| // matches how Markdown renderers treat nested fences — the last ``` is | ||
| // the one that closes the block the LLM opened). | ||
| let close = content.rfind("```")?; | ||
| let inner = content[..close].trim(); | ||
| let parsed = serde_json::from_str::<Value>(inner).ok()?; | ||
| matches!(parsed, Value::Object(_) | Value::Array(_)).then_some(parsed) | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
rfind picks the wrong closing fence when the response has more than one fenced block.
close = content.rfind("```") scans for the last triple-backtick in the entire remainder of text, not the fence that actually closes the block just opened. This works only when the fenced block is the sole/last fence in the text (as in the new build_agent_result_prefers_fenced_json_over_balanced_brace_extraction test). If the response has an earlier unrelated fence (e.g. an example code block) followed by the JSON fence — a common LLM pattern — rfind spans across both fences, the resulting inner slice is not valid JSON, and the block that should extract cleanly falls through to extract_balanced_json/the {text} fallback instead.
🐛 Proposed fix
- let close = content.rfind("```")?;
+ let close = content.find("```")?;This trades the (rare) nested-fence-inside-the-block case for correctly handling the (common) multiple-separate-fences case. Consider adding a regression test with two fenced blocks (e.g. a code example followed by the JSON block) to lock this in.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/openhuman/tinyflows/caps.rs` around lines 439 - 453, Update
extract_fenced_json_block to locate the closing fence with content.find("```")
rather than rfind, so it closes the first fenced block after the opening fence
and handles multiple separate fenced blocks correctly. Preserve the existing
JSON parsing and object/array validation behavior, and add a regression test
covering an unrelated fenced block followed by a JSON fenced block.
| fn extract_balanced_json(text: &str) -> Option<Value> { | ||
| let text = text.trim(); | ||
| let bytes = text.as_bytes(); | ||
| let len = bytes.len(); | ||
| for start in 0..len { | ||
| let open_byte = bytes[start]; | ||
| let (open, close) = match open_byte { | ||
| b'{' => (b'{', b'}'), | ||
| b'[' => (b'[', b']'), | ||
| _ => continue, | ||
| }; | ||
| let mut depth = 0u32; | ||
| for end in start..len { | ||
| let b = bytes[end]; | ||
| if b == open { | ||
| depth = depth.checked_add(1)?; | ||
| } else if b == close { | ||
| depth = depth.checked_sub(1)?; | ||
| if depth == 0 { | ||
| // Found a balanced span — try to parse it. | ||
| let candidate = &text[start..=end]; | ||
| if let Ok(parsed) = serde_json::from_str::<Value>(candidate) { | ||
| if matches!(parsed, Value::Object(_) | Value::Array(_)) { | ||
| return Some(parsed); | ||
| } | ||
| } | ||
| // Span didn't parse; continue scanning from the position | ||
| // after this false-positive open byte. | ||
| break; | ||
| } | ||
| } | ||
| } | ||
| } | ||
| None | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Balanced-span scan is not string-aware and can silently fail to extract valid JSON — plus quadratic worst case.
This is the last recovery strategy before falling back to {text, agent_ref}, so its correctness matters most. Two issues:
- Depth counting operates on raw bytes with no notion of JSON string literals. A string value containing an unescaped bracket/brace of the same type being scanned (e.g.
{"note": "see [draft]"}when scanning[/], or["a]b"]) drivesdepthto 0 prematurely. The resulting candidate fails to parse, the inner loopbreaks, and — because the outer loop never revisitsstart— the correct span starting at that position is never retried. For a plain array like["a]b", "c"]this makes the function returnNoneeven though valid JSON is present. - When no closing bracket is ever found for a given
start, the inner loop scans tolenbefore moving on; repeated across many candidatestartpositions this is O(n²) in the worst case (e.g. text containing many unmatched{/[characters).
🛡️ Proposed fix (adds string/escape awareness)
fn extract_balanced_json(text: &str) -> Option<Value> {
let text = text.trim();
let bytes = text.as_bytes();
let len = bytes.len();
for start in 0..len {
let open_byte = bytes[start];
let (open, close) = match open_byte {
b'{' => (b'{', b'}'),
b'[' => (b'[', b']'),
_ => continue,
};
let mut depth = 0u32;
+ let mut in_string = false;
+ let mut escaped = false;
for end in start..len {
let b = bytes[end];
- if b == open {
- depth = depth.checked_add(1)?;
- } else if b == close {
- depth = depth.checked_sub(1)?;
- if depth == 0 {
- // Found a balanced span — try to parse it.
- let candidate = &text[start..=end];
- if let Ok(parsed) = serde_json::from_str::<Value>(candidate) {
- if matches!(parsed, Value::Object(_) | Value::Array(_)) {
- return Some(parsed);
- }
- }
- // Span didn't parse; continue scanning from the position
- // after this false-positive open byte.
- break;
- }
- }
+ if in_string {
+ if escaped {
+ escaped = false;
+ } else if b == b'\\' {
+ escaped = true;
+ } else if b == b'"' {
+ in_string = false;
+ }
+ continue;
+ }
+ match b {
+ b'"' => in_string = true,
+ _ if b == open => depth = depth.checked_add(1)?,
+ _ if b == close => {
+ depth = depth.checked_sub(1)?;
+ if depth == 0 {
+ let candidate = &text[start..=end];
+ if let Ok(parsed) = serde_json::from_str::<Value>(candidate) {
+ if matches!(parsed, Value::Object(_) | Value::Array(_)) {
+ return Some(parsed);
+ }
+ }
+ break;
+ }
+ }
+ _ => {}
+ }
}
}
None
}For the quadratic-scan concern, consider bounding the scanned length (e.g. only scan up to some max prefix of final_text) as a cheap mitigation if unbounded model output can reach this path. Add regression tests covering a string value with an embedded stray bracket/brace to lock in the fix.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| fn extract_balanced_json(text: &str) -> Option<Value> { | |
| let text = text.trim(); | |
| let bytes = text.as_bytes(); | |
| let len = bytes.len(); | |
| for start in 0..len { | |
| let open_byte = bytes[start]; | |
| let (open, close) = match open_byte { | |
| b'{' => (b'{', b'}'), | |
| b'[' => (b'[', b']'), | |
| _ => continue, | |
| }; | |
| let mut depth = 0u32; | |
| for end in start..len { | |
| let b = bytes[end]; | |
| if b == open { | |
| depth = depth.checked_add(1)?; | |
| } else if b == close { | |
| depth = depth.checked_sub(1)?; | |
| if depth == 0 { | |
| // Found a balanced span — try to parse it. | |
| let candidate = &text[start..=end]; | |
| if let Ok(parsed) = serde_json::from_str::<Value>(candidate) { | |
| if matches!(parsed, Value::Object(_) | Value::Array(_)) { | |
| return Some(parsed); | |
| } | |
| } | |
| // Span didn't parse; continue scanning from the position | |
| // after this false-positive open byte. | |
| break; | |
| } | |
| } | |
| } | |
| } | |
| None | |
| } | |
| fn extract_balanced_json(text: &str) -> Option<Value> { | |
| let text = text.trim(); | |
| let bytes = text.as_bytes(); | |
| let len = bytes.len(); | |
| for start in 0..len { | |
| let open_byte = bytes[start]; | |
| let (open, close) = match open_byte { | |
| b'{' => (b'{', b'}'), | |
| b'[' => (b'[', b']'), | |
| _ => continue, | |
| }; | |
| let mut depth = 0u32; | |
| let mut in_string = false; | |
| let mut escaped = false; | |
| for end in start..len { | |
| let b = bytes[end]; | |
| if in_string { | |
| if escaped { | |
| escaped = false; | |
| } else if b == b'\\' { | |
| escaped = true; | |
| } else if b == b'"' { | |
| in_string = false; | |
| } | |
| continue; | |
| } | |
| match b { | |
| b'"' => in_string = true, | |
| _ if b == open => depth = depth.checked_add(1)?, | |
| _ if b == close => { | |
| depth = depth.checked_sub(1)?; | |
| if depth == 0 { | |
| let candidate = &text[start..=end]; | |
| if let Ok(parsed) = serde_json::from_str::<Value>(candidate) { | |
| if matches!(parsed, Value::Object(_) | Value::Array(_)) { | |
| return Some(parsed); | |
| } | |
| } | |
| break; | |
| } | |
| } | |
| _ => {} | |
| } | |
| } | |
| } | |
| None | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/openhuman/tinyflows/caps.rs` around lines 458 - 492, Update
extract_balanced_json to track JSON string-literal and escape state while
counting delimiters, so brackets or braces inside quoted values do not affect
balancing and valid spans are still parsed. Also bound or otherwise avoid
rescanning the full remaining text for every unmatched opening delimiter to
prevent quadratic behavior, and add regression coverage for embedded stray
delimiters in string values.
|
|
||
| // ── build_agent_result improvements (issue #5151) ──────────────────── | ||
|
|
||
| #[test] | ||
| fn build_agent_result_extracts_embedded_json_from_prose_text() { | ||
| // When the agent's final text wraps JSON in prose without fence | ||
| // blocks (e.g. the LLM explains the result before outputting the | ||
| // data), build_agent_result must still extract the object rather than | ||
| // falling back to {text, agent_ref} which kills the downstream | ||
| // output_parser. | ||
| let request = json!({ | ||
| "output_parser": { | ||
| "schema": { "type": "object", "required": ["name"] } | ||
| } | ||
| }); | ||
| let result = build_agent_result( | ||
| "agent-1", | ||
| "The result is: { \"name\": \"Alice\", \"age\": 30 }", | ||
| &request, | ||
| ); | ||
| assert_eq!(result, json!({ "name": "Alice", "age": 30 })); | ||
| } | ||
|
|
||
| #[test] | ||
| fn build_agent_result_extracts_embedded_array_from_prose_text() { | ||
| let request = json!({ | ||
| "output_parser": { | ||
| "schema": { "type": "array" } | ||
| } | ||
| }); | ||
| let result = build_agent_result( | ||
| "agent-1", | ||
| "Here is the list: [1, 2, 3]", | ||
| &request, | ||
| ); | ||
| assert_eq!(result, json!([1, 2, 3])); | ||
| } | ||
|
|
||
| #[test] | ||
| fn build_agent_result_falls_back_to_text_when_no_json_found_in_prose() { | ||
| // Pure prose with no JSON-like content must still fall back to the | ||
| // safe {text, agent_ref} shape. | ||
| let request = json!({ | ||
| "output_parser": { | ||
| "schema": { "type": "object", "required": ["name"] } | ||
| } | ||
| }); | ||
| let result = build_agent_result( | ||
| "agent-1", | ||
| "I searched for the information but could not find it.", | ||
| &request, | ||
| ); | ||
| assert_eq!( | ||
| result, | ||
| json!({ "text": "I searched for the information but could not find it.", | ||
| "agent_ref": "agent-1" }) | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn build_agent_result_prefers_fenced_json_over_balanced_brace_extraction() { | ||
| // When both a fenced block and loose prose-with-JSON are present, | ||
| // the fenced block wins (it's the canonical / better-specified | ||
| // format). | ||
| let request = json!({ | ||
| "output_parser": { | ||
| "schema": { "type": "object" } | ||
| } | ||
| }); | ||
| let text = "Some text\n```json\n{\"from_fence\": true}\n```\nmore text { \"from_brace\": true }"; | ||
| let result = build_agent_result("agent-1", text, &request); | ||
| assert_eq!(result, json!({ "from_fence": true })); | ||
| } |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift
These additions keep growing an already oversized module.
caps.rs already spans well past 5,600 lines; this PR adds two new extraction helpers, a reworked pipeline branch, and ~70 lines of tests on top of that. As per coding guidelines, "Prefer files of approximately 500 lines or fewer and keep modules small, single-responsibility, and composed through clear boundaries."
Consider extracting parse_llm_json, extract_fenced_json_block, extract_balanced_json, and their dedicated tests into a focused sibling module (e.g. src/openhuman/tinyflows/json_extract.rs), re-exported for use by build_agent_result. This is a larger, file-wide effort beyond just this diff, but pays off in reviewability/testability for a module that will likely keep growing.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/openhuman/tinyflows/caps.rs` around lines 5620 - 5692, Extract
parse_llm_json, extract_fenced_json_block, and extract_balanced_json from
caps.rs into a focused sibling module such as json_extract.rs, moving their
dedicated tests there as well. Expose the required functions to
build_agent_result through the module boundary, preserving fenced JSON
precedence, balanced extraction, and existing fallback behavior.
Source: Coding guidelines
1 participant
When an agent node in a workflow declares an output_parser schema, the model's reply is expected to be valid JSON. But LLMs often wrap JSON in prose text (e.g. 'The result is: { "name": "Alice" }'), which parse_llm_json() can't handle, causing the entire workflow run to hard-fail.
This fix adds two fallback extraction strategies in build_agent_result():
json …blocks anywhere in the text (not just at the start)Both are tried in sequence before falling back to the {text, agent_ref} shape, which previously killed the downstream output_parser.
Fixes #5151
Summary by CodeRabbit