LLM generates malformed XML closing tag causing extraction failure

[shepelt]

Issue

When generating code files, Claude Sonnet occasionally generates malformed XML closing tags that break file extraction.

Example

Project ID: 3jWgMMickCJqLWQCB (on macserver)
Model: anthropic/claude-haiku-4-5-20251001

The LLM generated:

<sg-file path="src/components/ui/button.tsx" summary="Created button UI component">
... code content ...
</sx-file>  ❌ WRONG - should be </sg-file>

Impact

1. ❌ File extraction fails - content not written to workspace
2. ❌ Raw XML tags left in message display, confusing users
3. ✅ In this case, build still succeeded because button.tsx already existed in scaffold

Root Cause

Claude Sonnet made a typo in the closing tag: </sx-file> instead of </sg-file>

Sigrid's XML parser expects exact tag matching:

• Opening: <sg-file>
• Closing: </sg-file>

When tags don't match, the parser:

1. Fails to extract file content
2. Fails to clean up tags from message content
3. Leaves raw XML visible to users

Suggested Fixes

Option 1: More lenient parsing

• Accept common typos like </sx-file>, </sg-files>
• Use fuzzy matching for closing tags

Option 2: Better error recovery

• Strip malformed tags from message display even if extraction fails
• Show user-friendly error message instead of raw XML

Option 3: Prompt engineering

• Add explicit examples in system prompt emphasizing exact tag format
• Add validation reminder before generating closing tags

Related Code

File extraction likely happens in:

• filetooling.js
• llm-static.js or llm-dynamic.js

[shepelt]

MongoDB Evidence

Query from macserver MongoDB:

db.projects.findOne({_id: "3jWgMMickCJqLWQCB"})

Project Details:

• ID: 3jWgMMickCJqLWQCB
• User: 리노군 (c4DapJ4HoPDhbA4pp)
• Prompt: "로또 번호 추천하는 앱 만들어줘" (Create lottery number recommendation app)
• Model: anthropic/claude-haiku-4-5-20251001
• Status: running (stuck - should be completed)
• Created: 2025-11-11 11:18:55 GMT
• Workspace: /tmp/bdceddc71ceb0845

Message Structure:

The last message has:

• type: "files_updated"
• role: "system"
• files: Array with 9 files (properly extracted)
• content: Text with raw <sg-file> tags still visible

Files Array (9 files - extracted successfully):

1. src/lib/initDb.ts
2. src/lib/lottery.ts
3. src/components/LotteryNumberGenerator.tsx
4. src/components/SavedLotteryNumbers.tsx
5. src/pages/Index.tsx
6. src/components/ui/textarea.tsx
7. src/components/ui/input.tsx
8. src/components/ui/card.tsx
9. ❌ Missing: src/components/ui/button.tsx (extraction failed)

Content Field (2,413 chars):

The content field shows raw XML for button.tsx:

로또 번호 추천 앱을 만들어드리겠습니다! ...

<sg-file path="src/components/ui/button.tsx" summary="Created button UI component">
import * as React from "react"
... (full button.tsx code) ...
</sx-file>  ❌ TYPO HERE

완성했습니다! 🎰
...

Analysis

What worked:

• First 8 files with correct </sg-file> tags → extracted to files array ✅
• Build succeeded (button.tsx already in scaffold) ✅

What broke:

• Last file (button.tsx) with </sx-file> typo → NOT extracted ❌
• Raw XML left in content field → shown to user ❌

This proves:

• File extraction works correctly for properly formatted tags
• Message cleanup fails when closing tag doesn't match
• One malformed tag breaks the user experience even if build succeeds

[shepelt]

Related to NOBI Issue

This is the same root cause as nobi#8: https://github.com/shepelt/nobi/issues/8

NOBI #8 shows another instance where <sg-file> tags are being displayed raw to users instead of being extracted and cleaned up.

Cross-repo relationship:

• Sigrid (this repo): Generates code with <sg-file> tags
• NOBI: Displays the generated content to users

The bug manifests in both repos:

1. Sigrid fails to extract files when tags are malformed
2. NOBI displays the raw XML to users instead of rendering properly

Fix should address both:

• Sigrid: More robust XML parsing/extraction
• NOBI: Fallback rendering if tags aren't cleaned up

[shepelt]

Potential Solution: Switch to Native Tool Calling

Instead of XML tags, use Claude's native tool calling API which has been available since Claude 3.

Current approach (XML tags):

<sg-file path="src/file.tsx">content</sg-file>  ❌ Can have typos

Alternative (Tool calling):

{
  "tools": [{
    "name": "write_file",
    "description": "Write content to a file",
    "input_schema": {
      "type": "object",
      "properties": {
        "path": { "type": "string" },
        "content": { "type": "string" }
      },
      "required": ["path", "content"]
    }
  }]
}

Benefits:

• ✅ JSON schema validation - no parsing errors
• ✅ Structured output - no malformed tags
• ✅ Can create multiple files in one response (array of tool calls)
• ✅ Built-in error detection/retry
• ✅ Works with all Claude models

Open questions:

• ❓ Performance impact vs XML approach?
• ❓ Cost difference (token usage)?
• ❓ Supports parallel file creation?

Testing needed:
Benchmark XML vs tool calling to measure actual speed/cost differences.

Resources:

• https://docs.anthropic.com/claude/docs/tool-use
• Claude 3.5 Sonnet, Claude 4 family all support tool use

[shepelt]

create an experimental optimized file writer
https://github.com/shepelt/sigrid/tree/experiment/megawriter

problems with tool calling

• no streamed file output
• streaming + tool calling LLM compatibility
• still slow (30~40% slower than XML)

[shepelt]

http://studioline.tailcdff5e.ts.net:3000/project/uJ3FZruhjCcMXhQE7

haiku not respecting output format at all