Skip to content

[BUG] Edits fail because the model thinks apply_diff supports substring matching in SEARCH #868

Description

@lowlight-ua

Describe the bug

The apply_diff tool's system prompt instructions do not make clear that the SEARCH block must contain complete, whole lines (not substrings). This is not a problem with code, usually, because lines are short. But editing Markdown files often fails, because lines in Markdown files can be long (containing entire paragraphs), and the model is tempted to provide only a substring of the line in the SEARCH block. Zoo code can't match the block and fails the edit.

To Reproduce
Steps to reproduce the behavior:

Pass a large Markdown file with long lines into the chat, and ask to perform inline edits (e.g. find-replace).

Some edits are likely to be unsuccessful, and Zoo may eventually abort the mission due to a loop of failures. The error message returned by the tool is similar to: "No sufficiently similar match found at line: 12 (29% similar, needs 100%)".

Task history will show that the failed edits exhibit the "substring in SEARCH" issue, whereas in the successful edits, the whole line is supplied.

Expected behavior

(a) Substring matching is supported and the edit is applied correctly, OR
(b) The system prompt instructions explicitly warn: "The SEARCH block must contain complete lines. Partial-line (substring) matching is not supported. When the target line is very long (e.g. a Markdown paragraph), you must copy the entire line into the SEARCH block."

Screenshots
If applicable, add screenshots to help explain your problem.

Video
Add a video of the incident happening, it will help us debug

What version of zoo are you running
Version: 3.66.0 (a5a490a)

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions