Skip to content

fix(search): use NUL-separated filelist + xargs -0 for paths with spaces#693

Closed
lg320531124 wants to merge 2 commits into
DeusData:mainfrom
lg320531124:fix/space-in-search-code-path
Closed

fix(search): use NUL-separated filelist + xargs -0 for paths with spaces#693
lg320531124 wants to merge 2 commits into
DeusData:mainfrom
lg320531124:fix/space-in-search-code-path

Conversation

@lg320531124

Copy link
Copy Markdown
Contributor

Problem

search_code returns 0 matches when the indexed repo path contains spaces (e.g. /Users/name/My Project/).

Root cause: write_scoped_filelist() writes newline-separated paths, and xargs splits on whitespace — so a path like /tmp/test repo/src/main.c becomes two arguments: /tmp/test and repo/src/main.c.

Fix

  • write_scoped_filelist(): write NUL-separated paths via fwrite() instead of fprintf() with newline separators
  • build_grep_cmd(): add -0 flag to xargs so it reads NUL-terminated entries

Non-scoped mode was already safe (root_path is single-quoted in the grep command: grep -rn ... '%s').

Verification

Indexed and searched a repo at /tmp/test repo/search_code now returns correct results:

{"results":[{"node":"main","qualified_name":"tmp-test-repo.src.main.main","label":"Function","file":"src/main.c","start_line":2,"end_line":5,"match_lines":[3]}],"total_grep_matches":1,"total_results":1}

Previously: 0 matches.

Closes #687

@lg320531124 lg320531124 requested a review from DeusData as a code owner June 29, 2026 09:11
@lg320531124 lg320531124 force-pushed the fix/space-in-search-code-path branch 2 times, most recently from d0a0c48 to 8954b9b Compare June 29, 2026 13:19
@DeusData

Copy link
Copy Markdown
Owner

Huge thanks for opening this PR and for the work you put into it.

The maintainer shop is currently full, so this may sit for a bit before it gets a proper review. We will come back to this as soon as possible with real feedback; I wanted to make sure it did not sit unacknowledged in the meantime.

@DeusData DeusData added bug Something isn't working ux/behavior Display bugs, docs, adoption UX priority/high Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker. labels Jun 29, 2026
When a repo path contains spaces (e.g. /Users/name/My Project/), the
scoped search pipeline breaks because:

1. write_scoped_filelist() writes newline-separated paths → xargs
   splits on spaces, passing truncated paths to grep
2. xargs has no -0 flag, so it can't handle NUL separators

Fix: write NUL-separated paths in write_scoped_filelist() via fwrite(),
and add -0 flag to xargs in build_grep_cmd().

Non-scoped mode was already safe (root_path is single-quoted in the
grep command).

Closes DeusData#687

Signed-off-by: lg320531124 <lg320531124@users.noreply.github.com>
…tent

The NUL-separated filelist from the previous commit breaks the Windows
search path: PowerShell 'Get-Content | ForEach-Object { Select-String
-LiteralPath src/mcp/mcp.c }' reads line-by-line and rejects NUL bytes with
'Illegal characters in path', making search_code return empty and
failing test_mcp.c:1098 (search_code_multi_word ASSERT HandleRequest).

Windows never needed NUL — Get-Content feeds each path as one line to
-LiteralPath, which accepts spaces without shell word-splitting. NUL was
only needed on Unix for 'xargs -0' to survive spaces/newlines in paths.

Branch on _WIN32: '\n' on Windows (PowerShell line reader), '\0' on
Unix (xargs -0). Fixes the CI regression introduced while fixing DeusData#687.

Signed-off-by: lg320531124 <lg320531124@users.noreply.github.com>
@DeusData

DeusData commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Thank you @lg320531124 — you correctly root-caused #687, and the fix concept is exactly right: a newline-separated filelist + plain xargs splits on spaces, while NUL-separated records + xargs -0 keep each path a single argument (and you correctly kept newline for the Windows PowerShell Get-Content -LiteralPath path). No command injection either — filenames flow through the filelist to xargs, never a shell.

One issue kept it from merging as-is: the filelist writer builds each record in a fixed char buf[CBM_SZ_2K] (2048) with unbounded memcpys of <root> + / + <file> + separator. Indexed entries are absolute paths, and on Linux PATH_MAX is 4096 — so a long workspace root + nested path exceeds 2048 and overflows the stack buffer (a memory-safety regression that's actually worse than the 0-matches bug, and green CI didn't catch it because no test exercises a long path).

I've distilled your fix into #751 (3dd2471), co-authored with you: same NUL / xargs -0 approach + your Windows newline branch, but the record is written piece-by-piece with fwrite/fputc (no fixed buffer, safe for any length), plus a reproduce-first test that indexes a repo under "<tmp>/my project" and asserts the match is found (RED before → GREEN after). Full suite green.

Closing as superseded by the distilled PR, with full credit to you. Really appreciate the correct diagnosis. 🙏

@DeusData DeusData closed this Jul 1, 2026
kapoorsunny pushed a commit to kapoorsunny/codebase-memory-mcp that referenced this pull request Jul 2, 2026
search_code's scoped grep writes an "<root>/<file>" filelist and pipes it to
xargs. With a newline-separated filelist + plain `xargs`, a space anywhere in
the absolute path splits one file into several bogus arguments, so grep finds
nothing and scoped search returns 0 matches for any repo indexed under a path
containing a space (DeusData#687).

Fix: write NUL-separated records and consume them with `xargs -0`, so each path
stays a single argument regardless of spaces. On Windows the scoped path uses
PowerShell `Get-Content | Select-String -LiteralPath`, which reads line-by-line
and already handles spaces, so the filelist keeps newline separators there.

write_scoped_filelist writes each record piece-by-piece with fwrite/fputc (no
fixed-size stack buffer), so arbitrarily long absolute paths cannot overflow.

Reproduce-first: search_code_scoped_path_with_spaces_issue687 indexes a repo
rooted at "<tmp>/my project" and asserts total_grep_matches > 0 (RED on the old
newline + plain-xargs code -> 0 matches; GREEN after). Full suite green.

Distilled from DeusData#693 (thanks @lg320531124). Keeps that PR's correct NUL/xargs-0
idea and its Windows newline branch, but replaces its char buf[2048] + unbounded
memcpy (a stack buffer overflow reachable via long indexed paths, PATH_MAX 4096)
with the direct piece-by-piece write, and adds the reproduce-first guard it
lacked.

Closes DeusData#687

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-Authored-By: lg320531124 <lg320531124@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority/high Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker. ux/behavior Display bugs, docs, adoption UX

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(search): search_code returns 0 matches when repo path contains spaces on macOS

2 participants