Skip to content

Commit ad64852

Browse files
committed
merge: skip the manifest git probe outside a work tree (compactfix)
2 parents ff49aa8 + 2d7435a commit ad64852

3 files changed

Lines changed: 150 additions & 0 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ All notable changes to Token-Goat are documented in this file. Format follows Ke
1010

1111
### Fixed
1212

13+
- **The pre-compact manifest no longer runs git in a folder that is not inside a git repository.** When the manifest outgrew its size cap it asked git whether the working tree was dirty, even for a folder with no repository, where the answer could only be "nothing". On a slow machine that git run was cut off at the 50 ms limit, and the probe was likely still holding the folder when a Windows test tried to delete its scratch folder (`EBUSY`); this cause was not reproduced locally. The probe now looks for a `.git` entry in the folder or a parent (following a junction or symlink to the real folder), and skips git when there is none. A set `GIT_DIR` still counts as a repository.
1314
- **A search that walks a whole disk is now also refused when it follows a `cd` to a root or home directory.** `cd / && find . -name x` walks the same disk as `find / -name x`, but the check only looked at the path given to `find`, so the `cd` hid it. The check now follows the `cd`, and only a search of the whole directory is refused: a `find` limited to a depth, a name pattern in a subfolder or a path below the root still runs.
1415
- **A file you already read is no longer blocked from being read again when it has changed.** The refusal to re-read a whole file (markdown, JSON and HTML, source files with a line count, truncated reads, session output and tool-result files, and the `.improve-state` JSON) now fires only when the file is known to be unchanged: the saved bytes still match, or the size and modified time are unchanged. A file that was empty when first read and has since been filled counts as changed, and the 50 ms of slack that let a quick edit pass for no change is gone. The separate refusal for memory files is removed.
1516
- **Pulling or rebasing in a checkout that is not your installed token-goat no longer restarts your worker or repoints your hooks.** The git hook that runs after a pull already did this only for the main checkout. It now also checks where the global `token-goat` package (from `npm root -g`) really lives. If that is a different copy, such as the npm release, the hook only rebuilds the bundle and says so, so developing token-goat while running the npm copy no longer swaps your hooks, shims and worker onto the checkout's build. When npm cannot say, it syncs as before. The hook now prints which case it took, and `TG_POST_MERGE_SYNC=1` forces the full sync.

‎src/manifest.ts‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
/** The compaction manifest: what this session touched, rendered for whoever reads it next. Ports the intent of `hooks_compact.py` / `build_manifest`: a concise summary of the files read, the files edited and the web URLs fetched, so a compaction preserves that context instead of dropping it. The manifest is intentionally compact -- aim well under 2000 chars. Two things the discarded builder did are deliberately not here. It inferred a session goal from the file names it saw and put that sentence at the top of the injected text: an inference stated as fact to the one reader who cannot check it, and wrong exactly when a session changed direction, which is when a manifest matters most. It also grouped rows by directory, which collapses the per-file read counts the rows exist to carry. Its noise-path filtering was worth keeping and is here, through {@link isNoisePath}. Reopen either decision by measuring against a real compaction, not by preference. It lives apart from the hook that emits it because it has two callers with different jobs. `hooks_compact.ts` builds it at `pre_compact` and hands it to the summarizing model; `cache_session_commands.ts` builds it for `compact-hint`, which reports what the next compaction will carry. Those two used to build different text from different code, so the command sized a manifest the hook would never produce and a reader checking one learned nothing about the other. One builder, two callers, and the module holds no `registerHook` call of its own -- importing it from a command must not install a hook as a side effect. */
22

3+
import { existsSync, realpathSync } from 'node:fs'
4+
import { dirname, join, resolve } from 'node:path'
5+
36
import { resolveOnPath, spawnResolvedSync } from './process_util.js'
47

58
import { WEB_FETCH_KEY_SEP, epochReadCounts, getSessionFiles, getSessionWebFetches, getSessionBashOutputs, getSessionBashReruns, mergeEpochBase, wasEditedSinceCompaction } from './session.js'
@@ -169,8 +172,24 @@ export function buildManifest(sessionId?: string, cwd?: string, transcriptPath?:
169172
return buildManifestParts(sessionId, cwd, transcriptPath).text
170173
}
171174

175+
/** Whether `cwd` or an ancestor holds a `.git` entry (a directory, or the file a linked worktree or submodule has), the marker git itself walks up for. An exported GIT_DIR names a repository independent of the directory, so it counts as inside. Looks only at the filesystem, so the answer does not depend on how long a process takes to start. */
176+
function insideGitWorkTree(cwd: string): boolean {
177+
if (process.env['GIT_DIR'] !== undefined) return true
178+
let dir = resolve(cwd)
179+
// Git walks up from the physical directory, so a junction or symlink is followed first; an unreadable path keeps the unresolved spelling.
180+
try { dir = realpathSync.native(dir) } catch { /* keep the unresolved path */ }
181+
for (;;) {
182+
if (existsSync(join(dir, '.git'))) return true
183+
const parent = dirname(dir)
184+
if (parent === dir) return false
185+
dir = parent
186+
}
187+
}
188+
172189
/** Detect real, pre-compaction git dirty state for `cwd` -- `hasPendingDiff` mirrors the Python predecessor's `_get_git_diff_stat_summary()` signal (`git diff --stat HEAD` non-empty: tracked working-tree/staged changes vs HEAD), `hasUncommittedChanges` mirrors `_get_uncommitted_changes()` (`git status --porcelain` non-empty: also catches untracked files `diff --stat HEAD` misses). Both feed {@link computeAdaptiveBudget}'s git-derived bonuses via {@link adaptiveCharBonus}. Uses {@link runGit} (the only git spawn site in the codebase) with `hints.git_hint_max_ms` (same bound `hooks_session.ts`'s own git-hint calls use) so a slow/hung git can never block compaction; any spawn failure or non-zero exit fails soft to `false` for that signal. */
173190
function gitDirtySignals(cwd: string): { hasPendingDiff: boolean; hasUncommittedChanges: boolean } {
191+
// A directory outside every work tree has no dirty state to read; git would only exit 128 after a spawn that, past the short probe bound, is killed while its process may still hold `cwd` open.
192+
if (!insideGitWorkTree(cwd)) return { hasPendingDiff: false, hasUncommittedChanges: false }
174193
const timeoutMs = loadConfig().hints.git_hint_max_ms
175194
let hasPendingDiff = false
176195
try {
Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
/** The manifest's git dirty-state probe must not spawn git for a directory outside every work tree. CI run 38987886278 failed tests/compact.test.ts on Windows with EBUSY from rmSync of a scratch project directory: the two failing tests were the only ones that both pass a cwd and exceed the manifest cap, which is what runs `git diff` and `git status` there under the 50 ms hints.git_hint_max_ms bound. A spawn that is killed at that bound is not finished with the directory it started in, and in a non-repository the spawn could only ever answer "nothing". */
2+
3+
import * as fs from 'node:fs'
4+
import * as os from 'node:os'
5+
import * as path from 'node:path'
6+
7+
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
8+
9+
import { CAN_JUNCTION } from './helpers/can-symlink.js'
10+
11+
import { importSessionState, recordFileEdit, recordFileRead } from '../src/session.js'
12+
import { saveSessionState } from '../src/session_store.js'
13+
import { buildManifest } from '../src/manifest.js'
14+
import { runGit } from '../src/util.js'
15+
16+
// No real git runs in this file: the probe's argument and cwd are what is asserted, so the answer cannot depend on how fast a process starts.
17+
vi.mock('../src/util.js', async (importOriginal) => {
18+
const actual = await importOriginal<Record<string, unknown>>()
19+
return { ...actual, runGit: vi.fn(() => ({ stdout: '', stderr: '', exitCode: 0 })) }
20+
})
21+
22+
const runGitMock = vi.mocked(runGit)
23+
24+
// HAND-DERIVED: every fixture is built on disk by this file (a bare `.git` marker, a junction to a folder holding one) and the expected probe calls follow from what git itself walks up for, not from reading the matcher.
25+
26+
/** The nearest ancestor of `dir` that holds a `.git` entry, or null when none does. */
27+
function gitAncestor(dir: string): string | null {
28+
for (let cur = path.resolve(dir); ; cur = path.dirname(cur)) {
29+
if (fs.existsSync(path.join(cur, '.git'))) return cur
30+
if (path.dirname(cur) === cur) return null
31+
}
32+
}
33+
34+
/** A session whose manifest is well past the default 1600-character cap, so the cap engages and the adaptive bonus (the git probe) is consulted. */
35+
function seedLongSession(sessionId: string): void {
36+
recordFileEdit('/proj/src/edited.ts')
37+
for (let i = 0; i < 45; i++) recordFileRead(`/proj/src/read-only-file-number-${i}.ts`)
38+
saveSessionState(sessionId)
39+
}
40+
41+
describe('manifest git probe needs a work tree', () => {
42+
const dirs: string[] = []
43+
let prevGitDir: string | undefined
44+
45+
beforeEach(() => {
46+
importSessionState({ files: [], hintsShown: [], webFetches: [], bashOutputs: [], curlDownloads: [] })
47+
runGitMock.mockClear()
48+
prevGitDir = process.env['GIT_DIR']
49+
delete process.env['GIT_DIR']
50+
})
51+
52+
afterEach(() => {
53+
if (prevGitDir === undefined) delete process.env['GIT_DIR']
54+
else process.env['GIT_DIR'] = prevGitDir
55+
for (const dir of dirs.splice(0)) fs.rmSync(dir, { recursive: true, force: true })
56+
})
57+
58+
function scratch(): string {
59+
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'tg-manifest-gitprobe-'))
60+
dirs.push(dir)
61+
return dir
62+
}
63+
64+
it('spawns no git for a directory outside every work tree, even when the cap engages', () => {
65+
const projectDir = scratch()
66+
const ancestor = gitAncestor(projectDir)
67+
expect(ancestor, `precondition: the temp directory ${os.tmpdir()} must sit outside every work tree, but ${String(ancestor)} holds a .git entry; point TMPDIR/TEMP at a folder outside it`).toBeNull()
68+
fs.writeFileSync(path.join(projectDir, 'package.json'), '{}')
69+
seedLongSession('no-worktree-session')
70+
71+
const manifest = buildManifest('no-worktree-session', projectDir)
72+
73+
expect(manifest, 'the cap must engage, or this asserts on a manifest that never consulted the probe').toContain('manifest truncated at ')
74+
expect(runGitMock).not.toHaveBeenCalled()
75+
})
76+
77+
it('still probes git when the directory itself is a work tree', () => {
78+
const projectDir = scratch()
79+
fs.mkdirSync(path.join(projectDir, '.git'))
80+
seedLongSession('worktree-session')
81+
82+
const manifest = buildManifest('worktree-session', projectDir)
83+
84+
expect(manifest).toContain('manifest truncated at ')
85+
const probes = runGitMock.mock.calls.map(([args, opts]) => [args[0], opts?.cwd])
86+
expect(probes).toEqual([['diff', projectDir], ['status', projectDir]])
87+
})
88+
89+
it('still probes git when a work tree is an ancestor of the directory', () => {
90+
const root = scratch()
91+
fs.writeFileSync(path.join(root, '.git'), 'gitdir: ../elsewhere')
92+
const projectDir = path.join(root, 'packages', 'app')
93+
fs.mkdirSync(projectDir, { recursive: true })
94+
seedLongSession('nested-session')
95+
96+
buildManifest('nested-session', projectDir)
97+
98+
expect(runGitMock.mock.calls.map(([args]) => args[0])).toEqual(['diff', 'status'])
99+
})
100+
101+
it.skipIf(!CAN_JUNCTION)('still probes git when the directory is a junction into a work tree', () => {
102+
const repo = scratch()
103+
fs.mkdirSync(path.join(repo, '.git'))
104+
const target = path.join(repo, 'app')
105+
fs.mkdirSync(target)
106+
const outside = scratch()
107+
const link = path.join(outside, 'link')
108+
fs.symlinkSync(target, link, 'junction')
109+
expect(gitAncestor(outside), `precondition: ${outside} must sit outside every work tree, but ${String(gitAncestor(outside))} holds a .git entry`).toBeNull()
110+
seedLongSession('junction-session')
111+
112+
try {
113+
buildManifest('junction-session', link)
114+
} finally {
115+
fs.rmSync(link, { recursive: true, force: true })
116+
}
117+
118+
expect(runGitMock.mock.calls.map(([args]) => args[0])).toEqual(['diff', 'status'])
119+
})
120+
121+
it('still probes git when GIT_DIR names the repository', () => {
122+
const projectDir = scratch()
123+
process.env['GIT_DIR'] = path.join(projectDir, 'elsewhere.git')
124+
seedLongSession('git-dir-session')
125+
126+
buildManifest('git-dir-session', projectDir)
127+
128+
expect(runGitMock.mock.calls.map(([args]) => args[0])).toEqual(['diff', 'status'])
129+
})
130+
})

0 commit comments

Comments
 (0)