libgit2 1.9.4: git_stash_apply doesn't update the working directory (breaks SwiftGit stash apply/pop/branch)

[odrobnik]

Summary

When SwiftPorts' SwiftGit is built against GitKit (libgit2 1.9.4), its
git stash apply / pop / branch operations no longer restore the stashed
change to the working tree. The exact same SwiftGit code passes against the
odrobnik/libgit2 fork SwiftPorts used previously. This blocked SwiftPorts'
adoption of GitKit and was decoupled in Cocoanetics/SwiftPorts#59 (reverted to
the fork to keep stash working).

Repro (SwiftGit GitClientStashTests)

1. git init, commit file.txt = v1.
2. Write file.txt = v2 (working-tree change).
3. git_stash_save → working tree correctly reverts to v1. ✅
4. git_stash_apply(index: 0) → working tree should become v2, but stays v1. ❌

save / list / drop / show / clear all pass; only the operations that
go through git_stash_apply (apply / pop / branch) fail. So git_stash_apply
computes the merge but never writes the working directory.

What I ruled out

• Explicitly setting apply_opts.checkout_options.checkout_strategy = GIT_CHECKOUT_SAFE before git_stash_apply is a no-op — the tests still
  fail with it set.
• In src/libgit2/stash.c, normalize_apply_options() only ORs in
  GIT_CHECKOUT_NO_REFRESH; it does not default the strategy to SAFE
  (unlike what some older versions did). The apply path then
  git_checkout_index(modified_index, …) with that strategy.

Hypothesis (needs confirmation)

After the merge yields modified_index, GIT_CHECKOUT_SAFE skips writing the
file because the working-tree copy looks "modified" relative to the post-merge
index baseline — so nothing is written. The fix is likely an explicit checkout
baseline (HEAD tree) for the stash-apply checkout, or this is a genuine
libgit2 1.9.x change worth an upstream report. GIT_CHECKOUT_FORCE makes the
test pass but can clobber unrelated uncommitted changes, so it's not a safe fix.

Impact / next step

Re-adopting GitKit in SwiftPorts needs either a SwiftGit stashApply fix that
works under libgit2 1.9.4, or confirmation + an upstream libgit2 fix. Until
then SwiftPorts stays on the fork.

[odrobnik]

Root cause confirmed — it's the merge, not the checkout (my original hypothesis was wrong)

Reproduced against GitKit's libgit2 1.9.4 and isolated it:

• Setting the stash-apply checkout strategy to GIT_CHECKOUT_FORCE still leaves the working tree at the pre-stash content (v1, not the stashed v2). Since FORCE writes modified_index unconditionally, this proves git_stash_apply produces a wrong modified_index — the merge step (merge_index_and_tree → modified_index) never incorporates the stashed change. The checkout is faithfully writing a merge result that equals the base.
• Therefore no consumer-side (SwiftGit) change can fix this — checkout strategy / baseline are irrelevant; the bug is upstream of the checkout.

The difference vs the working fork

checkout.c and the git_stash_apply function are byte-identical between libgit2 1.9.0 and 1.9.4. The working fork (odrobnik/libgit2, what SwiftPorts is back on) differs in stash.c's temporary-index creation:

• fork (works): git_index_new_ext(&i, GIT_INDEX_OPTIONS_FOR_REPO(repo))
• 1.9.4 (broken): git_index__new(&i, repo->oid_type)

git_index_new_ext / GIT_INDEX_OPTIONS_FOR_REPO are a newer (sha256-era) index API that does not exist in libgit2 1.9.4. git log v1.9.0..v1.9.4 and ..main show no stash.c changes, so the fork is on a different libgit2 lineage (sha256/experimental), not a newer release.

Fix options for GitKit

1. Vendor a libgit2 with the working stash temp-index handling (the fork's lineage) — but that's not a clean upstream release tag, conflicting with GitKit's "tag == libgit2 release" policy.
2. Backport the stash temp-index change to 1.9.4 (requires the git_index_options / git_index_new_ext API too) via GitKit's compile-time patch mechanism.
3. Check GitKit's build config — confirm whether a libgit2 build define (e.g. sha256/index-related) GitKit sets vs the fork's packaging contributes, since the merge-result bug may be config-sensitive.
4. SwiftGit workaround — reimplement stash apply via git_merge_trees + git_checkout_index (bypassing git_stash_apply), if that path produces the correct merge under 1.9.4.

Until one of these is done, SwiftPorts stays on the fork (decoupled in Cocoanetics/SwiftPorts#59).

[odrobnik]

✅ Verified: building GitKit from libgit2 main fixes the stash bug

Bumped GitKit's vendored libgit2 submodule from v1.9.4 to main (44c05e5d1) and re-ran SwiftGit against it:

• All 7 stash tests pass (apply / pop / branch restore the working tree correctly).
• Full SwiftGit suite: 155/155 on re-run. (One config-search test, bridgeRedirectsConfigSearchToSandboxHome, is flaky — it mutates libgit2's process-global GIT_OPT_SET_SEARCH_PATH, so it's test-ordering sensitive; passes isolated and on re-run. Unrelated to stash.)

main's stash.c builds its temp merge indexes with git_index_new_ext(GIT_INDEX_OPTIONS_FOR_REPO(repo)), which 1.9.4 lacks — that's the fix.

What the libgit2-main bump required (build port vs 1.9.4)

1. reftable backend — main added src/libgit2/refdb_reftable.c, wired in unconditionally, backed by the vendored deps/reftable/ lib. Added deps/reftable to sources: + a header search path, and excluded its CMakeLists.txt/LICENSE.
2. PCRE → PCRE2 — main switched the bundled regex engine from deps/pcre to deps/pcre2 (and GIT_REGEX_BUILTIN now means PCRE2, which needs a full PCRE2 config: PCRE2_CODE_UNIT_WIDTH, PCRE2_STATIC, match limits, char tables…). For this verification I used GIT_REGEX_REGCOMP (system POSIX regex) and dropped the bundled engine — all 155 SwiftGit tests pass with it, so regcomp appears sufficient for SwiftGit's needs, but a production migration should either configure PCRE2 properly or confirm regcomp covers git's full regex surface.
3. Re-vendored the public headers for main.

Open decision (not code)

main isn't a release tag, which conflicts with GitKit's "tag == libgit2 release" policy. Options:

• Pin a main commit now (stopgap; pre-release, moving target).
• Wait for libgit2 to ship the fix in a release (1.9.5 / 1.10) and bump then — cleanest.
• Backport just the stash temp-index change to 1.9.4 (needs the git_index_options/git_index_new_ext API backported too).

A working experiment/libgit2-main branch exists locally with all of the above.

[odrobnik]

libgit2 1.9.2 also fails — the bug spans all 1.9.x releases, fix is main-only

Tested GitKit built from libgit2 v1.9.2 (a clean release tag, no reftable/PCRE2 porting needed): SwiftGit stash apply/pop/branch still fail, identical to 1.9.4.

So there is no 1.9.x release that fixes this — stash.c is unchanged across v1.9.0→v1.9.4 (all use the buggy git_index__new(repo->oid_type) for the temp merge index), and the fix (git_index_new_ext(GIT_INDEX_OPTIONS_FOR_REPO(repo))) only landed on main. The odrobnik fork works because it carries that main change on an older base — not because it's 1.9.2.

Net: the fix requires libgit2 main

Options unchanged, but with no clean-release shortcut:

1. Pin a libgit2 main commit (pre-release) + the build port (reftable, PCRE2/regcomp). Verified working.
2. Wait for libgit2 to ship the fix in a tagged release (next after 1.9.4) and bump GitKit then.
3. Backport the git_index_options/git_index_new_ext API + the stash temp-index change to 1.9.4.

[odrobnik]

Final root cause — correcting this issue's premise: libgit2 1.9.4 is NOT buggy

Full archaeology (fork lineage + upstream bisect + define isolation) found the real story:

1. The ibrahim fork never patched stash. Its v1.9.2 branch (and the 1.9.1/1.9.2 tags both forks publish) is a libgit2 main snapshot — merge-base 5ab90d3c2, v1.9.0+115, June 2025 — plus packaging commits. It is not based on the v1.9.2 release (it even contains commits the real v1.9.2 lacks). It "works" because main works.
2. The fix commit I was hunting doesn't exist. Bisecting v1.9.0..main with GitKit's (old) define matrix flips from fail to pass exactly at the merge of libgit2 PR #6994 — a build-system PR that renamed the feature defines (GIT_USE_NSEC→GIT_NSEC etc.). No source bugfix anywhere; the flip is the dialect boundary.
3. GitKit's matrix (inherited from the main-based fork) speaks the post-#6994 dialect; the pinned v1.9.4 reads the pre-#6994 one. Result: GIT_USE_NSEC (and TLS-backend selection, NTLM, iconv) silently compiled out. Without nsec, same-second working-tree changes are invisible to stat comparison — exactly what git_stash_apply/pop/branch trip over. Isolation test: flipping only GIT_USE_NSEC↔GIT_NSEC toggles the stash failures on/off.
4. My earlier comments' hypotheses (broken merge in 1.9.x, git_index_new_ext as the fix, "pin main / backport" options) are superseded — the git_index_new_ext stash.c change is a cosmetically equivalent API rename (GIT_INDEX_OPTIONS_FOR_REPO(r) = {version, r->oid_type}, same value as before).

Fix: #2 — translate the define matrix to v1.9.4's spellings. Submodule stays pristine at the release tag; verified all 7 stash tests + full 155-test SwiftGit suite green on macOS. Bonus: TLS/NTLM/iconv are now actually enabled (they were silently off in the shipped 1.9.4 build).