Skip to content

Fix git-annex build dependencies and test environment - #298

Draft
yarikoptic-gitmate wants to merge 2 commits into
masterfrom
claude/vibrant-ptolemy-pgp75l
Draft

yarikoptic-gitmate wants to merge 2 commits into
masterfrom
claude/vibrant-ptolemy-pgp75l

Conversation

@yarikoptic-gitmate

@yarikoptic-gitmate yarikoptic-gitmate commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

git-annex 10.20261005 (upstream cac57ad, "Support building with crypton 1.1.0") now always Build-Depends on ram, the fork of memory that crypton >= 1.1.0 switched to. That broke every build at dependency resolution.

  • Stack builds (macOS, macOS ARM64, Windows): upstream fixed these in 10.20261006 (ed720b7, "Fix stack build."), which adds ram-0.22.0 to stack.yaml. This PR therefore no longer carries a stack.yaml patch.
  • Ubuntu/Debian standalone build: still fails in our buildenv image with Cabal-8010 … missing … ram. This PR fixes that.

The Debian build still uses crypton 1.0.4, so git-annex keeps importing memory. ram only has to be present to satisfy the dependency.

Changes

  • buildenv (ff29fb9c8d):

    • bump DEBIAN_SNAPSHOT to 20261006;
    • apt-get install libghc-ram-dev, which build-dep does not pull in.

    forky provides libghc-ram-dev 0.19.0-1. Otherwise the bump left the toolchain alone: ghc 9.10.3, crypton 1.0.4 and git 2.53.0 are unchanged. The image only gets published from master, so the Ubuntu build picks this up after merge.

  • template (e3295ec2fe): the template now uses the same dawidd6/action-send-mail v21 pin as the generated workflows. Dependabot's bump in [gh-actions](deps): Bump dawidd6/action-send-mail from 17 to 21 #292 only edited the generated files, so regenerating would have reverted it. The generated workflows are unchanged.

The datalad psutil warning that broke test_run_datalad_help is not handled here. datalad 1.7.1 no longer emits it at import time.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JSwNgwK1r3h7xSYGduuhRa

Since git-annex 10.20261005 (upstream cac57ad "Support building with
crypton 1.1.0") the executable unconditionally Build-Depends on ram, the
fork of memory that crypton >= 1.1.0 switched to.  Our debianstandalone
build has failed since then (e.g. run 37441250450):

  Configuring git-annex-10.20261005...
  Error: [Cabal-8010]
  Encountered missing or private dependencies:
      ram

`apt-get build-dep git-annex` follows Debian's older git-annex, which
does not need ram, and the 20260924 snapshot image has no ram at all.
Install libghc-ram-dev explicitly, and move the snapshot forward to
today so that it is available, rather than building ram from Hackage as
we do for file-io.

Not verified locally: this environment cannot reach any Debian host,
so the build-linux-buildenv job is the first real check that
libghc-ram-dev is in forky as of 20261006.  What was verified, on the
current datalad/buildenv-git-annex:latest (GHC 9.10.3, crypton 1.0.4):
once ram (0.22.1) is in the global package db, `./Setup configure` and a
full `./Setup build` of upstream 10.20261005 succeed, and `git-annex
version` still lists OsPath.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JSwNgwK1r3h7xSYGduuhRa

Copy link
Copy Markdown
Collaborator Author

Build git-annex on Ubuntu / build-package fails here with the same error as on master: Error: [Cabal-8010] Encountered missing or private dependencies: ram. See run 37480591701.

That is expected, and this PR cannot make it pass:

  • The Ubuntu job builds inside the published datalad/buildenv-git-annex:latest image.
  • This PR fixes that image in ff29fb9 by bumping DEBIAN_SNAPSHOT to 20261006 and installing libghc-ram-dev.
  • build-linux-buildenv.yaml only pushes the image from master, so the Ubuntu job keeps using the old image until this PR is merged.

What this PR can check is whether the new image builds, i.e. whether libghc-ram-dev exists in forky at that snapshot. Build Linux buildenv image (run 37480591697) is running on this head now.

I am not re-running the Ubuntu job: it would fail the same way until the new image is published. After the merge and image push, the next Ubuntu run, scheduled or dispatched, should pick it up.


Generated by Claude Code

Copy link
Copy Markdown
Collaborator Author

Build git-annex on Windows / test-datalad (release) still fails, but for a different reason (job). The psutil change fixed the original failure: test_run_datalad_help no longer appears. Installing psutil also un-skipped datalad's psutil-dependent open-file tests, and three of them fail on Windows:

FAILED datalad/support/tests/test_openfiles.py::test_lsof_fallback_triggered - AssertionError: lsof fallback should detect write-open file: {}
  assert 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\datalad_temp_tree_…\\testfile.txt' in {}
FAILED datalad/support/tests/test_openfiles.py::test_cli_detects_open_file - AssertionError: assert 0 == 1  (stdout='No files open for writing.')
FAILED datalad/tests/test_utils.py::test_get_open_files - KeyError: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\datalad_temp_tree_…\\1'
= 3 failed, 1224 passed, 219 skipped …

All three involve a temp path that the runner gives in 8.3 short form (C:\Users\RUNNER~1\…), while Path.resolve() returns the long form (C:\Users\runneradmin\…). Most likely datalad's open-file detection (datalad/support/openfiles.py, get_open_files) and these tests do not normalize the two forms consistently, so nothing matches. I have not verified this.

This is a datalad bug on Windows, not a git-annex one, and this PR has no clean fix for it:

  • Skipping psutil on Windows would bring back the original failure.
  • Pointing TEMP at a long-form path would only hide the datalad bug.

The datalad-side fix belongs with datalad/datalad#7954. I am not re-running the job, since the failure is deterministic.


Generated by Claude Code

Copy link
Copy Markdown
Collaborator Author

Build git-annex on macOS / test-annex (custom-config1, macos-15-intel) failed (job). git-annex deadlocked inside its own test suite. In the testremote group exporttree=yes; key size 1048577; key size 2097152, the log shows nothing from 16:54:47 to 17:01:22 and then:

git-annex: thread blocked indefinitely in an STM transaction

This does not appear to be caused by this PR:

  • No relevant code change. The only upstream code change since the last green Intel build (10.20260901+git90-g8ce197e622) is cac57ad in Utility/Hash/Crypton.hs. With lts-24.52's crypton-1.0.6 it still compiles the memory import, and the ram extra-dep this PR adds goes unused. The tested binary is therefore functionally the same as in the previous runs.
  • Only this job failed. On this head the same test suite passed in every other test-annex job: Intel normal and crippled-tmp, ARM64 normal, crippled-tmp, custom-config1 and macos-latest, and Windows normal and custom-config1. ARM64 custom-config1 runs the same annex.stalldetection=1KB/120s config.
  • History. This job passed in each of the previous 36 macOS Intel runs that reached the tests (2026-09-07 to 2026-10-05). This is its first STM deadlock.

So this looks like a rare, timing-dependent STM deadlock in git-annex itself, here with stall detection on the slower Intel runner. No fix exists yet. GitHub only allows re-running failed jobs once a run has finished, so I will re-run it once after this macOS run completes. If it fails the same way again, it is a reproducible upstream bug worth reporting to git-annex.


Generated by Claude Code

@yarikoptic
yarikoptic force-pushed the claude/vibrant-ptolemy-pgp75l branch from dce8290 to d76543b Compare October 6, 2026 21:39
…lows

Dependabot's bump of dawidd6/action-send-mail to v21 (#292) only edited
the generated build-*.yaml workflows, so the .j2 template still pinned
the v17 SHA.  Regenerating with `make -C .github/workflows/template`
would therefore silently revert that bump.  Set the template to the same
v21 SHA; regenerating now leaves the generated workflows unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JSwNgwK1r3h7xSYGduuhRa
@yarikoptic
yarikoptic force-pushed the claude/vibrant-ptolemy-pgp75l branch from d76543b to e3295ec Compare October 6, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants