Read missing annexed content with datalad-fuse, cached by annex key

[CodyCBakerPhD]

This PR and the one stacked on it are an alternative to #1941 and #1933. They build on datalad-fuse 0.5.1, the current PyPI release, instead of a git-only reader of the git-annex branch. The goal is the same: dandi validate on a DataLad clone of a Dandiset, streaming file content without downloading it.

Changes

• dandi.support.datalad_fuse.AnnexedReadableFile is a Readable for a locked annexed file whose content isn't present locally.
  • It streams the content through datalad-fuse's FsspecAdapter (no FUSE mount, caching=False). The adapter asks git-annex where the content is (whereis) and reads it over HTTP(S) with fsspec.
  • get_annexed_readable(path) returns None in any of these cases:
    • datalad-fuse isn't installed;
    • the path isn't a broken symlink;
    • git-annex isn't initialized in the repository (otherwise DataLad would initialize it);
    • the key records no size;
    • git-annex knows no URL for the content.
• Caching by annex key: get_metadata, get_neurodata_types and nwb_has_external_links use the new pynwb_utils.annex_fingerprint as their custom_fingerprint.
  • It wraps fscacher 0.5.0's annex_key_fingerprint, which keys on the path paired with the annex key.
  • Locked annexed files are cached by key, whether they are streamed or present locally. Only content-hash backends count, so WORM and URL keys fall back to stat().
  • Unlike Memoize readable fingerprints #1941, this needs no fingerprint hook on Readable.
• Dependencies:
  • New datalad extra: datalad-fuse >= 0.5.1, which also requires git-annex.
  • fscacher >= 0.5.0.
• Tests:
  • make_git_annex_dandiset() builds a real git-annex repository (content added, URL registered, then dropped).
  • A range_http_server fixture serves files with range requests.
  • New skipif.no_git_annex condition.
  • A new CI job (mode: datalad, ubuntu, Python 3.12) installs git-annex and .[datalad] from PyPI. The other jobs skip these tests.

Compared to #1941

410 added lines against #1941's 990. It drops AnnexKey, AnnexRepo, the URL-log parsing, the fake-annex fixtures and the Readable.get_fingerprint hook. In exchange, streaming requires git-annex and DataLad.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Qn5WBSiQgoZoL4fytF6nEr

---

Generated by Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 92.91339% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.86%. Comparing base (092c448) to head (7964633).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
dandi/support/datalad_fuse.py	87.37%	13 Missing ⚠️
dandi/tests/fixtures.py	92.95%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1955      +/-   ##
==========================================
+ Coverage   90.73%   90.86%   +0.12%     
==========================================
  Files          92       94       +2     
  Lines       14130    14381     +251     
==========================================
+ Hits        12821    13067     +246     
- Misses       1309     1314       +5     

Flag	Coverage Δ
unittests	90.86% <92.91%> (+0.12%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.