engine_pin() has two routes to a jackdaw commit, and a consumer that installs the kit as a wheel while supplying jackdaw from a local editable checkout hits neither. The commit is sitting right there and resolvable; provenance just never asks for it. Result: commit: None, is_attributable() returns False, and every result stamped by such a consumer is unattributable — even when the engine is at exactly the commit this project pins.
Found while wiring a downstream project up as a consumer of the kit, on main (1.1.0).
Why both routes miss
bench/provenance.py:
REPO_ROOT = PACKAGE_ROOT.parents[1] if PACKAGE_ROOT.parent.name == "src" else None # L72
ENGINE_PATH = REPO_ROOT / "vendor" / "jackdaw-balatro" if REPO_ROOT is not None else None # L73
ENGINE_PATH is None whenever the kit is installed as a wheel — PACKAGE_ROOT.parent is site-packages, not src. It never gets as far as looking at the consumer's engine checkout.
direct_url.json → vcs_info.commit_id is only populated when jackdaw was itself installed from a git URL. A consumer resolving jackdaw from a local path or workspace member gets dir_info instead:
{"url": "file:///…/vendor/jackdaw-balatro", "dir_info": {"editable": true}}
No vcs_info, so commit is None.
Both are correct in isolation, and None is the honest answer to a question the code doesn't know how to ask. The gap is that the two routes between them cover jackhammer's own repo and a git-URL install, and the consumer path is neither.
Reproduce
Any project that installs jackhammer-benchmark from a wheel/git and supplies jackdaw from a local editable checkout:
$ python -c "from jackhammer.bench import provenance as p; print(p.engine_pin())"
{'name': 'jackdaw', 'dist_version': '0.1.0', 'commit': None, 'dirty': None, 'source': 'unknown'}
In my case the checkout is at de733ebd494a5da71fb7049b3b6b18ecd039786c — byte-identical to the commit this project's own pyproject.toml pins — and it still reports unknown.
Suggested fix
There's a third route, and direct_url.json already hands it over: when dir_info is present, url is a filesystem path. git -C <path> rev-parse HEAD (plus the existing status --porcelain for dirty) answers exactly the question _pin already knows how to answer — it's the same code path as ENGINE_PATH, just pointed at a location discovered from metadata rather than assumed from the source layout. Something like:
direct_url = json.loads(dist.read_text("direct_url.json") or "{}")
if "dir_info" in direct_url:
local = _pin(Path(url2pathname(urlparse(direct_url["url"]).path)))
if local["commit"] is not None:
return {..., **local, "source": "editable-checkout"}
dirty genuinely matters for this route in a way it doesn't for installed-vcs: an editable checkout is a mutable source tree, so a consumer editing the engine would correctly stamp dirty: True and fail is_attributable().
Why it's worth fixing rather than documenting
The failure is quiet and it degrades in the wrong direction. A consumer whose engine has drifted from the pinned commit gets stamped unattributable — which reads as a provenance nuisance, not as "this number ran on an engine the kit does not declare." The distinction between unknown and wrong is exactly what the provenance layer exists to preserve, and here they collapse into the same output.
Happy to send a PR if the approach looks right.
engine_pin()has two routes to a jackdaw commit, and a consumer that installs the kit as a wheel while supplyingjackdawfrom a local editable checkout hits neither. The commit is sitting right there and resolvable; provenance just never asks for it. Result:commit: None,is_attributable()returnsFalse, and every result stamped by such a consumer is unattributable — even when the engine is at exactly the commit this project pins.Found while wiring a downstream project up as a consumer of the kit, on
main(1.1.0).Why both routes miss
bench/provenance.py:ENGINE_PATHisNonewhenever the kit is installed as a wheel —PACKAGE_ROOT.parentissite-packages, notsrc. It never gets as far as looking at the consumer's engine checkout.direct_url.json→vcs_info.commit_idis only populated whenjackdawwas itself installed from a git URL. A consumer resolvingjackdawfrom a local path or workspace member getsdir_infoinstead:{"url": "file:///…/vendor/jackdaw-balatro", "dir_info": {"editable": true}}No
vcs_info, socommitisNone.Both are correct in isolation, and
Noneis the honest answer to a question the code doesn't know how to ask. The gap is that the two routes between them cover jackhammer's own repo and a git-URL install, and the consumer path is neither.Reproduce
Any project that installs
jackhammer-benchmarkfrom a wheel/git and suppliesjackdawfrom a local editable checkout:In my case the checkout is at
de733ebd494a5da71fb7049b3b6b18ecd039786c— byte-identical to the commit this project's ownpyproject.tomlpins — and it still reportsunknown.Suggested fix
There's a third route, and
direct_url.jsonalready hands it over: whendir_infois present,urlis a filesystem path.git -C <path> rev-parse HEAD(plus the existingstatus --porcelainfordirty) answers exactly the question_pinalready knows how to answer — it's the same code path asENGINE_PATH, just pointed at a location discovered from metadata rather than assumed from the source layout. Something like:dirtygenuinely matters for this route in a way it doesn't forinstalled-vcs: an editable checkout is a mutable source tree, so a consumer editing the engine would correctly stampdirty: Trueand failis_attributable().Why it's worth fixing rather than documenting
The failure is quiet and it degrades in the wrong direction. A consumer whose engine has drifted from the pinned commit gets stamped
unattributable— which reads as a provenance nuisance, not as "this number ran on an engine the kit does not declare." The distinction between unknown and wrong is exactly what the provenance layer exists to preserve, and here they collapse into the same output.Happy to send a PR if the approach looks right.