Skip to content

engine_pin() cannot resolve a commit when jackdaw is an editable/local install #15

Description

@idIing

engine_pin() has two routes to a jackdaw commit, and a consumer that installs the kit as a wheel while supplying jackdaw from a local editable checkout hits neither. The commit is sitting right there and resolvable; provenance just never asks for it. Result: commit: None, is_attributable() returns False, and every result stamped by such a consumer is unattributable — even when the engine is at exactly the commit this project pins.

Found while wiring a downstream project up as a consumer of the kit, on main (1.1.0).

Why both routes miss

bench/provenance.py:

REPO_ROOT = PACKAGE_ROOT.parents[1] if PACKAGE_ROOT.parent.name == "src" else None   # L72
ENGINE_PATH = REPO_ROOT / "vendor" / "jackdaw-balatro" if REPO_ROOT is not None else None  # L73
  1. ENGINE_PATH is None whenever the kit is installed as a wheel — PACKAGE_ROOT.parent is site-packages, not src. It never gets as far as looking at the consumer's engine checkout.
  2. direct_url.jsonvcs_info.commit_id is only populated when jackdaw was itself installed from a git URL. A consumer resolving jackdaw from a local path or workspace member gets dir_info instead:
{"url": "file:///…/vendor/jackdaw-balatro", "dir_info": {"editable": true}}

No vcs_info, so commit is None.

Both are correct in isolation, and None is the honest answer to a question the code doesn't know how to ask. The gap is that the two routes between them cover jackhammer's own repo and a git-URL install, and the consumer path is neither.

Reproduce

Any project that installs jackhammer-benchmark from a wheel/git and supplies jackdaw from a local editable checkout:

$ python -c "from jackhammer.bench import provenance as p; print(p.engine_pin())"
{'name': 'jackdaw', 'dist_version': '0.1.0', 'commit': None, 'dirty': None, 'source': 'unknown'}

In my case the checkout is at de733ebd494a5da71fb7049b3b6b18ecd039786c — byte-identical to the commit this project's own pyproject.toml pins — and it still reports unknown.

Suggested fix

There's a third route, and direct_url.json already hands it over: when dir_info is present, url is a filesystem path. git -C <path> rev-parse HEAD (plus the existing status --porcelain for dirty) answers exactly the question _pin already knows how to answer — it's the same code path as ENGINE_PATH, just pointed at a location discovered from metadata rather than assumed from the source layout. Something like:

direct_url = json.loads(dist.read_text("direct_url.json") or "{}")
if "dir_info" in direct_url:
    local = _pin(Path(url2pathname(urlparse(direct_url["url"]).path)))
    if local["commit"] is not None:
        return {..., **local, "source": "editable-checkout"}

dirty genuinely matters for this route in a way it doesn't for installed-vcs: an editable checkout is a mutable source tree, so a consumer editing the engine would correctly stamp dirty: True and fail is_attributable().

Why it's worth fixing rather than documenting

The failure is quiet and it degrades in the wrong direction. A consumer whose engine has drifted from the pinned commit gets stamped unattributable — which reads as a provenance nuisance, not as "this number ran on an engine the kit does not declare." The distinction between unknown and wrong is exactly what the provenance layer exists to preserve, and here they collapse into the same output.

Happy to send a PR if the approach looks right.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions