Build-time flake inputs

[edolstra]

Motivation

This PR does two things:

• It adds a new builtin derivation builder named builtin:fetch-tree, which is similar to builtins.fetchTree but works at build time rather than eval time.

• It allows flake inputs to be fetched at build time. Example:

  inputs.repo1 = {
    type = "github";
    owner = "DeterminateSystems";
    repo = "blabla";
    flake = false; # <-- currently required
    buildTime = true;
  };

  call-flake.nix maps this to a builtin:fetch-tree derivation. Thus you can pass it to other derivations, and it will be fetched at build time rather than eval time. (It will still be fetched at eval time to create/update locks.)

  Inputs like this can be passed to derivations as usual, e.g.

  outputs = { self, nixpkgs, repo1 }: {
    packages.x86_64-linux.default =
      with import nixpkgs { system = "x86_64-linux"; };
      stdenv.mkDerivation {
        ...
        src = repo1;
      };
  };

Note that trying to access them at evaluation time (e.g. import repo1) will trigger import-from-derivation behaviour, i.e. it will cause a build during evaluation.

Context

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[github-actions]

🚀 Deployed on https://68a5df7d04548729ff2c4c80--determinate-nix.netlify.app

[ConnorBaker]

My memory is that, because of the way derivations are added to the store by way of registerOutputs, they are scanned for references to the Nix store. Since the eval-time fetcher doesn't do that (and so the retrieved sources can contain paths to the Nix store), the build-time fetcher will reject some of the sources which the eval-time fetcher would accept (since fixed-output derivations are not allowed to contain references to the Nix store).

Is that correct or something to worry about?

[edolstra]

Nix only scans for references that are part of the input closure of a derivation. It doesn't scan for arbitrary references. For the build-time fetcher, the input closure is empty, so no sources will ever be rejected by the build-time fetcher. This also means Nix won't find references that are "hard-coded" (e.g. part of the tarball), but that's the same for other types of derivations (e.g. fetchurl in Nixpkgs).

[cole-h]

Aside from my 1 comment, this looks fine to me!

[cole-h]

What are the implications of not doing this / is it small enough to fix this now?

[edolstra]

Just that it's slower / uses more disk space than necessary, since it will first fetch into the tarball cache and then copy it to the store. But we can improve that later.