Add nix evaluators

[Rucadi]

Following compiler-explorer/compiler-explorer#6198 I introduce the installation of the evaluators for nix, to avoid the current status where compiler-explorer returns:

Internal Compiler Explorer error: Error: No executable provided
    at Module.execute (file:///infra/.deploy/lib/exec.js:552:15)
    at NixCompiler.exec (file:///infra/.deploy/lib/base-compiler.js:317:27)
    at NixCompiler.runCompiler (file:///infra/.deploy/lib/base-compiler.js:446:35)
    at NixCompiler.runCompiler (file:///infra/.deploy/lib/compilers/nix.js:40:48)
    at NixCompiler.doCompilation (file:///infra/.deploy/lib/base-compiler.js:1784:18)
    at async file:///infra/.deploy/lib/base-compiler.js:2233:63
    at async env.enqueue.abandonIfStale (file:///infra/.deploy/lib/base-compiler.js:2215:25)
    at async file:///infra/.deploy/node_modules/p-queue/dist/index.js:230:36
Compiler returned: -1

and to fix issue compiler-explorer/compiler-explorer#7811

I tested the "ce" command and it installed correctly, the statically-linked binaries however are taken from my own repo, created for this specific purpose since statically-linked versions are not officially distributed.

I have tested these binaries output but I have not tested the integration within compiler-explorer.

[mattgodbolt]

I'm a bit dubious at installing unofficial binaries. We don't do that with any other compiler or language -- is there another way to do this? What's the issue with the official binaries? Just trying to understand

[Rucadi]

I'm a bit dubious at installing unofficial binaries. We don't do that with any other compiler or language -- is there another way to do this? What's the issue with the official binaries? Just trying to understand

I totally know this is not standard and a big leap of faith,

You can get a "release" binary from: https://releases.nixos.org/nix/nix-2.29.0/nix-2.29.0-x86_64-linux.tar.xz

If you extract it, you have these files:

Where in green I tried to mark where nix binary resides, which depends on all the others in one way or the other.

These dependencies are harcoded to be in the path /nix/store:

root@4463495413aa:/tmp/nix-2.29.0-x86_64-linux/store/3qh0pcp0wvqc2gd5qzir4mlfcgwmdfyy-nix-2.29.0/bin# patchelf --print-rpath nix /nix/store/d957sdhrf264qriy8ljf5gvd06pj94s6-nix-util-2.29.0/lib:/nix/store/6f0vw2cndq4rkvnbmra82v20s211xbph-nix-store-2.29.0/lib:/nix/store/5cipxqa03irszsvahvb90631vqbmlng1-nix-fetchers-2.29.0/lib:/nix/store/45m0syxl368gqmnhxxpqpnpng9yvpf84-boehm-gc-8.2.8/lib:/nix/store/idsf5yhic1ndlip4bdrm9a74qwlybavd-nix-expr-2.29.0/lib:/nix/store/bcgs71lziv2x7gfx59b983grjihr42dm-nix-main-2.29.0/lib:/nix/store/9gwx0mni6laljwnlcbnmf02ywgkbh1ag-nix-flake-2.29.0/lib:/nix/store/ahs3n8il4l6szc0ngvhr2prxjx9k1vbw-nix-cmd-2.29.0/lib:/nix/store/g3s0z9r7m1lsfxdk8bj88nw8k8q3dmmg-glibc-2.40-66/lib:/nix/store/gv7z0km39q3fgzavpic8vrl7smh5n2w6-gcc-14.2.1.20250322-lib/lib

What I do to avoid that, is to use nix itself to generate a statically-linked version of the binary, which is generated from a nix flake using official sources, so it can be replicated to see that it yields the exact same hash.

Obviously, I understand that it would be better to get an official distribution from an official repo of the statically-linked version, but due to the nature of nix as package manager, it requires the "setup" of groups/folders/daemons in order to work as package manager, so the use of only the evaluator part of nix is less extended and a statically-linked version is usually not useful at all.

I 100% understand if this is a problem to have it in my repo, but I see that there are some compilers being built for compiler-explorer, maybe we could use the same approach? (if the building is containerized, doing the build from a containerized system with nix already installed is trivial)

[Rucadi]

For clarity, building a statically-liked version from a flakes-enabled nix installation is something like:

nix build github:nixos/nix#.hydraJobs.buildStatic.nix-everything.x86_64-linux
(builds the latest master)
nix build github:nixos/nix/2.29.0#.hydraJobs.buildStatic.nix-everything.x86_64-linux
(builds 2.29.0 specific tag)
Then the binary is in the result/bin/nix

[mattgodbolt]

Thanks @Rucadi ! I'm going to cc @pwaller in case he has thoughts on this nix-y stuff :)

[mattgodbolt]

Would it make sense to have some kind of "nix" installer type instead?

[mattgodbolt]

(I'm not against using your repo, but wondering if there's a more "proper" way that opens the door to other stuff too)

[Rucadi]

Sorry, I don't know if I'm following this:

Would it make sense to have some kind of "nix" installer type instead?

You mean installing nix directly in the machine? if that can be done, then that's fine, it probably is easy to get away with a single-user installation https://nixos.org/download/ depending if systemd is available or not.

From a nix installation, then, you can install any different versions that you want, but the disk usage will be higher.

About:

(I'm not against using your repo, but wondering if there's a more "proper" way that opens the door to other stuff too)

I just wanted to unblock the "nix in compiler-explorer" so it can be already used.

I don't have a any strong desire to maintain those versions in my github and I understand that there may be more trust if they are generated from an action inside this repo, for example.

Generating a statically-linked versions is super easy if you already have a working installation of nix.

[mattgodbolt]

Sorry @Rucadi - I meant a more general systemwide nix install. we can't have a "single-user" install. We'd need it to run and work off of EFS/NFS, so maybe that's a non-starter for this.

[pwaller]

Nix has multiple pieces to it. There is the evaluator of the language, which is what is under test here, and I believe this doesn't require any kind of install (hence the proposal to get a static binary for it). That seems reasonable to me.

I'm in a position to Matt bootstrap into nix and get these static binaires directly from nix build github:nixos/nix#.hydraJobs.buildStatic.nix-everything.x86_64-linux and into NFS if that is what is desired. Or do we need them to come from a URL? I'm unclear on this part.

[mattgodbolt]

I'm trying to understand, really - just trying to get a principled approach to installing these NIX static binaries rather than have them be on @Rucadi 's github page. We can build them ourselves if it's easy (which I think it is) -- we could make our misc-builder project have support for this

[pwaller]

I think there are two approaches:

1. Install nix on the systems.
2. Get your hands on a dependency-free nix binary.

For 2 the most straightforward way is to run nix build github:nixos/nix#.hydraJobs.buildStatic.nix-everything.x86_64-linux on a nix system (if I've understood @Rucadi correctly). Once you have the binary, you can invoke it.

There may be some details about storage, I think the evaluator may write out some files as a side effect of evaluation. That can probably either be redirected to a tmpfs, or possibly configured out entirely, but needs a bit of thought as to whether it's a problem or not.

[Rucadi]

I think there are two approaches:

1. Install nix on the systems.
2. Get your hands on a dependency-free nix binary.

For 2 the most straightforward way is to run nix build github:nixos/nix#.hydraJobs.buildStatic.nix-everything.x86_64-linux on a nix system (if I've understood @Rucadi correctly). Once you have the binary, you can invoke it.

There may be some details about storage, I think the evaluator may write out some files as a side effect of evaluation. That can probably either be redirected to a tmpfs, or possibly configured out entirely, but needs a bit of thought as to whether it's a problem or not.

You are 100% right, that command would be enough to build the latest master branch.

The user can create a store in any accessible place using the --store argument, which I try to mitigate in the compiler-explorer implementation using a dummy store (which doesn't allow writting). If it is desired to support derivations and/or internet access, then the store should point to a tmpfs probably.

I leave here a small Dockerfile that would generate it:

FROM nixos/nix AS builder

RUN nix --experimental-features 'nix-command flakes' build github:nixos/nix/2.29.0#.hydraJobs.buildStatic.nix-everything.x86_64-linux && \
    cp -L result/bin/nix /tmp/nix

FROM scratch
COPY --from=builder /tmp/nix /nix
ENTRYPOINT ["/nix"]