Skip to content
cramtPublic

About

Pure-Nix builder for pnpm v9 monorepos — no pnpm install at build time

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

pnpm2nix

Pure-Nix builder for pnpm v9 monorepos. Turns pnpm-lock.yaml into per-app Nix derivations without running pnpm install at build time.

Why

pnpm install inside a Nix sandbox is fragile — it wants network, a global store, and lifecycle scripts. This project reconstructs pnpm's node_modules/.pnpm layout directly from the lockfile using only Nix primitives (fetchurl, runCommand, hardlinks, symlinks).

Key properties

  • No pnpm install at build time. Packages are fetched individually via fetchurl with integrity hashes from the lockfile, then assembled into pnpm's virtual store layout.
  • Platform filtering. Native binaries for foreign platforms (e.g. @esbuild/win32-x64 on a linux build) are automatically excluded. Cuts the store size roughly in half for typical monorepos.
  • Per-dependency caching. Every snapshot is its own derivation ("cell"); the shared .pnpm/ view is a pure-symlink compose layer per app, pruned to that app's dependency closure. Bumping a dep rebuilds the affected cells and the apps whose closure actually changed — nothing else.
  • Source isolation. Each app build only sees its own source tree. Touching app B doesn't invalidate app A's cache.
  • Workspace prepare scripts. Shared packages' prepare (a types build, codegen) runs once per package and every app builds against the prepared tree, as after a local pnpm install.

Quick Start

Flake input

{
  inputs = {
    pnpm2nix.url = "github:cramt/pnpm2nix";
    pnpm2nix.inputs.nixpkgs.follows = "nixpkgs";
  };
}

Basic usage

let
  pnpm2nix = inputs.pnpm2nix.lib.${system};

  workspace = pnpm2nix.mkPnpmWorkspace {
    workspace = ./.;
    apps = [
      { name = "web"; path = "apps/web"; }
      { name = "api"; path = "apps/api"; }
    ];
    packages = [ "packages/ui" "packages/utils" ];
    nodejs = pkgs.nodejs_22;
    # `pnpm` is optional — if omitted, pnpm2nix reads the workspace's
    # package.json "packageManager" field and builds the matching version
    # from npm using a hash from pnpm-versions.json.
  };
in {
  packages.web = workspace.apps.web;
  packages.api = workspace.apps.api;
}

pnpm version handling

pnpm2nix derives the pnpm version from the workspace's package.json packageManager field (the same field corepack reads). This keeps Nix builds, CI, and local development on the exact same pnpm version without anyone having to bump it in multiple places.

{
  "packageManager": "pnpm@10.5.2"
}

Versions are looked up in pnpm-versions.json at the root of this repo, which maps <version> → npm tarball SRI integrity hash.

pnpm 12+ is a native Rust binary; its pnpm npm tarball is only a wrapper that downloads the binary on first run, which the sandbox forbids. For those versions pnpm2nix fetches the prebuilt @pnpm/exe.<target> package instead (the static musl build on linux), with hashes from pnpm-exe-versions.json (<version> → <target> → integrity).

Both files are populated by:

node scripts/update-pnpm-versions.mjs

The script fetches the npm packuments for pnpm (and the @pnpm/exe.* targets) and writes/updates every published version's integrity hash. It is idempotent — existing entries are preserved (npm tarballs are immutable), only newly-published versions are added. Run it whenever you bump pnpm and the new version isn't yet in the file (you'll get an actionable error from Nix telling you to run it).

To override the auto-derived pnpm explicitly — e.g. for a one-off experiment or to use a pnpm with custom patches — pass pnpm to mkPnpmWorkspace, or use the mkPnpm helper:

let
  pnpm2nix = inputs.pnpm2nix.lib.${system};
in
  pnpm2nix.mkPnpmWorkspace {
    # ...
    pnpm = pnpm2nix.mkPnpm {
      version = "10.5.2";
      hash = "sha512-...";
      nodejs = pkgs.nodejs_22;
    };
  }

API

mkPnpmWorkspace

The main entry point. Takes a workspace config and returns built apps.

pnpm2nix.mkPnpmWorkspace {
  # Required
  workspace = ./.;                    # Path to monorepo root
  apps = [                           # Apps to build
    {
      name = "my-app";               # Derivation name
      path = "apps/my-app";          # Relative path from workspace root
      # Optional per-app overrides:
      version = "1.0.0";             # Default: "0.0.0"
      script = "build";              # pnpm script to run (default: "build")
      distDir = "dist";              # Output directory (default: "dist")
      buildEnv = {};                 # Extra env vars for this app
      extraNativeBuildInputs = [];   # Extra build inputs for this app
    }
  ];
  nodejs = pkgs.nodejs_22;           # Node.js to use

  # Optional
  pnpm = null;                       # Defaults to the pnpm version pinned by
                                     # `packageManager` in workspace/package.json.
                                     # Pass an explicit derivation to override.
  packages = [];                     # Shared workspace package paths
  pnpmLockYaml = workspace + "/pnpm-lock.yaml";
  buildEnv = {};                     # Env vars for all builds
  extraNativeBuildInputs = [];       # Extra inputs for all builds
  extraNodeModuleSources = [];       # Files to inject (e.g. .npmrc)
  noDevDependencies = false;         # Exclude devDependencies
  prepareWorkspacePackages = true;   # Run shared packages' `prepare` scripts,
                                     # as `pnpm install` does (see below)
  packageSrc = null;                 # path -> src for those prepare builds
                                     # (default: workspace minus app dirs)
}

Returns:

{
  apps = {
    my-app = <derivation>;   # Built app output
  };
  nodeModules = {
    "." = <derivation>;           # Root node_modules
    "apps/my-app" = <derivation>; # Per-importer node_modules
  };
  prepared = {
    "packages/my-lib" = <derivation>; # Package dir after `prepare`
  };
  pnpmStore = <derivation>;  # Full compose farm over every snapshot
  pnpm = <derivation>;       # The pnpm used to build this workspace
                             # (handy for `nix run` utility scripts)
  passthru = {
    parsed; fetched; extracted; farmLib; importerNodeModules; farm;
  };
}

Lower-level API

Each pipeline stage is exposed for advanced use or debugging:

pnpm2nix.lockfile    # path → workspaceYamlPath-or-null → parsed attrset (IFD)
                     # workspaceYamlPath is needed for pnpm 11+ patchedDependencies
                     # paths; pass null when unused.
pnpm2nix.fetch       # parsed → { "pkg@ver" = <tarball>; ... }
pnpm2nix.extract     # parsed → fetched → workspaceSrc → { "pkg@ver" = <extracted>; ... }
                     # workspaceSrc is the monorepo root, needed to resolve
                     # patchedDependencies patch files; pass it even when unused.
pnpm2nix.farm        # parsed → extracted → { cells, compose, composeFor, ... }
pnpm2nix.nodeModules # parsed → farmLib → { mkImporterNodeModules, ... }

Pipeline

pnpm-lock.yaml
      |
      v
 [lockfile]     IFD: Python/PyYAML parses YAML to JSON to Nix attrset
      |
      v
  [fetch]       One fetchurl per unique package (content-addressed)
      |
      v
 [extract]      Untar + patchShebangs per package
      |
      v
  [cells]       One derivation per snapshot (dep cycles share a cell);
      |         package files + absolute symlinks to dep cells.
      v
 [compose]      Pure-symlink .pnpm/ views: one per importer, pruned to
      |         its dependency closure. Rebuilds in ~1s.
      v
[nodeModules]   Per-importer symlink layers (tiny, cheap to rebuild)
      |
      v
 [workspace]    Per-app stdenv.mkDerivation (pnpm run build)

Platform Filtering

Packages with os/cpu fields in the lockfile (esbuild, rollup, sharp, workerd, turbo, etc.) are automatically filtered by the build platform. On linux-x64, darwin/windows/android/aix variants are excluded entirely — not fetched, not extracted, not staged.

Supported platform mappings:

Nix system npm os npm cpu
x86_64-linux linux x64
aarch64-linux linux arm64
x86_64-darwin darwin x64
aarch64-darwin darwin arm64

Unknown systems fall back to including everything (no filtering).

Requirements

  • pnpm-lock.yaml v9 (pnpm 9 through 12). Older lockfile versions are not supported. pnpm 11+ lockfiles that lead with an env-lockfile document (the packageManagerDependencies pin) are handled; that document is skipped.
  • Nix with flakes enabled.
  • The lockfile parser uses IFD (Import From Derivation), which requires Python 3 + PyYAML at eval time. These come from nixpkgs and are cached after first eval.

Limitations

  • Dependency lifecycle scripts (postinstall, etc.) are not run; only workspace packages' own prepare is. Packages like esbuild and sharp that ship prebuilt binaries work fine; packages that compile native code in postinstall will not.
  • Git/tarball dependencies without integrity hashes are unsupported — there's no hash to fetch them content-addressed. pnpm2nix fails at parse time with a clear error naming the offending dependency, rather than silently dropping it (which would surface as a runtime module-not-found). Only registry tarballs with integrity in the lockfile are supported.
  • Reverse-dependency cascade. Bumping a package rebuilds the cells of everything that transitively depends on it. Each cell is one small package copy and they build in parallel, but very popular leaves (tslib, debug) touch a few hundred cells.
  • Bumping nodejs rebuilds the farm. patchShebangs rewrites each package's #!/usr/bin/env node to the concrete Node store path at the extract layer, so every extracted package with a shebang bin references nodejs — a Node bump invalidates those extracts and cascades through every cell, compose, and app. Bumping only the lockfile does not.
  • Hoist fallback needs the sandbox. Undeclared-dependency resolution (pnpm's .pnpm/node_modules hoist) is provided via a /build/.p2n-hoist indirection that assumes the standard Nix sandbox build dir.

How it Works

See ARCHITECTURE.md for a deep dive into the cell/compose design, the SCC handling for dependency cycles, the hoist fallback, and the full pipeline internals.

License

MIT

About

Pure-Nix builder for pnpm v9 monorepos — no pnpm install at build time

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages