Skip to content

linux-quickstart.md

Latest commit

 

History

History
276 lines (216 loc) · 10.1 KB

linux-quickstart.md

File metadata and controls

276 lines (216 loc) · 10.1 KB
Raw

Linux QuickStart

Installing or building the headless mere.run CLI on Linux.

Be clear about what you are getting. The Linux path is real but deliberately narrow: CLI only, Ubuntu-style hosts, and CUDA on x86_64 or arm64. There is no CPU-only Linux release and no studio app here; macOS remains the primary development and runtime-validation environment for this repo.

Nothing is treated as supported until it has been built and smoke-tested on the host class it targets — CUDA artifacts on real CUDA hardware, never a cross-build taken on faith.

Current validation boundary

  • Linux package artifacts are headless CLI-only.
  • Package validation must cover the portable tarball, Debian package, runtime library bundling, and package manifests on the target Linux host class.
  • CUDA package validation must include a real CUDA GPU host.
  • Linux packages do not include MereRun.app, the SwiftUI studio, the macOS installer UI, or the DMG layout.
  • Published Linux package builds must use MERERUN_LINUX_ACCEL=cuda on both x86_64 and arm64. CPU builds are test fixtures, not release artifacts.
  • NVIDIA GB10/DGX Spark is a valid arm64 CUDA target when the package is built and smoke-tested on matching hardware.
  • Current CUDA validation should be treated as limited to the exact hosts that have run the CUDA package and smoke path.

Install with apt

Use this path on Debian or Ubuntu-style systems after building a matching .deb, or when a matching .deb is attached to a release:

version=0.23.0
case "$(uname -m)" in
  x86_64|amd64) package="./dist/linux/mere-run-cuda_${version}_amd64.deb" ;;
  aarch64|arm64) package="./dist/linux/mere-run_${version}_arm64.deb" ;;
  *) echo "unsupported Linux CUDA architecture" >&2; exit 1 ;;
esac

sudo apt install "$package"
mere.run --version
mere.run status

The Debian package installs the mere.run command and the colocated runtime assets that the CLI needs at runtime.

Install the portable tarball

Use this path after building a matching tarball, or when a matching tarball is attached to a release:

version=0.23.0
case "$(uname -m)" in
  x86_64|amd64) archive="mere-run-${version}-linux-x86_64-cuda.tar.gz" ;;
  aarch64|arm64) archive="mere-run-${version}-linux-arm64.tar.gz" ;;
  *) echo "unsupported Linux CUDA architecture" >&2; exit 1 ;;
esac

tar -xzf "./dist/linux/$archive"
cd "${archive%.tar.gz}"
./install.sh
mere.run --version
mere.run status

The tarball installer copies the CLI and its runtime assets together. The packaged launcher resolves its own install location and sets LD_LIBRARY_PATH for the bundled runtime libraries before it starts the real mere.run binary. On CUDA 13 hosts it also resolves CUDA CCCL from CUDA home, versioned Toolkit roots, and target-specific include directories, then adds that cuda/std root to CPATH and MERERUN_CUDA_CCCL_INCLUDE_PATH. MLX CUDA kernels use that explicit path while JIT-compiling in the installed package.

Install the x86_64 CUDA tarball

Use this artifact for GPU worker images, remote trainers, and other x86_64 CUDA hosts after building it, or when a matching CUDA tarball is attached to a release:

version=0.23.0
tar -xzf "./dist/linux/mere-run-${version}-linux-x86_64-cuda.tar.gz"
cd "mere-run-${version}-linux-x86_64-cuda"
./install.sh
mere.run --version
mere.run status

The CUDA tarball is the right build pack for companion remote-runner plugins. It is not a source/bootstrap archive and should not compile on paid GPU time. No CPU-only tarball is published.

First commands

After installing:

mere.run --help
mere.run guide --list
mere.run model list
mere.run model capabilities
mere.run model capabilities --recommended
mere.run status

Use mere.run model capabilities before pulling large checkpoints. It gives a machine-local view of the recommended public model IDs for the current host.

Media commands

Linux media paths expect ffmpeg and ffprobe:

sudo apt-get update
sudo apt-get install -y ffmpeg

If the binaries are not on PATH, point the CLI at explicit locations:

export MERERUN_FFMPEG=/usr/bin/ffmpeg
export MERERUN_FFPROBE=/usr/bin/ffprobe

Linux x86_64 CUDA package path

The package script can build x86_64 CUDA artifacts with CUDA development packages and MERERUN_SKIP_MLX_CUDA_EXAMPLE=1. To build locally on a Linux x86_64 CUDA development host:

MERERUN_LINUX_ACCEL=cuda MERERUN_SKIP_MLX_CUDA_EXAMPLE=1 \
  scripts/package-linux.sh --version 0.23.0 --artifact-suffix cuda
ls dist/linux/

On rented GPU builders, keep paid compile time down by setting MERERUN_NATIVE_BUILD_JOBS to the worker's real vCPU count and MERERUN_CUDA_ARCHITECTURES to the target build policy. For example, an RTX A4000 builder that should also ship forward-compatible H100 PTX can use:

MERERUN_LINUX_ACCEL=cuda \
MERERUN_SKIP_MLX_CUDA_EXAMPLE=1 \
MERERUN_NATIVE_BUILD_JOBS=14 \
MERERUN_CUDA_ARCHITECTURES="86-real;90-virtual" \
  scripts/package-linux.sh --version 0.23.0 --artifact-suffix cuda

Run a real GPU smoke on the target runtime class before widening support claims.

Linux arm64 CUDA package path

Linux arm64 is only useful for this project when the CUDA lane works. Do not publish or recommend CPU-only arm64 packages as release artifacts.

Build arm64 packages on a native arm64 Linux host with an NVIDIA GPU, driver, CUDA Toolkit, nvcc, CUDA CCCL headers, cuDNN, NCCL, Swift, OpenBLAS/LAPACK headers, and ffmpeg available. On NVIDIA's Ubuntu 24.04 SBSA repo this includes cuda-cccl-13-0, libcudnn9-dev-cuda-13, and libnccl-dev:

MERERUN_LINUX_ACCEL=cuda scripts/package-linux.sh --version 0.23.0
ls dist/linux/

CUDA .deb packages are gated to binaries linked against CUDA 12 or CUDA 13. The packager reads the libcudart.so SONAME from the built executable and refuses to emit a .deb when the toolkit major is unknown or unsupported, rather than attaching incorrect runtime dependencies. CUDA 12 packages target NVIDIA's 12.8 package series and accept the corresponding Lambda Stack library packages as Debian alternatives. CUDA 13 packages declare: cuda-cccl-13-0, cuda-cudart-dev-13-0, cuda-cudart-13-0, cuda-nvrtc-13-0, libcublas-13-0, libcufft-13-0, libcudnn9-cuda-13, and libnccl2. CUDA 12 packages likewise require cuda-cudart-dev-12-8. MLX compiles specialized kernels with NVRTC during inference, so CUDA headers such as cuda_bf16.h are runtime requirements even though mere.run itself is already compiled. Use an artifact suffix such as cuda12 or cuda13 when publishing multiple CUDA lanes. Maintainers who supply an exact MERERUN_PACKAGE_LINUX_DEPS value deliberately bypass this gate; that override is copied verbatim into Depends, so the caller owns package compatibility.

For source-only CUDA checks, prepare native artifacts and then export the environment printed by the script before building:

MERERUN_LINUX_ACCEL=cuda scripts/prepare-linux-native.sh
swift build --product mere.run

The preparation script first lets SwiftPM resolve dependencies under the active Linux Swift toolchain, then builds the CMake CUDA bridge from that exact mlx-swift checkout. This keeps the bridge aligned when an older Swift toolchain selects a compatibility revision different from a Mac checkout. MLX_SWIFT_CUDA_COMMIT is available only as a deliberate maintainer diagnostic override; normal source builds should use the SwiftPM-selected revision.

Do not describe a CUDA configuration as supported unless it has been run on that matching host.

Spark CUDA smoke status

The arm64 CUDA smoke path has been validated on NVIDIA GB10/DGX Spark after installing the package and pulling public managed models. MLX-backed image, speech, vision, music, SFX, video, embedding, anonymization, and dense Gemma chat paths are expected to run there with the packaged CUDA setup. The packaged Linux CUDA build also carries the matching llama-cli; mere.run text code uses that subprocess on Linux so GGUF coding models do not share a process with MLX CUDA. If a future CUDA run fails inside an upstream MLX or llama.cpp kernel, keep that failure in the release notes or PR description until the exact package has been rebuilt and rerun on matching hardware.

Quantized MLX paths default to MERERUN_MLX_CUDA_NATIVE_QUANT=auto: each process probes native quantized_mm and GatherQMM separately, reports the selected backend on stderr, and falls back to dense compatibility if the linked runtime rejects a kernel. Run scripts/e2e_gb10.sh --quant-mode auto after each CUDA package rebuild; native is the explicit fail-loud validation mode.

Build from source on Linux

Install the package layer first:

sudo apt-get update
sudo apt-get install -y cmake ninja-build pkg-config gfortran libcurl4-openssl-dev zlib1g-dev libopenblas-dev liblapacke-dev ffmpeg gzip unzip zip

CUDA package builds also require CMake 3.25 or newer because the CUDA bridge build follows upstream mlx-swift. On Ubuntu 22.04/Jammy images, install a newer CMake before running the CUDA package path:

sudo apt-get install -y python3-pip
sudo python3 -m pip install --upgrade "cmake>=3.25,<4"
cmake --version

On Linux arm64, older distro Clang packages can shadow Swift's bundled Clang and fail MLX bf16 header compilation. The Linux scripts probe for this and select a bf16-capable C++ driver, including a local clang++ shim backed by Swift's clang-17 when needed; if not, set CXX to the Swift toolchain clang++ path before building.

Then run the headless Linux checks:

swift build
swift test
./scripts/check-linux.sh
swift run mere.run --help

Do not run the macOS app bundle commands on Linux. mere.run.app, the SwiftUI studio, and DMG packaging are macOS-only.

Verify release assets

When a Linux release includes SHA256SUMS, place it beside the matching tarball or .deb before checking it:

tag=v0.23.0

curl -L "https://github.com/sawfwair/mere-run/releases/download/${tag}/SHA256SUMS" -o SHA256SUMS
sha256sum -c SHA256SUMS

If you only downloaded one asset, sha256sum -c will report the missing file as not found. That is expected; download the matching tarball or .deb before running the full manifest check.