Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added chapter on pixi #68

Merged
merged 5 commits into from
Feb 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions _quarto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ book:
- part: "Github"
chapters:
- github/gh-collaboration.qmd
- part: "pixi"
amrei-bp marked this conversation as resolved.
Show resolved Hide resolved
chapters:
- pixi/pixi.qmd
- summary.qmd
- references.qmd

Expand Down
160 changes: 160 additions & 0 deletions pixi/pixi.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
## Intro to Pixi

This section is a brief summary of the concepts discussed in the meeting.

## Why Pixi?

Pixi is a package management tool that serves as a replacement for Conda or Mamba. It is designed to be faster, multithreaded, and flexible. Like Conda, Pixi environments are not isolated, which allows you to interact with third-party tools that are available in your system's PATH.

### Compatibility

Pixi is somehwat compatible with Conda. You can:

- Use the same Conda channels (e.g., `conda-forge`, `bioconda`).
- Use existing Conda environment files with Pixi.

However, you cannot activate a conda environment using Pixi, or vice-versa.

### Key Difference

In Pixi, the environment configuration file (`.pixi.toml`) lives in the project directory. This ensures that the environment is tied to the project and simplifies reproducibility.

---

## Getting Started

### Installation

1. Install Pixi and add it to your shell's configuration (e.g., `.bashrc` or `.zshrc`).

```{.bash}
curl -fsSL https://pixi.sh/install.sh | bash
```

2. Initialize a Pixi environment in your project:

```{.bash}
pixi init -c conda-forge -c bioconda -p linux-64 -p osx-arm64
```

Here:
- `-c` specifies the channels to use (e.g., `conda-forge`, `bioconda`)
- `-p` sets the platforms

By default, Pixi will set the environment for your platform (e.g., `linux-64`), but you can specify additional platforms as needed.

The `pixi init`command will create two files:

- `.pixi.toml`: Specifies the environment configuration.
- `.pixi.lock`: Locks down the specific package versions for reproducibility.

You should commit both files to your version control system to share the exact environment setup with collaborators and increase `reproducibility`.


### Adding Packages

You can add packages to your environment using the `pixi add` command:

```{.bash}
pixi add bwa samtools
```

You can add Python packages from PyPI using Pixi:

```{.bash}
pixi add python
pixi add --pypi multiqc
```

Alternatively, you can directly modify the `.pixi.toml` file in your project directory:

Under `[dependencies]`, or platform specific dependencies (such as `[target.linux-64.dependencies]` you can add a line for each package you want to include `nextflow = "24.10.4.*"`for example.

---

## Managing Environments

### Files in the Directory

When you use Pixi, it creates a `.pixi` folder in your project directory. This folder contains the libraries and binaries needed for the environment. If needed, you can safely delete this folder.


---

## Tasks in Pixi

Pixi allows you to define and run tasks directly in the `.pixi.toml` file. For example:

```toml
[tasks]
name-of-task = "nextflow run main.nf -profile PDC"
```

One can also add tasks via the command line:

```{.bash}
pixi task add hello python hello_world.py
```

### Task Features

Tasks can be combined or run conditionally. The example in the [documentation](https://pixi.sh/latest/features/advanced_tasks/) is to specify that an application should be complied before being run. The command `depends-on`can be used here. With this one can chain tasks for complex workflows.

---

## Working with the Shell

Pixi provides a shell environment based on the Deno shell. Many basic Bash commands still work, allowing for:

- Chaining tools.
- Command substitution.

### Activating the Environment

To activate the environment:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should make clear that there are two ways of interacting with Pixi. The first is interactively through pixi shell which is the equivalent of conda activate environment-name, but also through running tasks pixi run task-name.
pixi run command-line-string also works so you can run a one-off command inside the pixi environment and then return to your current shell.


```{.bash}
pixi shell
```

---

## Advanced Features

### Features

PIXI allows multiple "features" in a single project. Features are isolated from each other, helping to avoid version clashes between tools.
amrei-bp marked this conversation as resolved.
Show resolved Hide resolved

### Intel emulation on ARM Macs

Although packages are increasingly being built for ARM architecture CPUS, not all tools are built for `osx-arm64`. However, they may have been built for `osx-64` (i.e., intel architecture CPUS), in addition to `linux-64`. MacOS includes the tool Rosetta, which can be used to emulate intel on arm Macs, at the cost of performance.

By supporting only `linux-64` and `osx-64` as platforms, Pixi will automatically run the tools using Rosetta emulation on Mac ARM computers.

```bash
pixi init \
--channel "conda-forge" \
--channel "bioconda" \
--platform "linux-64" \
--platform "osx-64"
```

---

## Additional Commands

### Updating Pixi

To update Pixi:

```{.bash}
pixi self-update
```

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include pixi clean cache and pixi search here too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does pixi search work?

### Cleaning the cache

To clean the cache:

```{.bash}
pixi clean cache
```