Change to use Pixi for package & environment management

.gitattributes

@@ -0,0 +1,2 @@
+# SCM syntax highlighting & preventing 3-way merges
+pixi.lock merge=binary linguist-language=YAML linguist-generated=true -diff

.github/workflows/pytest-with-coverage.yaml

@@ -12,14 +12,14 @@ jobs:
     permissions:
       contents: read
       pull-requests: write
+
     strategy:
       fail-fast: false
       matrix:
-        python-version: [ '3.12', '3.13', '3.14' ]
-    uses: UBC-MOAD/gha-workflows/.github/workflows/pytest-with-coverage.yaml@main
+        environment: [ 'test-py312', 'test-py313', 'test-py314' ]
+
+    uses: UBC-MOAD/gha-workflows/.github/workflows/pixi-pytest-with-coverage.yaml@main
     with:
-      python-version: ${{ matrix.python-version }}
-      conda-env-file: envs/environment-test.yaml
-      conda-env-name: reshapr-dev-test
+      environment: ${{ matrix.environment }}
     secrets:
       CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

.github/workflows/sphinx-linkcheck.yaml

@@ -10,14 +10,5 @@ jobs:
   sphinx-linkcheck:
     permissions:
       contents: read
-    strategy:
-      fail-fast: false
-      matrix:
-        # Need to specify Python version here because we use test env which gets its
-        # Python version via matrix
-        python-version: [ '3.14' ]
-    uses: UBC-MOAD/gha-workflows/.github/workflows/sphinx-linkcheck.yaml@main
-    with:
-      python-version: ${{ matrix.python-version }}
-      conda-env-file: envs/environment-test.yaml
-      conda-env-name: reshapr-dev-test
+
+    uses: UBC-MOAD/gha-workflows/.github/workflows/pixi-sphinx-linkcheck.yaml@main

.gitignore

@@ -118,6 +118,10 @@ ENV/
 env.bak/
 venv.bak/
 
+# pixi environments
+.pixi/*
+!.pixi/config.toml
+
 # Spyder project settings
 .spyderproject
 .spyproject

.readthedocs.yaml

@@ -3,13 +3,20 @@ version: 2
 build:
   os: ubuntu-24.04
   tools:
-    python: "mambaforge-23.11"
+    python: "latest"
 
-conda:
-  environment: envs/environment-rtd.yaml
+  jobs:
+    create_environment:
+      - asdf plugin add pixi
+      - asdf install pixi latest
+      - asdf global pixi latest
+    install:
+      - pixi install -e docs
+    build:
+      html:
+        - pixi run -e docs sphinx-build -T -b html docs $READTHEDOCS_OUTPUT/html
 
 sphinx:
-    builder: html
     configuration: docs/conf.py
     fail_on_warning: false
 

README.rst

@@ -36,6 +36,9 @@ Reshapr
 |                            | .. image:: https://img.shields.io/badge/version%20control-git-blue.svg?logo=github                                                                                                         |
 |                            |     :target: https://github.com/UBC-MOAD/Reshapr                                                                                                                                           |
 |                            |     :alt: Git on GitHub                                                                                                                                                                    |
+|                            | .. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/prefix-dev/pixi/main/assets/badge/v0.json                                                                 |
+|                            |     :target: https://pixi.prefix.dev/latest/                                                                                                                                               |
+|                            |     :alt: Pixi                                                                                                                                                                             |
 |                            | .. image:: https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white                                                                                    |
 |                            |     :target: https://pre-commit.com                                                                                                                                                        |
 |                            |     :alt: pre-commit                                                                                                                                                                       |

docs/dask_clusters.rst

@@ -75,8 +75,7 @@ a ``bash`` loop to resample day-averaged datasets to get month-averaged datasets
 
    .. code-block:: bash
 
-       $ conda activate reshapr
-       (reshapr)$ dask-scheduler
+       $ pixi run dask scheduler
 
    Use :kbd:`Control-b ,` to rename the ``tmux`` terminal to ``dask-scheduler``.
 
@@ -98,8 +97,7 @@ a ``bash`` loop to resample day-averaged datasets to get month-averaged datasets
 
    .. code-block:: bash
 
-       $ conda activate reshapr
-       (reshapr)$ dask-worker --nworkers=1 --nthreads=1 142.103.36.12:8786 &
+       $ pixi fun dask worker --nworkers=1 --nthreads=1 142.103.36.12:8786 &
 
    Use :kbd:`Control-b ,` to rename the ``tmux`` terminal to ``dask-workers``.
 

docs/examples/2xrez_physics_ONC_SCVIP.rst

@@ -52,7 +52,7 @@ server :kbd:`salish` with the command:
 
 .. code-block:: bash
 
-    (reshapr)$ reshapr extract Reshapr/docs/examples/extract_ONC_SCVIP_physics_hour_avg.yaml
+    $ pixi run reshapr extract Reshapr/docs/examples/extract_ONC_SCVIP_physics_hour_avg.yaml
 
 where :file:`Reshapr/docs/examples/extract_ONC_SCVIP_physics_hour_avg.yaml`
 is an extraction processing configuration YAML file containing:

docs/examples/atlantis_diatoms_nudge.rst

@@ -78,7 +78,7 @@ server :kbd:`salish` with the command:
 
 .. code-block:: bash
 
-    (reshapr)$ reshapr extract Reshapr/docs/examples/extract_atlantis_diatoms.yaml
+    $ pixi run reshapr extract Reshapr/docs/examples/extract_atlantis_diatoms.yaml
 
 where :file:`Reshapr/docs/examples/extract_atlantis_diatoms.yaml`
 is an extraction processing configuration YAML file containing:

docs/examples/extract_SHEM_heterotrophic_bacteria.yaml

@@ -0,0 +1,21 @@
+# Reshapr configuration to extract heterotrophic bacteria from
+# Jose's SHEM tuning/pred_flag research run
+
+dataset:
+  model profile: /home/dlatorne/MEOPAR/analysis-doug/notebooks/SHEM/model_profiles/Jose-SHEM-tuning-pred_flag.yaml
+  time base: day
+  variables group: biology
+
+dask cluster: nibi_cluster.yaml
+
+start date: 2018-02-26
+end date: 2018-07-02
+
+extract variables:
+  - heterotrophic_bacteria
+
+extracted dataset:
+  name: SHEM_day_tuning_pred_flag_heterotrophic_bacteria
+  description: Daily heterotrophic bacteria extracted from SHEM tuning/pred_flag run;
+               all depths
+  dest dir: /scratch/dlatorne/test-reshapr/

docs/examples/extract_nibi.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+
+#SBATCH --job-name=test-reshapr
+#SBATCH --ntasks=16
+# force all cores to be on the same node
+#SBATCH --ntasks-per-node=16
+#SBATCH --mem-per-cpu=8000M
+#SBATCH --time=1:30:00
+#SBATCH --mail-user=dlatorne@eoas.ubc.ca
+#SBATCH --mail-type=ALL
+#SBATCH --account=def-allen
+# stdout and stderr file paths/names
+#SBATCH --output=/scratch/dlatorne/test-reshapr/stdout
+#SBATCH --error=/scratch/dlatorne/test-reshapr/stderr
+
+WORK_DIR=$HOME/MOAD/reshapr-expts/
+
+pixi run -m $HOME/MOAD/Reshapr reshapr -v debug extract $WORK_DIR/extract_nibi.yaml --start-date 2007-01-01 --end-date 2007-12-31

docs/examples/nibi_research_run_extraction.rst

@@ -122,8 +122,7 @@ request an interactive session with
   a value for ``--time`` of tens of minutes to a few hours should be sufficient for most extractions.
 
 Once your interactive session starts,
-activate your ``reshapr`` conda environment and run your extraction with the
-:command:`reshapr extract` command.
+run your extraction with the :command:`pixi run -m $HOME/MOAD/Reshapr reshapr extract` command.
 An example of doing that looks like:
 
 .. code-block:: text
@@ -136,8 +135,7 @@ An example of doing that looks like:
     salloc: Granted job allocation 7615270
     salloc: Waiting for resource configuration
     salloc: Nodes c205 are ready for job
-    analysis-doug$ conda activate reshapr
-    (/home/dlatorne/conda_envs/reshapr) analysis-doug$ reshapr extract extract_SHEM_heterotrophic_bacteria.yaml
+    analysis-doug$ pixi run -m $HOME/MOAD/Reshapr reshapr extract extract_SHEM_heterotrophic_bacteria.yaml
     2026-01-26 14:43:35 [info     ] loaded config                  config_file=extract_SHEM_heterotrophic_bacteria.yaml
     2026-01-26 14:43:35 [info     ] loaded model profile           model_profile_yaml=/home/dlatorne/MEOPAR/analysis-doug/notebooks/SHEM/model_profiles/Jose-SHEM-tuning-pred_flag.yaml
     2026-01-26 14:43:40 [info     ] dask cluster dashboard         dashboard_link=http://127.0.0.1:8787/status dask_config_yaml=nibi_cluster.yaml
@@ -146,7 +144,7 @@ An example of doing that looks like:
     2026-01-26 14:44:48 [info     ] total time                     t_total=67.281958341598511
 
 Be sure to use the path
-(relative or absolute) to your extraction YAML file in the :command:`reshapr extract` command.
+(relative or absolute) to your extraction YAML file in the :command:`pixi run -m $HOME/MOAD/Reshapr reshapr extract` command.
 
 
 .. _SHEM-DayAvgModelProfile:
@@ -207,7 +205,7 @@ For example:
 
 .. code-block:: text
 
-    reshapr info ~/MEOPAR/analysis-doug/notebooks/SHEM/model_profiles/Jose-SHEM-tuning-pred_flag.yaml
+    pixi run -m $HOME/MOAD/Reshapr reshapr info ~/MEOPAR/analysis-doug/notebooks/SHEM/model_profiles/Jose-SHEM-tuning-pred_flag.yaml
     /home/dlatorne/MEOPAR/analysis-doug/notebooks/SHEM/model_profiles/Jose-SHEM-tuning-pred_flag.yaml:
       Jose's SalishSeaCast v202111 NEMO SHEM config results stored on
       nibi. 26feb18-02jul18 tuning pred_flag run.
@@ -231,7 +229,7 @@ with:
 
 .. code-block:: text
 
-    reshapr info ~/MEOPAR/analysis-doug/notebooks/SHEM/model_profiles/Jose-SHEM-tuning-pred_flag.yaml day biology
+    pixi run -m $HOME/MOAD/Reshapr reshapr info ~/MEOPAR/analysis-doug/notebooks/SHEM/model_profiles/Jose-SHEM-tuning-pred_flag.yaml day biology
     /home/dlatorne/MEOPAR/analysis-doug/notebooks/SHEM/model_profiles/Jose-SHEM-tuning-pred_flag.yaml:
       Jose's SalishSeaCast v202111 NEMO SHEM config results stored on
       nibi. 26feb18-02jul18 tuning pred_flag run.

docs/examples/prodigy_model_obs_assignment.rst

@@ -53,10 +53,10 @@ server :kbd:`salish` with the commands:
 
 .. code-block:: bash
 
-    (reshapr)$ reshapr extract Reshapr/docs/examples/extract_evans_nitrate_day_avg.yaml
-    (reshapr)$ reshapr extract Reshapr/docs/examples/extract_evans_physics_day_avg.yaml
-    (reshapr)$ reshapr extract Reshapr/docs/examples/extract_evans_nitrate_hour_avg.yaml
-    (reshapr)$ reshapr extract Reshapr/docs/examples/extract_evans_physics_hour_avg.yaml
+    $ pixi run reshapr extract Reshapr/docs/examples/extract_evans_nitrate_day_avg.yaml
+    $ pixi run reshapr extract Reshapr/docs/examples/extract_evans_physics_day_avg.yaml
+    $ pixi run reshapr extract Reshapr/docs/examples/extract_evans_nitrate_hour_avg.yaml
+    $ pixi run reshapr extract Reshapr/docs/examples/extract_evans_physics_hour_avg.yaml
 
 where the 4 extraction processing configuration YAML files contain:
 

docs/index.rst

@@ -21,6 +21,15 @@ Reshapr Documentation
 
 This is the documentation for the :py:obj:`Reshapr` Python package.
 
+:py:obj:`Reshapr` is a Python package and command-line tool designed to efficiently extract time series
+and perform analysis on large-scale ocean and climate model datasets.
+Built on top of :py:obj:`Xarray` and :py:obj:`Dask`,
+it simplifies the process of working with multi-gigabyte NetCDF-4 files that often exceed available system memory.
+:py:obj:`Reshapr` codifies best practices for data chunking and parallel computing,
+providing an optimized workflow for handling results from models like SalishSeaCast,
+HRDPS,
+and CANESM2 across various computing platforms—from local workstations to HPC clusters.
+
 .. toctree::
    :caption: Contents:
    :maxdepth: 2

docs/installation.rst

@@ -19,6 +19,14 @@
 Installation
 ************
 
+:py:obj:`Reshapr` uses Pixi_ for dependency and environment management.
+If you don't already have Pixi_ installed,
+please follow its `installation instructions`_ to do so.
+
+.. _Pixi: https://pixi.prefix.dev/latest/
+.. _`installation instructions`: https://pixi.prefix.dev/latest/installation/
+
+
 Get the Code
 ============
 
@@ -44,64 +52,52 @@ from the :guilabel:`Code` button on the `repository`_ page.
     .. _connecting to GitHub using SSH: https://docs.github.com/en/authentication/connecting-to-github-with-ssh
 
 
-Conda Environment and Package Installation
-==========================================
+Package Installation
+====================
 
-Running :py:obj:`Reshapr` in an isolated `Conda`_ environment using is highly recommended.
-Assuming that you have `Miniconda3`_ installed,
-you can create and activate an environment called :kbd:`reshapr` that will have
-all of the Python packages necessary use :py:obj:`Reshapr`,
-and install the package with the commands below.
+Use Pixi to create an isolated environment for :py:obj:`Reshapr` to avoid conflicts with
+other Python packages installed on your system.
+That environment will have all of the Python packages necessary to use the :program:`reshapr`
+command that is provided by the :py:obj:`Reshapr` package.
 
-.. _Conda: https://docs.conda.io/en/latest/
-.. _Miniconda3: https://docs.conda.io/en/latest/miniconda.html
-
-.. code-block:: bash
+.. code-block:: console
 
     $ cd Reshapr
-    $ conda env create -f envs/environment-user.yaml
-    $ conda activate reshapr
+    $ pixi install
 
-:py:obj:`Reshapr` is installed in `editable install mode`_ as part of the conda environment
-creation process.
-That means that the package is installed from the cloned repo via symlinks so that
-it will be automatically updated as the repo evolves.
-Please see :ref:`ReshaprUpdateInstallation` for more details.
+When you are in the :file:`Reshapr/` directory
+(or a sub-directory)
+you can run the :program:`reshapr` command with with the :command:`pixi run` command.
+Example:
 
-.. _editable install mode: https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs
+.. code-block:: console
 
-You can confirm that you have successfully installed :py:obj:`Reshapr`
-by typing :kbd:`reshapr` in your activated environment.
-You should see the top level command help information and the list of available
-:ref:`ReshaprSubcommands`.
+    $ pixi run reshapr help
 
 You can show the version of :py:obj:`Reshapr` that you have installed
 and other important information with:
 
 .. code-block:: bash
 
-    (reshapr)$ reshapr info
+    $ pixi run reshapr info
 
-To deactivate the environment use:
-
-.. code-block:: bash
-
-    (reshapr)$ conda deactivate
-
-.. note::
-    If you want to make code changes in the :py:obj:`Reshapr` package,
-    please see :ref:`ReshaprPackagedDevelopment` for information on how to work in
-    the package development environment.
+A common use-case is to execute the :command:`reshapr extract` command in the directory containing
+your extraction configuration YAML file.
+To accomplish that,
+you have to tell Pixi where to find the :file:`Reshapr/` directory so that it can use the
+correct environment.
+You do that by using the ``-m`` or ``--manifest`` option of :command:`pixi run`.
+Example:
 
-If you want to use :py:obj:`Reshapr` in a different `Conda`_ environment,
-edit your environment description to include the packages listed in the
-:kbd:`dependencies:` section of :file:`envs/environment-user.yaml`,
-then update your environment and install :py:obj:`Reshapr` with:
+.. code-block:: console
 
-.. code-block:: bash
+    $ cd $HOME/MEOPAR/analysis-doug/notebooks/SHEM/
+    $ pixi run -m $HOME/MOAD/Reshapr reshapr extract extract_SHEM_heterotrophic_bacteria.yaml
 
-    (your-env)$ conda env update -f your-env-yaml
-    (your-env)$ python3 -m pip install --editable path/to/Reshapr
+For doing development,
+testing,
+and documentation of the :py:obj:`Reshapr` package,
+please see the :ref:`ReshaprDevelopmentEnvironment` section.
 
 
 .. _ReshaprUpdateInstallation:
@@ -116,30 +112,15 @@ from GitHub:
 .. code-block:: bash
 
     $ cd Reshapr
-    $ conda activate reshapr
-    (reshapr)$ git pull
-
-You may also need to do:
-
-.. code-block:: bash
-
-    (reshapr)$ python3 -m pip install --editable .
-
-to complete the update if there have been new :ref:`ReshaprSubcommands` or options
-added since your last update.
+    $ git pull
+    $ pixi install
 
 
 Uninstalling
 ============
 
 If you want to uninstall :py:obj:`Reshapr`,
-you can remove the `Conda`_ environment with:
-
-.. code-block:: bash
-
-    $ conda env remove -n reshapr
-
-You can remove your clone of the `repository`_ with:
+you can remove your clone of the `repository`_ and the Pixi environment with:
 
 .. code-block:: bash