Use the Github action on a self-hosted runner

[HealthyPear]

Discussed in showyourwork/showyourwork#472

^{Originally posted by HealthyPear September 9, 2024}
As my workflows take quite a while and I am forced to use private repositories, I need to run them on a self-hosted machine.

Unfortunately the machine I can use is not self-administered, and the pip installation is externally managed, leading to this error when showyourwork gets installed,

  WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
  Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
  To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
  error: externally-managed-environment
  
  × This environment is externally managed
  ╰─> To install Python packages system-wide, try
      zypper install python311-xyz, where xyz is the package
      you are trying to install.
      
      If you wish to install a non-rpm packaged Python package,
      create a virtual environment using python3.11 -m venv path/to/venv.
      Then use path/to/venv/bin/python and path/to/venv/bin/pip.
      
      If you wish to install a non-rpm packaged Python application,
      it may be easiest to use `pipx install xyz`, which will manage a
      virtual environment for you. Install pipx via `zypper install python311-pipx` .
  
  note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.

Did anyone ever manage to use showyourwork's GitHub action in pair with an action which provides its own pip installation?

I tried with both setup-python and setup-micromamba, but showyourwork always tries to use the default pip installation.

Maybe this is a limitation of the action? Technically this action should download and install its own miniconda installation,

showyourwork-action/src/conda.js

Line 55 in 202dc71

exec(`pip install -U ${SHOWYOUWORK_SPEC}`, "Install showyourwork");

so I do not understand if it's a mistake on my side.

UPDATE

OK I found what is the problem and I think it's something that needs to be changed on the GitHub action side.

The part where showyourwork is installed by the action is performed in src/conda.js,

async function setupConda() {

  if (CACHE_CONDA) {
    // Restore conda cache
    core.startGroup("Restore conda cache");
    const conda_cacheKey = await cache.restoreCache(
      conda_paths,
      conda_key,
      conda_restoreKeys
    );
    core.endGroup();
  }

  // Download and setup conda
  if (!shell.test("-d", "~/.conda")) {
    exec(
      "wget --no-verbose https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ./conda.sh", 
      "Download conda"
    );
    exec("bash ./conda.sh -b -p ~/.conda && rm -f ./conda.sh", "Install conda");
    core.startGroup("Configure conda");
    exec("conda config --add pkgs_dirs ~/conda_pkgs_dir");
    exec("conda install -y pip");
    core.endGroup();
  }

  // Install showyourwork
  exec(`pip install -U ${SHOWYOUWORK_SPEC}`, "Install showyourwork");

where const conda_paths = ["~/.conda", "~/.condarc", "~/conda_pkgs_dir"];.

The problem is that if you are running on a self-hosted runner as in this case the conda installation might be already present (i.e. the .conda folder exists and there is something inside. Though, this something might not be the conda installation but just some user-related info about the existing environments (see conda/conda#5388).
In my case it contains only an environments.txt files with the path of all existing environments, but my conda installation is somewhere else.

There is nothing in the GitHub Action that deals with an existing conda installation at some path.

Given this I think this should be a proper issue.

[HealthyPear]

I think a good solution should be to add a new option to the action where one inputs the path of the existing conda installation and an environment for the runner, like

conda-installation-path:
    description: "Path to an existing installation of conda when calling this action"
    required: false
    default: "~/"
conda-env-runner:
    description: "Name of the conda environment to use on the runner"
    required: false
    default: "showyourwork-runner" # maybe this could be suffixed with the RUNNER_NAME, GITHUB_REPOSITORY and GITHUB_REF_NAME

then, if defined, we use that to activate the base environment, create the environment (which should be cached optionally cached I guess?) and continue.

[HealthyPear]

@dfm any ideas on this? I am not an expert so I might take more time than needed to solve this

[HealthyPear]

I do not see a conda activate before calling pip install -U ${SHOWYOUWORK_SPEC}, this to me says that we always use the runner's system's python/pip installation to do everything right?

this might not be a problem for github runners, because I guess they are e.g. docker containers removed after a while, but for self-hosted machines this might cause trouble

[HealthyPear]

Actually, why do we need that setup_conda function? can't we just use e.g. https://github.com/marketplace/actions/setup-miniconda?

[HealthyPear]

Couple of thoughts, just for me to not forget:

1. not clear (to me :P) what happens (this regardless of the runner type actually) to the intermediate products from e.g. a rule in the Snakefile, namely stuff that ends up in src/data; if the runner creates a new instance at each push/job I guess everything would be lost unless we cache it somewhere (if this is true, what if we use the cache action on xxxx/src/data?)

2. how to install showyourwork on a self-hoste machine: here the problem is (at least for me) that the action starts always from the home but that is a "real" home on some machine which I would reaaaally like to not touch! I think it would be best to modify the action to install / look for conda cache from a folder inside where the runner is installed (either the default _work folder or a _home folder in the same parent directory)

3. what if we update the action to use existing actions on the marketplace to do the setup?
   I mean for example setup-miniconda for both use cases and something like setup-apptainer for self-hosted runners