Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add .github/workflows/ci.yml for implementing CI/CD #88

Open
cgpu opened this issue Aug 24, 2021 · 2 comments
Open

Add .github/workflows/ci.yml for implementing CI/CD #88

cgpu opened this issue Aug 24, 2021 · 2 comments

Comments

@cgpu
Copy link
Collaborator

cgpu commented Aug 24, 2021

NOTE: GitHub actions requires additional configuration for running on GPU instances.
It is recommended to implement first the cpu only mode.

There are also limitations in the resources that can be used when testing with Github Actions:

Hardware specification for Linux virtual machines (used by default)
https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources

Max cpus: 2-core CPU
Max memory: 7 GB of RAM memory
Max disk size: 14 GB of SSD disk space

To be able to add CI/CD successfully therefore, a minimal test dataset is required.

Here is the template file that implements both docker and singularity CI.
It needs to be created in a folder in the root of the repo named .github/workflows/ci.yml (the folder names are reserved, the file name can be changed eg for ci.yml to continues-integration.yml etc)

name: splicing-pipelines-nf CI
# This workflow is triggered on pushes and PRs to the repository.
on: [pull_request]

jobs:
  docker:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        nxf_ver: ['20.01.0', '']
    steps:
      - uses: actions/checkout@v1
      - name: Install Nextflow
        run: |
          export NXF_VER=${{ matrix.nxf_ver }}
          wget -qO- get.nextflow.io | bash
          sudo mv nextflow /usr/local/bin/
      - name: Basic workflow tests
        run: |
          nextflow run ${GITHUB_WORKSPACE} --config conf/test.config
  singularity:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        singularity_version: ['3.6.4']
        nxf_ver: ['20.01.0', '']
    steps:
      - uses: actions/checkout@v1
      - uses: eWaterCycle/setup-singularity@v6
        with:
          singularity-version: ${{ matrix.singularity_version }}
      - name: Install Nextflow
        run: |
          export NXF_VER=${{ matrix.nxf_ver }}
          wget -qO- get.nextflow.io | bash
          sudo mv nextflow /usr/local/bin/
      - name: Basic workflow tests
        run: |
          nextflow run ${GITHUB_WORKSPACE}  --config conf/test.config

Many configs can be tested using the matrix stretegy.

@liuyang2006
Copy link
Collaborator

Docker test has been passed, we will add singularity test after we push singularity image into a public portal.

@liuyang2006
Copy link
Collaborator

liuyang2006 commented Sep 28, 2021

Achieved by #108.
Results can be found here: https://github.com/TheJacksonLaboratory/nanome/actions/runs/1283587877.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants