Skip to content

About

A triage workflow that watches issues for Snakemake repositories, clusters them by topic, and ranks the resulting clusters by relevance.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

14 Commits

Folders and files

Repository files navigation

Snaketriage

Snakemake is a powerful and versatile workflow manager for all kinds of data processing tasks. However, over the past years a considerable backlog of open but potentially outdated, duplicated, or highly connected issues have accumulated.

The purpose of this project is to implement a triage workflow that watches the public Snakemake issues, groups issues by topic, and ranks the resulting clusters by relevance.

The goal is also to flag issues/meta-issues that are potentially outdated and superseded by the current code-base, and suggest those for closing or quick fixes.

Current scope

The tool can be run locally or as a Github action. Briefly, it consists of the following steps:

  1. Fetch open issues from the Snakemake GitHub repository.
  2. Embed issue text using a sentence-transformer model.
  3. Cluster issues by similarity into meta-issues, identify overarching topics
  4. Post-processing: Rank meta-issues by relevance (code impact, criticality, age)
  5. Render results as a static web page, served on GitHub.io

Project layout

  • src/snaketriage/cli.py — Main entrypoint for snaketriage
  • src/snaketriage/github_client.py — Fetches open issues from GitHub
  • src/snaketriage/embedding.py — Creates text embeddings for downloaded issues
  • src/snaketriage/clustering.py — Clusters issue embeddings and generates visualizations
  • data/raw/ — raw issue dumps
  • data/models/ — downloaded sentence-transformer models
  • data/embeddings/ — serialized embeddings and metadata
  • data/clusters/ — generated output, including cluster figures and tables

Setup

This project is managed by pixi for dependency management and tasks. The tool can be run and tested using pixi tasks, it can be installed via PyPi, or in the most usual case, run via GitHub Actions which will render and deploy a static website (planned, not yet implemented).

GitHub

# fetch latest version of the repository from GitHub
git clone https://github.com/MPUSP/snaketriage.git
cd snaketriage

# create and activate virtual environment for pixi
pixi install
pixi shell

# run dedicated tasks
pixi run test-small-repo

PyPi

  • TODO: Installation instructions for PyPi to be added once it is released

Unit tests

Test fetching and embedding issues using pytest:

pixi run pytest-github-client
pixi run pytest-embed
pixi run pytest-cluster

Formatting and linting

Code formatting and linting is performed using Ruff:

pixi run ruff check .
pixi run ruff format .

Environment

For authenticated reads against the GitHub API (higher rate limit), export your GitHub token as an environment variable:

export GITHUB_TOKEN=<your_token_here>

Authors

License

This project is licensed under the MIT License. See the LICENSE file for details.

About

A triage workflow that watches issues for Snakemake repositories, clusters them by topic, and ranks the resulting clusters by relevance.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages