Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

notes on developer installation, as of aug 2023 #2707

Open
ctb opened this issue Aug 9, 2023 · 9 comments · May be fixed by #2811
Open

notes on developer installation, as of aug 2023 #2707

ctb opened this issue Aug 9, 2023 · 9 comments · May be fixed by #2811
Labels
doc documentation content or issues

Comments

@ctb
Copy link
Contributor

ctb commented Aug 9, 2023

per https://sourmash.readthedocs.io/en/latest/developer.html,

  • it's not clear that the "general instructions" come after/are an ALTERNATIVE to the mamba install stuff
  • the mamba install stuff doesn't actually install tox, although tox-conda is installed? weird. Anyway you need to do a python -m pip install tox
  • we should probably explicitly suggest mambaforge... because we like mambaforge...

also: personally, I'm not a huge fan of the make test running tox, because it takes a lot of extra time vs just running pytest.

more maybe later.

@ctb
Copy link
Contributor Author

ctb commented Aug 12, 2023

further notes: add pip install -e . suggestion.

@luizirber
Copy link
Member

luizirber commented Aug 19, 2023

Additional issues: tox-conda is not compatible with tox 4, hit it in #2723 . I ended up adding an explicit 'tox>=3.27,<4' to the conda dev env instructions to get the right version.

(and I think python -m pip install tox will install tox 4? Or does it pick tox 3, which tox-conda restricts?)

@ctb
Copy link
Contributor Author

ctb commented Aug 21, 2023

this is an obvious thought, in retrospect, but: why don't we create a developer.yml conda environment file that people can use?

(this can be put into the CI as well)

@luizirber
Copy link
Member

this is an obvious thought, in retrospect, but: why don't we create a developer.yml conda environment file that people can use?

(this can be put into the CI as well)

Or pixi, I started a prototype branch in https://github.com/sourmash-bio/sourmash/compare/lirber/pixi and it works really well for setting up the dev env!

@ctb
Copy link
Contributor Author

ctb commented Aug 26, 2023

should also add instructions for plugins -

for working on a plugin, per @mr-eyes on slack -

  • first git clone, checkout a new branch, then pip install -e .
  • then edit the code and run locally until you get the expected results.
  • Finally push and create a PR.

@ctb
Copy link
Contributor Author

ctb commented Oct 14, 2023

notes here: https://hackmd.io/4DwRHfSvQbioykT-vrfyfA?view

install sourmash for development purposes using conda

Create a conda environment with basic requirements:

mamba create -y -n sourmash-dev python=3.10 pip \
    cxx-compiler make rust

Clone sourmash repo

Get a copy of the sourmash repository:

git clone https://github.com/dib-lab/sourmash.git
cd sourmash

Activate conda repo and install

First activate the conda environment:

conda activate sourmash-dev

and then install sourmash:

python -m pip install -e ".[all]"

The -e option to pip install installs sourmash in "developer" mode, such that the installed package is simply a link to the current directory. Directing pip to install ".[all]" results in pip installing the package in the current directory (the . argument) with all of the documentation and test packages ([all]) as specified in pyproject.toml.

Run tests

You can now run all the Python tests like so:

python -m pytest 

Develop!

At this point you can change code, and then run the sourmash command yourself, and/or run tests.

If you change any sourmash Rust code, it is safest to run make first to (re)compile the Rust extension code, and then run the tests. (You don't need to run make if you're just changing Python code.)

make
python -m pytest

Once you've run make, you can run sourmash in that conda environment and it will run the version of sourmash in the git working directory.

Additional information

Running subsets of tests

You can run specific subsets of tests using pytest -k. For example, this:

python -m pytest -k sbt

will run any test that has sbt in the name.

Building documentation

You can build the docs like so:

cd doc
make

and then the built docs will be in _build/html/.

Changing/creating new branches

As long as you're in the directory that you installed, you can change branches and/or create new branches, and the sourmash command will run the code on the branch.

So, for example,

git switch branch_from_github

will change to the branch_from_github, and then git commit and git push will make commits and send those commits back to github.

@AnneliektH
Copy link

These notes worked for me, with the addition of following this to test out a specific branch.

@ctb ctb linked a pull request Oct 15, 2023 that will close this issue
@luizirber
Copy link
Member

I like how the PyPA packaging guide has tabs for different options when listing commands, which might solve the confusion with our current docs (linear implies "you need to run them all").

Code: https://github.com/pypa/packaging.python.org/blob/83223a00484ddda6e516028ce97710adcc683f80/source/tutorials/installing-packages.rst
(it is RST blocks, but we can use them in markdown thru MyST)

@bluegenes
Copy link
Contributor

bluegenes commented Feb 8, 2024

bumping this -- what else is required for #2811?

our current mamba-based developer install doesn't work for macs

mamba create -n sourmash_dev 'tox>=3.27,<4' tox-conda rust git compilers pandoc libstdcxx-ng
Could not solve for environment specs
The following package could not be installed
└─ libstdcxx-ng   does not exist (perhaps a typo or a missing channel).

luizirber added a commit that referenced this issue Jul 28, 2024
Close #2793 

- Add pixi, first mentioned in
#2707 (comment)
- Use tabs in docs to help to point out only one option is needed
- Default recommendation is conda, add `dev.yml` with dev dependencies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc documentation content or issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants