Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: update developer instructions #2811

Open
wants to merge 71 commits into
base: latest
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
77e4f49
first try at revised index
ctb Aug 6, 2022
9f9f046
Merge branch 'latest' into update_doc_structure
ctb Aug 15, 2022
33eeb06
lots of details
ctb Aug 16, 2022
68fe6d7
much docs, so wow
ctb Aug 16, 2022
0bf170d
even more docs
ctb Aug 16, 2022
f3ccf11
even more docs
ctb Aug 16, 2022
4c60cd7
mo'
ctb Aug 16, 2022
471ddf4
add issue links
ctb Aug 16, 2022
419e433
fix more links
ctb Aug 16, 2022
8e9248f
notes
ctb Aug 17, 2022
91fbf7c
update text
ctb Aug 17, 2022
b67d38f
Apply suggestions from code review
ctb Aug 18, 2022
fbbcee5
address many of @ccbaumler suggestions
ctb Aug 18, 2022
87ab0dc
Update doc/sourmash-internals.md
ctb Aug 18, 2022
ee17cf3
misc changes
ctb Aug 18, 2022
bb4a89d
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Aug 19, 2022
6d4a397
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Aug 20, 2022
8fce224
whups, add faq.md
ctb Aug 20, 2022
9e1fbfc
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Aug 21, 2022
df8d0e2
Merge branch 'latest' into update_doc_structure
ctb Aug 22, 2022
a967534
Merge branch 'latest' into update_doc_structure
ctb Aug 26, 2022
a1ec24b
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Aug 30, 2022
2b6e3df
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Aug 31, 2022
e468205
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Sep 23, 2023
2f1eb4b
update some text
ctb Sep 23, 2023
6576d07
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Sep 26, 2023
87ae6a2
more FAQ
ctb Sep 26, 2023
1d5b754
update / add verify notes
ctb Sep 26, 2023
7a254ee
more writing
ctb Sep 26, 2023
651ca6e
more
ctb Sep 26, 2023
01ed183
more
ctb Sep 27, 2023
8b2fac3
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Sep 27, 2023
d99d7c9
fix links etc
ctb Sep 27, 2023
c60da96
add faq
ctb Sep 27, 2023
0120b78
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Sep 28, 2023
47aa759
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Sep 29, 2023
b21282e
add funding acks
ctb Sep 29, 2023
de2b5a4
detailed gather output
ctb Sep 30, 2023
b789b73
minor updates
ctb Sep 30, 2023
fb45b6c
add information about save formats and memory usage
ctb Sep 30, 2023
69dfbc0
add section on choosing a reference for read mapping
ctb Sep 30, 2023
e828ff9
add section on retrieving reads to FAQ
ctb Sep 30, 2023
4d2a91b
collection load order
ctb Sep 30, 2023
4e97e8e
update author/copyright
ctb Sep 30, 2023
571c865
minor corrections
ctb Sep 30, 2023
c6251d3
close to done with a first pass
ctb Oct 1, 2023
1d1e460
finish off internals
ctb Oct 1, 2023
4ed0367
finish things off?
ctb Oct 1, 2023
a4c1244
clean up missing refs
ctb Oct 3, 2023
3967b15
add publications
ctb Oct 3, 2023
30ed122
Merge branch 'latest' into update_doc_structure
ctb Oct 3, 2023
362a48f
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Oct 13, 2023
1f44043
Merge branch 'update_doc_structure' of https://github.com/sourmash-bi…
ctb Oct 13, 2023
535d7eb
update new -> index
ctb Oct 13, 2023
6026ecb
Apply suggestions from code review
ctb Oct 13, 2023
247d4e4
Merge branch 'update_doc_structure' into update_doc_structure_index
ctb Oct 14, 2023
82aa9ed
update
ctb Oct 14, 2023
14e9a51
add global toc
ctb Oct 14, 2023
c30aea6
remove ToC at top
ctb Oct 14, 2023
a08a36f
try unhiding
ctb Oct 14, 2023
d128c65
add sidebar
ctb Oct 14, 2023
13fde86
add TOC to sidebar
ctb Oct 14, 2023
8144c3c
upd sidebar
ctb Oct 14, 2023
6a5bb1a
rename internals page
ctb Oct 14, 2023
817092f
clean up
ctb Oct 14, 2023
2aea751
add developer quickstart
ctb Oct 15, 2023
b944b69
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Oct 15, 2023
8441727
Merge branch 'update_doc_structure' into update_doc_add_dev_and_install
ctb Oct 15, 2023
32ecc0d
clean up dev instructions a bit
ctb Oct 15, 2023
16bf8b5
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Oct 15, 2023
499e172
Merge branch 'latest' of https://github.com/sourmash-bio/sourmash int…
ctb Oct 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions doc/developer-quickstart.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Developer quickstart with conda

The instructions below are for people interested in developing sourmash.
They should get you up and running with a sourmash development environment
in about 15 minutes.

## Install conda (or mamba)

(You don't need to do this if you already have conda and mamba installed!)

Follow the
[installation instructions](https://github.com/conda-forge/miniforge#install)
for installing `mambaforge` (a conda distribution that uses
[`mamba`](https://github.com/TheSnakePit/mamba) and the
[`conda-forge`](https://conda-forge.org/) channel by default).

## Create a conda environment with basic requirements:

First, install all of the necessary base packages:
```
mamba create -y -n sourmash-dev python=3.10 pip \
cxx-compiler make rust
```

## Clone sourmash repo

Get a copy of the sourmash repository from GitHub:
```
git clone https://github.com/dib-lab/sourmash.git
cd sourmash
```

## Activate conda environment and install the development version of sourmash

Activate your new conda environment, created above:
```
conda activate sourmash-dev
```
and then install sourmash from within the git working directory:
```
python -m pip install -e ".[all]"
```

The `-e` option to `pip install` installs sourmash in "developer"
mode, such that the installed package is simply a link to the current
directory. Directing pip to install `".[all]"` results in pip
installing the package in the current directory (the `.` argument)
with all of the documentation and test packages (`[all]`) as specified
in `pyproject.toml`.

## Run tests

You can run all the Python tests like so:
```
python -m pytest
```

## Develop!

You are now ready to develop on sourmash! Any code you edit or update will
be run by the tests and/or by executing `sourmash` at the command line.

If you change any sourmash Rust code, you will need run `make` before
running sourmash or the tests. This recompiles the Rust extension
code.

You don't need to run `make` if you're just changing Python code.

For example,
```
make
python -m pytest
```

Once you've run `make`, you can run `sourmash` in that conda
environment and it will run the version of sourmash in the git working
directory, and any changes to the Rust code will be included.

## Additional information

### Running subsets of tests

You can run specific subsets of tests using `pytest -k`. For example, this:
```
python -m pytest -k sbt
```
will run any test that has `sbt` in the name.

### Building documentation

You can build the docs like so:
```
cd doc
make
```
and then the built docs will be in `_build/html/`.

### Changing/creating new branches

As long as you're in the directory that you installed, you can change
branches and/or create new branches, and the `sourmash` command will
run the code on the branch.

So, for example,
```
git switch branch_from_github
```
will change to the `branch_from_github`, and then `git commit` and
`git push` will make commits and send those commits back to github.
39 changes: 16 additions & 23 deletions doc/developer.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@

# Developer information

## Developer quickstart with conda

You can quickly get started with a development environment using
conda; see the
[developer quickstart with conda](developer-quickstart.md)!

Read on for more details about sourmash development!

## Development environment

You can get the latest development branch with:
Expand All @@ -12,31 +20,15 @@ git clone https://github.com/sourmash-bio/sourmash.git
```
sourmash runs under Python 3.8 and later.

We recommend using `conda` or `Nix` for setting up an environment for developing
new features, running tests and code quality checks.
Here are some suggestions on how to set them up (note: you only need one =])

### Using mamba (conda alternative)

Follow the [installation instructions](https://github.com/conda-forge/miniforge#install) for
installing `mambaforge` (a conda distribution that uses
[`mamba`](https://github.com/TheSnakePit/mamba)
and the [`conda-forge`](https://conda-forge.org/) channel by default).
We recommend using `conda` or `Nix` for setting up an environment for
developing new features, running tests and code quality checks. Here
are some suggestions on how to set them up (note: you only need one
=])

Once `mamba` is installed, run
```
mamba create -n sourmash_dev 'tox>=3.27,<4' tox-conda rust git compilers pandoc
```
to create an environment called `sourmash_dev` containing the programs needed
for development.

To activate the new environment, run
```
conda activate sourmash_dev
```
and proceed to the ["Running tests and checks"](#running-tests-and-checks) section.
See the [developer quickstart with conda](developer-quickstart.md) for
conda instructions!

### Using Nix
### Conda/mamba alternative: Using Nix

Follow the [installation instructions](https://nixos.org/manual/nix/stable/#chap-installation)
for setting up Nix in your system (Linux or macOS).
Expand Down Expand Up @@ -296,6 +288,7 @@ make clean
```{toctree}
:maxdepth: 2

developer-quickstart
release
requirements
storage
Expand Down
3 changes: 3 additions & 0 deletions doc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,8 @@ X and Linux. They require about 5 GB of disk space and 5 GB of RAM.

## Developing and extending sourmash

* [Developer quickstart with conda](developer-quickstart.md)
* [Developer information](developer.md)
* [Releasing a new version of sourmash](release.md)

<!--
Expand All @@ -130,4 +132,5 @@ sidebar
command-line
api-example
databases
developer-quickstart
```
1 change: 1 addition & 0 deletions doc/toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ databases-advanced.md
databases.md
dev_plugins.md
developer.md
developer-quickstart.md
faq.md
funding.md
index.md
Expand Down
Loading