-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add docs for design decisions and update lock file related docs
- Loading branch information
1 parent
3a1775f
commit 20d7194
Showing
10 changed files
with
203 additions
and
33 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# Introduction | ||
|
||
**EZconda** takes the approach of keeping two files for managing `conda` environments. | ||
|
||
1. **[Specifications file](specfile.md)** contains the packages and channels (and *only those* packages/channels) that the [user requests via the command line](../user_guide/install_packages.md). | ||
|
||
2. **[Lock file](lockfile.md)** contains _**all**_ the packages along with the *exact version and build number and channels*. It also contains other metadata such as the system information where it was generated, etc. | ||
|
||
|
||
> The two files keep environments and corresponding specifications in sync as well as [support reproducible environment creation](../user_guide/recreate_env.md). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
# Lock File | ||
|
||
A lock file contains _**packages**_ that the user requested _**as well as all of it dependencies**_. | ||
|
||
It contains the exact specifications of all the packages and their dependencies, version, build number, and channels. | ||
|
||
Lock files are _**specific to the platform and architecture**_ they were generated on. | ||
|
||
!!! Note | ||
Information about the system's platform and architecture is also captured in the lock file and is used to validate the lock file when using it to recreate environments. | ||
|
||
## Format | ||
|
||
Lock file uses the following format to store the information described above - | ||
|
||
```TOML | ||
# minimum version of EZconda required | ||
[version] | ||
ezconda-min-version = "0.4.0" | ||
|
||
# system specifications | ||
[system] | ||
platform = "darwin" | ||
architecture = "64bit" | ||
machine = "x86_64" | ||
|
||
# name of the environment | ||
[environment] | ||
name = "mlproj" | ||
|
||
# all the packages | ||
[[packages]] | ||
base_url = "https://conda.anaconda.org/conda-forge" | ||
build_number = "0" | ||
build_string = "py38h5fc983b_0" | ||
channel = "conda-forge" | ||
dist_name = "pandas-1.0.5-py38h5fc983b_0" | ||
name = "pandas" | ||
platform = "osx-64" | ||
version = "1.0.5" | ||
|
||
... | ||
``` | ||
|
||
!!! Warning | ||
Lock files **are not meant to be edited by the user**. They are managed completely by EZconda. | ||
|
||
## Naming Convention | ||
|
||
Lock files are named after the environment and the platform and its architecture. | ||
|
||
``` | ||
<env>-<platform>-<architecture>.lock | ||
``` | ||
|
||
For instance, if the environment name is "mlproj" and is created on a Mac, the name of the lock file will be - | ||
|
||
``` | ||
mlproj-darwin-x86_64.lock | ||
``` | ||
|
||
!!! Note | ||
This automatically ensures that same environment created on a different platform will have a separate lock file. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
# Specifications File | ||
|
||
This file is similar to the `environment.yml` file in `conda`. | ||
|
||
However, the big difference is, it only tracks the dependencies that the [user sepcifies at the command line.](#user-specified-packages) and not its dependencies. | ||
|
||
??? Info | ||
An exhaustive description of the pacakges and its dependencies is in the [lock file](lockfile.md). | ||
|
||
## Format | ||
|
||
The specifications file format is similar to `conda` environment specifications files. This is primarily done to maintain compatibility with `conda`. | ||
|
||
In other words, you can use this specifications file generated by `ezconda` with `conda`. | ||
|
||
It specifies the packages (and not their dependencies) and channels of interest - | ||
|
||
```YAML title="mlproj.yml" | ||
# name of the environment | ||
name: mlproj | ||
|
||
# channels to search | ||
channels: | ||
- defaults | ||
- conda-forge | ||
|
||
# packages to install | ||
packages: | ||
- python=3.8 | ||
- numpy>=1.17 | ||
- pandas | ||
``` | ||
!!! Note | ||
You _**do not**_ need to add or remove packages manually from the environment.yml file. | ||
This is done **automatically** when you install/remove packages using ezconda. See [user guide](../user_guide/install_packages.md) for more details. | ||
## User Specified Packages | ||
If a user [installs](../user_guide/install_packages.md) `pandas` via the command line, the specifications file will only list `pandas` and not all the specific dependencies of `pandas` for that specific architecture. | ||
|
||
> This ensures that specifications file is platform independent whereas the [lock file](lockfile.md) is platform specific. | ||
|
||
<div class="termy"> | ||
|
||
```console | ||
$ ezconda install -n mlproj pandas | ||
// Adds (only) pandas to mlproj.yml specifications file | ||
``` | ||
|
||
</div> | ||
|
||
??? Example "mlproj.yml" | ||
```YAML | ||
# name of the environment | ||
name: mlproj | ||
# channels to search | ||
channels: | ||
- defaults | ||
# packages to install | ||
packages: | ||
- pandas | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters