From 927e7b605264ba03287b1afc5ae6c93bce4e199c Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Tue, 26 Jul 2022 18:46:10 +0000 Subject: [PATCH 01/12] update features reference --- proposals/devcontainer-features.md | 103 +++++++++++++++-------------- 1 file changed, 53 insertions(+), 50 deletions(-) diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index 93b5c80d..39430699 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -1,31 +1,9 @@ # Dev container features reference -Dev container features provide a smooth path for customizing your container definitions. Features are self contained units of code meant to facilitate creation of custom images for application or development containers. - -From a practical point of view, features are folders that contain units of code with different entrypoints for different lifecycle events. - -Features can be defined by a `devcontainer-feature.json` file in the root folder of the feature. The file is optional for backwards compatibility but it is required for any new features being authored. - -By default, features are installed in an order selected by the implementing tool. - -If any of the following properties are provided in the feature's `devcontainer-feature.json`, or the user's `devcontainer.json`, the order indicated by the propert(ies) are respected. - -- `installsAfter` property defined as part of `devcontainer-feature.json`. -- `id`. -- `overrideFeatureInstallOrder` in `devcontainer.json`. Allows users to control the order of execution of their features. - -The tool uses the `runsAfter` property to intelligently manage this order and ensure that if there are relationships between the features, they are respected. - -An end-user can explicitly provide an installation order for features given the `overrideFeatureInstallOrder` property of `devcontainer.json`. - -All feature `id` provided in `overrideFeatureInstallOrder` must also exist in the `features` property of a user's `devcontainer.json`. - -The provided features, indicated by `id`, will be installed in the specified order. Any remaining features in the features object that are not mentioned in the array will be installed in an undefined/implicit order, as determined as optimal by the tooling. - -| Property | Type | Description | -| :--- | :--- | :--- | -| overrideFeatureInstallOrder | array | Array made of the Id's of the features in the order the user wants them to be installed. | +Dev container 'features' are self-contained units of installation code and development container configuration. Features are designed to install atop a wide-range of base container images. +Features are generally designed to be installed on top of a subset of base container images (eg: debian-based images). +Feature metadata is captured by a `devcontainer-feature.json` file in the root folder of the feature. ## Folder Structure @@ -65,6 +43,7 @@ The properties of the file are as follows: | Property | Type | Description | | :--- | :--- | :--- | | id | string | Id of the feature/definition. The id should be unique in the context of the repository/published package where the feature exists. | +| version | string | The semantic version of the feature. | | name | string | Name of the feature/definition. | | description | string | Description of the feature/definition. | | documentationURL | string | Url that points to the documentation of the feature. | @@ -91,35 +70,34 @@ Options | id.default | string | Default value for the option. | | id.description | string | Description for the option. | -## devcontainer-collection.json properties - -A feature collection file is a compilation of the `devcontainer-feature.json` files for each individual feature. It inlines all the information in each `devcontainer-feature.json` for each feature. +## devcontainer.json properties -If the application finds a `devcontainer-collection.json` file in the root of a downloaded tar file, then it uses that file to find the particular feature that will be executed. -In addition to the list of features included, the file includes the following properties. +Features are referenced in `devcontainer.json` under the top level `features` object. +The properties are: | Property | Type | Description | | :--- | :--- | :--- | -| name | string | Name of the collection. | -| reference | string | Reference information of the repository and path where the code is stored. | -| version | string | Version of the code. | +| id | string | Reference to the particular feature to be included. | +| options | object | Type of the option .| -In most cases, the `devcontainer-collection.json` file can be generated automatically at the moment of creating a release. +The `id` is the main reference point for how to find and download a particular feature. `id` can be defined in any of the following ways: -## devcontainer.json properties +| Type | Description | Example | +| :--- | :--- | :--- | +| \/\/\[:\] | Reference to feature in OCI registry(*) | ghcr.io/user/repo/go:1 | +| https://<..URI..>/myFeatures.tgz#{feature} | Direct HTTPS URI to a tarball. | https:github.com/user/repo/releases/myFeatures.tgz#go| +| ./{local-path} -or- | A relative to directory with a devcontainer-feature.json. | ./myGoFeature | -Features are referenced in `devcontainer.json` , where the `features` tag consists of an object tag starting with the id of the feature and including the values of the options to pass to the feature itself. +` +(*) OCI registry must implement the [OCI Artifact Distribution Specification](https://github.com/opencontainers/distribution-spec). Some implementors can be [found here](https://oras.land/implementors/). -The `id` is the main reference point for how to find and download a particular feature. `id` can be defined in any of the following ways: +## Versioning -| Type | Description | -| :--- | :--- | -| feature | A simple name referencing a feature; it's included with the application used to execute `devcontainer.json`.| -| organization/repository/{feature or collectionl/feature}/@version | Reference to a particular release in a repository. | -| https://<../URI/..>/devcontainer-features.tgz#{optional feature} | Direct reference to a file download. | -| ./{local-path} -or- ../{local-path} | A path relative to devcontainer.json where a feature or feature collection can be found. | +Each feature is individually [versioned according to the semver specification](https://semver.org/). The `version` property in the respective `devcontainer-feature.json` file is updated to increment the feature's version. + +Tooling that handles releasing features will not republish features if that exact version has already been published; however, tooling must republish major and minor versions in accordance with the semver specification. ## Authoring @@ -133,19 +111,44 @@ If the feature is included in a folder as part of the repository that contains ` ## Release -A release is created when the objective is to have other users use a feature. +_For information on distribution features, see [devcontainer-features-distribution.md](./devcontainer-features-distribution.md)._ + + +## Execution + +### Installation Order -A release consists of the following: +By default, features are installed on top of a base image in an order determined as optimal by the implementing tool. -1.- Tar file with all the included files for the feature or feature collection. -2.- A copy of the `devcontainer-feature.json` or `devcontainer-collection.json` file that defines the contents of the tar file, with additional information added to validate it. +If any of the following properties are provided in the feature's `devcontainer-feature.json`, or the user's `devcontainer.json`, the order indicated by these propert(ies) are respected (with decreasing precedence). + +1. The `overrideFeatureInstallOrder` property in user's `devcontainer.json`. Allows users to control the order of execution of their features. +2. The `installsAfter` property defined as part of a feature's `devcontainer-feature.json`. + +#### (1) overrideFeatureInstallOrder + +This property is declared by the user in their `devcontainer.json` file. + +Any feature IDs listed in this array will be installed before all other features, in the provided order. Any omitted features will be installed in an order selected by the implementing tool, or ordered via the `installsAfter` property _after_ any features listed in the `overrideFeatureInstallOrder` array, if applicable. + +All feature `id` provided in `overrideFeatureInstallOrder` must also exist in the `features` property of a user's `devcontainer.json`. | Property | Type | Description | | :--- | :--- | :--- | -| version | string | SemVer version of the release. | -| checksum | string | Checksum of the tar file. | +| overrideFeatureInstallOrder | array | Array consisting of the feature `id` of features in the order the user wants them to be installed. | -## Execution +#### (2) installsAfter + +This propertry is defined in an individual feature's `devcontainer-feature.json` file by the feature author. `installsAfter` allows an author to provide the tooling hints on loose dependencies between features. + +After `overrideFeatureInstallOrder` is resolved, any remaining features that declare an `installsAfter` must be installed after the features declared in the property, provided that the features have also been declared in the `features` property. + +| Property | Type | Description | +| :--- | :--- | :--- | +| installsAfter | array | Array consisting of the feature `id` that should be installed before the given feature | + + +### Implementation Notes There are several things to keep in mind for an application that implements features: From f63be32fdb957de92a231d571ba83f5a6e427826 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Tue, 26 Jul 2022 19:24:53 +0000 Subject: [PATCH 02/12] distribution --- .../devcontainer-features-distribution.md | 185 ++++++++++++++++++ proposals/devcontainer-features.md | 3 + 2 files changed, 188 insertions(+) create mode 100644 proposals/devcontainer-features-distribution.md diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md new file mode 100644 index 00000000..2bd6e77d --- /dev/null +++ b/proposals/devcontainer-features-distribution.md @@ -0,0 +1,185 @@ +# Community Contribution and Discovery of Dev Container Features + +This specification defines a pattern where community members and organizations can author and self-publish [dev container 'features'](./devcontainer-features.md). + +Goals include: + +- For community authors, a "self-served" mechanism for dev container feature publishing. +- For community members, a largely self-served mechanism for marking features as "discoverable" by [supporting tools](../docs/specs/supporting-tools.md). +- For users, ease of discoverability of features via [supporting tools](../docs/specs/supporting-tools.md). +- For users, the abilty to validate the origin of the asset and its integrity when compared to previous pulls. +- For users, the ability for a user to pin to a particular version (absolute, or semantic version) of a feature to allow consistent, repeatable environments. + +## Source Code + +Features source code is stored in a git repository. + +For ease of authorship and maintenance, [1..n] features can share a single git repository. This set of features is referred to as feature collections, and will share the same [`devcontainer-collection.json`](#devcontainer-collection.json) file and namespace. + +Source code for the set follows the example file structure below: + +``` +. +├── README.md +├── src +│ ├── dotnet +│ │ ├── devcontainer-feature.json +│ │ ├── install.sh +│ │ └── ... +| ├ +│ ├── go +│ │ ├── devcontainer-feature.json +│ │ └── install.sh +| ├── ... +│ │ ├── devcontainer-feature.json +│ │ └── install.sh +├── test +│ ├── dotnet +│ │ ├── devcontainer-feature.json +│ │ └── ... +│ └── go +│ | └── test.sh +| ├── ... +│ │ └── test.sh +├── ... +``` + +Where `src` is a directory containing a sub-folder with the name of the feature (e.g. `dotnet` or `go`) with at least a file named `devcontainer-feature.json` that contains the feature metadata. Each feature sub-directory also contains an `install.sh` script that implementing tools will use to install the feature. Each sub-directory should be named such that it matches the `id` field of the `devcontainer-feature.json`. Other files can also be included in the sub-directory, and will be packaged along side the two required files. + +Optionally, a mirrored `test` directory can be included with an accompanying `test.sh` script. Implementing tools may use this to run tests against the given feature. + +## Versioning + +Each feature is individually [versioned according to the semver specification](https://semver.org/). The `version` property in the respective `devcontainer-feature.json` file is parsed to determine if the feature should be republished. + +Tooling that handles publishing features will not republish features if that exact version has already been published; however, tooling must republish major and minor versions in accordance with the semver specification. + +## Packaging + +Features are distributed as tarballs. The tarball contains the entire contents of the feature sub-directory, including the `devcontainer-feature.json`, `install.sh`, and any other files in the directory. + +The tarball is named `devcontainer-feature-.tgz`, where `` is the feature's `id` field. + +A reference implementation for packaging and distributing features is provided as a GitHub Action (https://github.com/devcontainers/action). + + +### devcontainer-collection.json + +The `devcontainer-collection.json` is an auto-generated metadata file. + +| Property | Type | Description | +| :--- | :--- | :--- | +| sourceInformation | object | Metadata from the implementing packaging tool. | +| features | array | The list of features that are contained in this collection.| + + + +Each features's `devcontainer-feature.json` metadata file is appended into the `features` top-level array. + +## Distribution + +There are several supported ways to distribute features. With appropriate authentication a user references a distributed feature in a `devcontainer.json` as defined in ['referencing a feature'](./devcontainer-features.md#Referencing-a-feature). + +### OCI Registry + +An OCI registry that implements the [OCI Artifact Distribution Specification](https://github.com/opencontainers/distribution-spec) serves as the primary distribution mechanism for features. + +Each packaged feature is pushed to the registry following the naming convention `//[:version]`, where version is the major, minor, and patch version of the feature, according to the semver specification. + +A custom media type `application/vnd.devcontainers` and `application/vnd.devcontainers.layer.v1+tar` are used as demonstrated below. + +For example, the `go` feature in the `devcontainers/features` namespace at version `1.2.3` would be pushed to the ghcr.io OCI registry. +```bash +# ghcr.io/devcontainers/features/go:1 +REGISTRY=ghcr.io +NAMESPACE=devcontainers/features +FEATURE=go + +ARTIFACT_PATH=devcontainer-feature-go.tgz + +for VERSION in 1 1.2 1.2.3 latest +do + oras push ${REGISTRY}/${NAMESPACE}/${FEATURE}:${VERSION} \ + --manifest-config /dev/null:application/vnd.devcontainers \ + ./${ARTIFACT_PATH}:application/vnd.devcontainers.layer.v1+tar`; +done + +``` + +`Namespace` is the globally identifiable name for the collection of features. (eg: `owner/repo` for the source code's git repository). + +The auto-generated `devcontainer-collection.json` is pushed to the registry with the same `namespace` as above and no accompanying `feature` name. The collection file is always tagged as `latest`. + +```bash +# ghcr.io/devcontainers/features +REGISTRY=ghcr.io +NAMESPACE=devcontainers/features + +oras push ${REGISTRY}/${NAMESPACE}:latest \ + --manifest-config /dev/null:application/vnd.devcontainers \ + ./devcontainer-collection.json.:application/vnd.devcontainers.layer.v1+json`; +``` + +### Directly reference tarball + +A feature can be referenced directly in a `devcontainer.json` file by an HTTP URI that points to the tarball from the [package step](#packaging). + + \ No newline at end of file diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index 39430699..cc5cf4ed 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -81,6 +81,9 @@ The properties are: | id | string | Reference to the particular feature to be included. | | options | object | Type of the option .| + +### Referencing a feature + The `id` is the main reference point for how to find and download a particular feature. `id` can be defined in any of the following ways: | Type | Description | Example | From a8b5691d00c5e44b6f192efef73e52485f2deb78 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Tue, 26 Jul 2022 19:39:42 +0000 Subject: [PATCH 03/12] various updates --- .../devcontainer-features-distribution.md | 76 +++---------------- proposals/devcontainer-features.md | 33 ++------ 2 files changed, 18 insertions(+), 91 deletions(-) diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md index 2bd6e77d..c1a60e26 100644 --- a/proposals/devcontainer-features-distribution.md +++ b/proposals/devcontainer-features-distribution.md @@ -14,7 +14,7 @@ Goals include: Features source code is stored in a git repository. -For ease of authorship and maintenance, [1..n] features can share a single git repository. This set of features is referred to as feature collections, and will share the same [`devcontainer-collection.json`](#devcontainer-collection.json) file and namespace. +For ease of authorship and maintenance, [1..n] features can share a single git repository. This set of features is referred to as a collection, and will share the same [`devcontainer-collection.json`](#devcontainer-collection.json) file and 'namespace' (eg. `owner/repo`). Source code for the set follows the example file structure below: @@ -78,7 +78,9 @@ Each features's `devcontainer-feature.json` metadata file is appended into the ` ## Distribution -There are several supported ways to distribute features. With appropriate authentication a user references a distributed feature in a `devcontainer.json` as defined in ['referencing a feature'](./devcontainer-features.md#Referencing-a-feature). +There are several supported ways to distribute features. Distribution is handled by the implementing packaging tool. + +A user references a distributed feature in a `devcontainer.json` as defined in ['referencing a feature'](./devcontainer-features.md#Referencing-a-feature). ### OCI Registry @@ -88,7 +90,10 @@ Each packaged feature is pushed to the registry following the naming convention A custom media type `application/vnd.devcontainers` and `application/vnd.devcontainers.layer.v1+tar` are used as demonstrated below. -For example, the `go` feature in the `devcontainers/features` namespace at version `1.2.3` would be pushed to the ghcr.io OCI registry. + +For example, the `go` feature in the `devcontainers/features` namespace at version `1.2.3` would be pushed to the ghcr.io OCI registry. + +_NOTE: The example below uses [`oras`](https://oras.land/) for demonstration purposes. A supporting tool should directly implement the required functionality from the aforementioned OCI artifact distribution specification._ ```bash # ghcr.io/devcontainers/features/go:1 REGISTRY=ghcr.io @@ -101,7 +106,7 @@ for VERSION in 1 1.2 1.2.3 latest do oras push ${REGISTRY}/${NAMESPACE}/${FEATURE}:${VERSION} \ --manifest-config /dev/null:application/vnd.devcontainers \ - ./${ARTIFACT_PATH}:application/vnd.devcontainers.layer.v1+tar`; + ./${ARTIFACT_PATH}:application/vnd.devcontainers.layer.v1+tar done ``` @@ -117,69 +122,10 @@ NAMESPACE=devcontainers/features oras push ${REGISTRY}/${NAMESPACE}:latest \ --manifest-config /dev/null:application/vnd.devcontainers \ - ./devcontainer-collection.json.:application/vnd.devcontainers.layer.v1+json`; + ./devcontainer-collection.json.:application/vnd.devcontainers.layer.v1+json ``` -### Directly reference tarball +### Directly Reference Tarball A feature can be referenced directly in a `devcontainer.json` file by an HTTP URI that points to the tarball from the [package step](#packaging). - \ No newline at end of file diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index cc5cf4ed..65c6dae6 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -7,33 +7,15 @@ Feature metadata is captured by a `devcontainer-feature.json` file in the root f ## Folder Structure -A feature is a self contained entity in a folder. A feature release would be a tar file that contains all files part of the feature. +A feature is a self contained entity in a folder with at least a `devcontainer-feature.json` and `install.sh` entrypoint script. Additional files are permitted and are packaged along side the required files. ``` +-- feature | +-- devcontainer-feature.json -| +-- install.sh (default) -| +-- (other files) -``` - -In case `devcontainer-feature.json` does not include a reference for the lifecycle scripts the application will look for the default script names and will execute them if available. - -In case there is intent to create a set of features that share code, it is possible to create a feature collection in the following way: - -``` -collectionFolder -+-- devcontainer-collection.json -+-- common (or similar) -| +-- (library files) -+-- feature1 -| +-- devcontainer-feature.json | +-- install.sh | +-- (other files) -+-- feature2 -(etc) ``` - ## devcontainer-feature.json properties the `devcontainer-feature.json` file defines information about the feature to be used by any supporting tools and the way the feature will be executed. @@ -70,8 +52,7 @@ Options | id.default | string | Default value for the option. | | id.description | string | Description for the option. | -## devcontainer.json properties - +## devcontainer.json properties (feature related) Features are referenced in `devcontainer.json` under the top level `features` object. @@ -84,13 +65,13 @@ The properties are: ### Referencing a feature -The `id` is the main reference point for how to find and download a particular feature. `id` can be defined in any of the following ways: +The `id` format specified dicates how a supporting tool will locate and download a given feature. `id` is one of the following: | Type | Description | Example | | :--- | :--- | :--- | -| \/\/\[:\] | Reference to feature in OCI registry(*) | ghcr.io/user/repo/go:1 | -| https://<..URI..>/myFeatures.tgz#{feature} | Direct HTTPS URI to a tarball. | https:github.com/user/repo/releases/myFeatures.tgz#go| -| ./{local-path} -or- | A relative to directory with a devcontainer-feature.json. | ./myGoFeature | +| `//[:]` | Reference to feature in OCI registry(*) | ghcr.io/user/repo/go:1 | +| `https://` | Direct HTTPS URI to a tarball. | https://github.com/user/repo/releases/devcontainer-feature-go.tgz | +| `./`| A relative directory to folder containing a devcontainer-feature.json. | ./myGoFeature | ` (*) OCI registry must implement the [OCI Artifact Distribution Specification](https://github.com/opencontainers/distribution-spec). Some implementors can be [found here](https://oras.land/implementors/). @@ -98,7 +79,7 @@ The `id` is the main reference point for how to find and download a particular f ## Versioning -Each feature is individually [versioned according to the semver specification](https://semver.org/). The `version` property in the respective `devcontainer-feature.json` file is updated to increment the feature's version. +Each feature is individually [versioned according to the semver specification](https://semver.org/). The `version` property in the respective `devcontainer-feature.json` file is updated to increment the feature's version. Tooling that handles releasing features will not republish features if that exact version has already been published; however, tooling must republish major and minor versions in accordance with the semver specification. From e1dc5b14f545d125201826366705427e0e06d9cb Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 14:40:18 +0000 Subject: [PATCH 04/12] @samruddhikhandale comments --- proposals/devcontainer-features-distribution.md | 8 ++++---- proposals/devcontainer-features.md | 1 - 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md index c1a60e26..e85c3880 100644 --- a/proposals/devcontainer-features-distribution.md +++ b/proposals/devcontainer-features-distribution.md @@ -5,7 +5,7 @@ This specification defines a pattern where community members and organizations c Goals include: - For community authors, a "self-served" mechanism for dev container feature publishing. -- For community members, a largely self-served mechanism for marking features as "discoverable" by [supporting tools](../docs/specs/supporting-tools.md). +- For community members, the ability to publish will allow [supporting tools](../docs/specs/supporting-tools.md) to implement their own discovery mechanisms. - For users, ease of discoverability of features via [supporting tools](../docs/specs/supporting-tools.md). - For users, the abilty to validate the origin of the asset and its integrity when compared to previous pulls. - For users, the ability for a user to pin to a particular version (absolute, or semantic version) of a feature to allow consistent, repeatable environments. @@ -14,7 +14,7 @@ Goals include: Features source code is stored in a git repository. -For ease of authorship and maintenance, [1..n] features can share a single git repository. This set of features is referred to as a collection, and will share the same [`devcontainer-collection.json`](#devcontainer-collection.json) file and 'namespace' (eg. `owner/repo`). +For ease of authorship and maintenance, [1..n] features can share a single git repository. This set of features is referred to as a collection, and will share the same [`devcontainer-collection.json`](#devcontainer-collection.json) file and 'namespace' (eg. `/`). Source code for the set follows the example file structure below: @@ -44,7 +44,7 @@ Source code for the set follows the example file structure below: ├── ... ``` -Where `src` is a directory containing a sub-folder with the name of the feature (e.g. `dotnet` or `go`) with at least a file named `devcontainer-feature.json` that contains the feature metadata. Each feature sub-directory also contains an `install.sh` script that implementing tools will use to install the feature. Each sub-directory should be named such that it matches the `id` field of the `devcontainer-feature.json`. Other files can also be included in the sub-directory, and will be packaged along side the two required files. +Where `src` is a directory containing a sub-folder with the name of the feature (e.g. `src/dotnet` or `src/go`) with at least a file named `devcontainer-feature.json` that contains the feature metadata, and an `install.sh` script that implementing tools will use as the entrypoint to install the feature. Each sub-directory should be named such that it matches the `id` field of the `devcontainer-feature.json`. Other files can also be included in the feature's sub-directory, and will be packaged along side the two required files. Any files that are not part of the feature's sub-directory (e.g. outside of `src/dotnet`) will not be packaged. Optionally, a mirrored `test` directory can be included with an accompanying `test.sh` script. Implementing tools may use this to run tests against the given feature. @@ -127,5 +127,5 @@ oras push ${REGISTRY}/${NAMESPACE}:latest \ ### Directly Reference Tarball -A feature can be referenced directly in a `devcontainer.json` file by an HTTP URI that points to the tarball from the [package step](#packaging). +A feature can be referenced directly in a user's [`devcontainer.json`](/docs/specs/devcontainer-reference.md#devcontainerjson) file by an HTTP URI that points to the tarball from the [package step](#packaging). diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index 65c6dae6..1c828372 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -30,7 +30,6 @@ The properties of the file are as follows: | description | string | Description of the feature/definition. | | documentationURL | string | Url that points to the documentation of the feature. | | licenseURL | string | Url that points to the license of the feature. | -| version | string | Version of the feature. | | keywords | array | List of keywords relevant to a user that would search for this definition/feature. | | options | object | Possible options to be passed as environment variables to the execution of the scripts | | containerEnv | object | A set of name value pairs that sets or overrides environment variables. | From fc92242597e1a90720b0778114d12c6ca71dc0fa Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 15:32:58 +0000 Subject: [PATCH 05/12] @alexdima comments and much more detail on options --- .../devcontainer-features-distribution.md | 4 +- proposals/devcontainer-features.md | 129 ++++++++++++++++-- 2 files changed, 122 insertions(+), 11 deletions(-) diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md index e85c3880..d9ae2df0 100644 --- a/proposals/devcontainer-features-distribution.md +++ b/proposals/devcontainer-features-distribution.md @@ -7,7 +7,7 @@ Goals include: - For community authors, a "self-served" mechanism for dev container feature publishing. - For community members, the ability to publish will allow [supporting tools](../docs/specs/supporting-tools.md) to implement their own discovery mechanisms. - For users, ease of discoverability of features via [supporting tools](../docs/specs/supporting-tools.md). -- For users, the abilty to validate the origin of the asset and its integrity when compared to previous pulls. +- For users, the ability to validate the integrity of previously fetched assets. - For users, the ability for a user to pin to a particular version (absolute, or semantic version) of a feature to allow consistent, repeatable environments. ## Source Code @@ -35,7 +35,7 @@ Source code for the set follows the example file structure below: │ │ └── install.sh ├── test │ ├── dotnet -│ │ ├── devcontainer-feature.json +│ │ ├── test.sh │ │ └── ... │ └── go │ | └── test.sh diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index 1c828372..fe19c249 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -51,16 +51,48 @@ Options | id.default | string | Default value for the option. | | id.description | string | Description for the option. | -## devcontainer.json properties (feature related) +## devcontainer.json properties -Features are referenced in `devcontainer.json` under the top level `features` object. +Features are referenced in a user's [`devcontainer.json`](/docs/specs/devcontainer-reference.md#devcontainerjson) under the top level `features` object. -The properties are: -| Property | Type | Description | -| :--- | :--- | :--- | -| id | string | Reference to the particular feature to be included. | -| options | object | Type of the option .| +A user can specify an arbitrary number of features. At build time, these features will be installed in an order defined by a combination of the [installation order rules and implementation](#Installation-Order). + +A single feature is provided as a key/value pair, where the key is the feature identifier, and the value is an object containing "options" (or empty for "default"). Each key in the feature object must be unique. + +These options are sourced as environment variables at build-time, as specified in [Option Resolution](#Option-Resolution). + + +Below is a valid `features` object provided as an example. +```jsonc +"features": { + "ghcr.io/user/repo/go:1": {}, + "https://github.com/user/repo/releases/devcontainer-feature-go.tgz": { + "optionA": "value" + }, + "./myGoFeature": { + "optionA": true, + "optionB": "hello", + "version" : "1.0.0" + } +} +``` + +An option's value can be provided as either a `string` or `boolean`, and should match what is expected by the feature in the `devcontainer-feature.json` file. + +As a shorthand, the value of a `feature` can be provided as a single string. This string is mapped to an option called `version`. In the example below, both examples are equivalent. +```jsonc +"features": { + "ghcr.io/owner/repo/go": "1.18" +} +``` +```jsonc +"features": { + "ghcr.io/owner/repo/go": { + "version": "1.18" + } +} +``` ### Referencing a feature @@ -75,7 +107,6 @@ The `id` format specified dicates how a supporting tool will locate and download ` (*) OCI registry must implement the [OCI Artifact Distribution Specification](https://github.com/opencontainers/distribution-spec). Some implementors can be [found here](https://oras.land/implementors/). - ## Versioning Each feature is individually [versioned according to the semver specification](https://semver.org/). The `version` property in the respective `devcontainer-feature.json` file is updated to increment the feature's version. @@ -96,7 +127,6 @@ If the feature is included in a folder as part of the repository that contains ` _For information on distribution features, see [devcontainer-features-distribution.md](./devcontainer-features-distribution.md)._ - ## Execution ### Installation Order @@ -130,6 +160,87 @@ After `overrideFeatureInstallOrder` is resolved, any remaining features that dec | :--- | :--- | :--- | | installsAfter | array | Array consisting of the feature `id` that should be installed before the given feature | +### Option Resolution + +A feature's 'options' - specified as the value of a single feature key/value pair in the user's `devcontainer.json` - are passed to the feature as environment variables. + +A supporting tool will parse the `options` object provided by the user. If a value is provided for a feature, it will be emitted to a file named `devcontainer-features.env` following the format `=`. + +To ensure a option that is valid as an environment variable, the follow substitutions are performed. + +```javascript +(str: string) => str + .replace(/[^\w_]/g, '_') + .replace(/^[\d_]+/g, '_') + .toUpperCase(); +``` + +This file is sourced at build-time for the feature `install.sh` entrypoint script to handle. + +Any options defined by a feature's `devcontainer-feature.json` that are omitted in the user's `devcontainer.json` will be implicitly exported as its default value. + +### Option Resolution Example + +Suppose a `python` feature has the following `options` parameters declared in the `devcontainer-feature.json` file: + +```jsonc +// ... +"options": { + "version": { + "type": "string", + "enum": ["latest", "3.10", "3.9", "3.8", "3.7", "3.6"], + "default": "latest", + "description": "Select a Python version to install." + }, + "pip": { + "type": "boolean", + "default": true, + "description": "Installs pip" + }, + "optimize": { + "type": "boolean", + "default": true, + "description": "Optimize python installation" + } +} +``` + +The user's `devcontainer.json` declared the python feature like so + +```jsonc + +"features": { + "python": { + "version": "3.10", + "pip": false + } +} +``` + +The emitted environment variables will be: + +```env +VERSION="3.10" +PIP="false" +OPTIMIZE="true" +``` + +These will be sourced and visible to the `install.sh` entrypoint script. The following `install.sh` would print +``` +3.10 +false +true +``` + +```bash +#!/usr/bin/env bash + +echo "Version is $VERSION" +echo "Pip? $PIP" +echo "Optimize? $OPTIMIZE" +``` + + ### Implementation Notes From e413478c5b10a2c6239e7d839d6de25dd93cae28 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 15:34:26 +0000 Subject: [PATCH 06/12] clearer --- proposals/devcontainer-features.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index fe19c249..f4e1ab06 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -225,12 +225,8 @@ PIP="false" OPTIMIZE="true" ``` -These will be sourced and visible to the `install.sh` entrypoint script. The following `install.sh` would print -``` -3.10 -false -true -``` +These will be sourced and visible to the `install.sh` entrypoint script. The following `install.sh`... + ```bash #!/usr/bin/env bash @@ -240,7 +236,12 @@ echo "Pip? $PIP" echo "Optimize? $OPTIMIZE" ``` - +...output the following +``` +3.10 +false +true +``` ### Implementation Notes From 9f5b5f25bd72e69e75e3316ea18b5a2673dc9967 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 15:36:26 +0000 Subject: [PATCH 07/12] goals --- proposals/devcontainer-features-distribution.md | 3 +-- proposals/devcontainer-features.md | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md index d9ae2df0..a665a9fa 100644 --- a/proposals/devcontainer-features-distribution.md +++ b/proposals/devcontainer-features-distribution.md @@ -5,8 +5,7 @@ This specification defines a pattern where community members and organizations c Goals include: - For community authors, a "self-served" mechanism for dev container feature publishing. -- For community members, the ability to publish will allow [supporting tools](../docs/specs/supporting-tools.md) to implement their own discovery mechanisms. -- For users, ease of discoverability of features via [supporting tools](../docs/specs/supporting-tools.md). +- For community members, the ability to publish features, thus allowing [supporting tools](../docs/specs/supporting-tools.md) to implement their own discovery mechanisms. - For users, the ability to validate the integrity of previously fetched assets. - For users, the ability for a user to pin to a particular version (absolute, or semantic version) of a feature to allow consistent, repeatable environments. diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index f4e1ab06..2296c82b 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -216,7 +216,6 @@ The user's `devcontainer.json` declared the python feature like so } } ``` - The emitted environment variables will be: ```env @@ -236,7 +235,7 @@ echo "Pip? $PIP" echo "Optimize? $OPTIMIZE" ``` -...output the following +...output the following: ``` 3.10 false From 6c59e9de01ef67c2e4ef413deefbbd21ff74e81e Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 17:26:00 +0000 Subject: [PATCH 08/12] @edgonmsft comments addressed' --- proposals/devcontainer-features-distribution.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md index a665a9fa..0e4b7d65 100644 --- a/proposals/devcontainer-features-distribution.md +++ b/proposals/devcontainer-features-distribution.md @@ -43,7 +43,7 @@ Source code for the set follows the example file structure below: ├── ... ``` -Where `src` is a directory containing a sub-folder with the name of the feature (e.g. `src/dotnet` or `src/go`) with at least a file named `devcontainer-feature.json` that contains the feature metadata, and an `install.sh` script that implementing tools will use as the entrypoint to install the feature. Each sub-directory should be named such that it matches the `id` field of the `devcontainer-feature.json`. Other files can also be included in the feature's sub-directory, and will be packaged along side the two required files. Any files that are not part of the feature's sub-directory (e.g. outside of `src/dotnet`) will not be packaged. +Where `src` is a directory containing a sub-folder with the name of the feature (e.g. `src/dotnet` or `src/go`) with at least a file named `devcontainer-feature.json` that contains the feature metadata, and an `install.sh` script that implementing tools will use as the entrypoint to install the feature. Each sub-directory should be named such that it matches the `id` field of the `devcontainer-feature.json`. Other files can also be included in the feature's sub-directory, and will be included during the [packaging step](#packaging) along side the two required files. Any files that are not part of the feature's sub-directory (e.g. outside of `src/dotnet`) will not included in the [packaging step](#packaging). Optionally, a mirrored `test` directory can be included with an accompanying `test.sh` script. Implementing tools may use this to run tests against the given feature. @@ -70,8 +70,6 @@ The `devcontainer-collection.json` is an auto-generated metadata file. | :--- | :--- | :--- | | sourceInformation | object | Metadata from the implementing packaging tool. | | features | array | The list of features that are contained in this collection.| - - Each features's `devcontainer-feature.json` metadata file is appended into the `features` top-level array. @@ -87,8 +85,9 @@ An OCI registry that implements the [OCI Artifact Distribution Specification](ht Each packaged feature is pushed to the registry following the naming convention `//[:version]`, where version is the major, minor, and patch version of the feature, according to the semver specification. -A custom media type `application/vnd.devcontainers` and `application/vnd.devcontainers.layer.v1+tar` are used as demonstrated below. +> The `namespace` is a unique indentifier for the collection of features. There are no strict rules for the `namespace`; however, one pattern is to set `namespace` equal to source repository's `/`. +A custom media type `application/vnd.devcontainers` and `application/vnd.devcontainers.layer.v1+tar` are used as demonstrated below. For example, the `go` feature in the `devcontainers/features` namespace at version `1.2.3` would be pushed to the ghcr.io OCI registry. @@ -128,3 +127,10 @@ oras push ${REGISTRY}/${NAMESPACE}:latest \ A feature can be referenced directly in a user's [`devcontainer.json`](/docs/specs/devcontainer-reference.md#devcontainerjson) file by an HTTP URI that points to the tarball from the [package step](#packaging). +### Addendum: Locally Referenced + +To aid in feature authorship, or in instances where a feature should not be published externally, individual features can be referenced locally from the project's file tree. + +A feature can be referenced directly in a user's [`devcontainer.json`](/docs/specs/devcontainer-reference.md#devcontainerjson) by relative path _inside_ the project directory. A local feature may not be referenced outside of the project directory (`../` is not allowed), nor is an absolute path allowed. + +The provided relative path is a path to the folder containing the feature's `devcontainer-feature.json` and `install.sh` file. From 1a6536f1f4faeaa76b89cf15de0ca2be2512d949 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 17:27:16 +0000 Subject: [PATCH 09/12] spelling --- proposals/devcontainer-features-distribution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md index 0e4b7d65..eda255ba 100644 --- a/proposals/devcontainer-features-distribution.md +++ b/proposals/devcontainer-features-distribution.md @@ -43,7 +43,7 @@ Source code for the set follows the example file structure below: ├── ... ``` -Where `src` is a directory containing a sub-folder with the name of the feature (e.g. `src/dotnet` or `src/go`) with at least a file named `devcontainer-feature.json` that contains the feature metadata, and an `install.sh` script that implementing tools will use as the entrypoint to install the feature. Each sub-directory should be named such that it matches the `id` field of the `devcontainer-feature.json`. Other files can also be included in the feature's sub-directory, and will be included during the [packaging step](#packaging) along side the two required files. Any files that are not part of the feature's sub-directory (e.g. outside of `src/dotnet`) will not included in the [packaging step](#packaging). +Where `src` is a directory containing a sub-folder with the name of the feature (e.g. `src/dotnet` or `src/go`) with at least a file named `devcontainer-feature.json` that contains the feature metadata, and an `install.sh` script that implementing tools will use as the entrypoint to install the feature. Each sub-directory should be named such that it matches the `id` field of the `devcontainer-feature.json`. Other files can also be included in the feature's sub-directory, and will be included during the [packaging step](#packaging) alongside the two required files. Any files that are not part of the feature's sub-directory (e.g. outside of `src/dotnet`) will not included in the [packaging step](#packaging). Optionally, a mirrored `test` directory can be included with an accompanying `test.sh` script. Implementing tools may use this to run tests against the given feature. From 7a83ea40a03af4320382bf741054f5ee1a794ed1 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 17:54:08 +0000 Subject: [PATCH 10/12] update goals --- proposals/devcontainer-features-distribution.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md index eda255ba..817d88b5 100644 --- a/proposals/devcontainer-features-distribution.md +++ b/proposals/devcontainer-features-distribution.md @@ -4,10 +4,10 @@ This specification defines a pattern where community members and organizations c Goals include: -- For community authors, a "self-served" mechanism for dev container feature publishing. -- For community members, the ability to publish features, thus allowing [supporting tools](../docs/specs/supporting-tools.md) to implement their own discovery mechanisms. +- For community authors, a "self-served" mechanism for dev container feature publishing, either publicly or privately. - For users, the ability to validate the integrity of previously fetched assets. -- For users, the ability for a user to pin to a particular version (absolute, or semantic version) of a feature to allow consistent, repeatable environments. +- For users, the ability for a user to pin to a particular version (absolute, or semantic version) of a feature to allow for consistent, repeatable environments. +- The ability to standardize publishing such that [supporting tools](../docs/specs/supporting-tools.md) may implement mechanisms for feature discoverability. ## Source Code From e7e3a6acc811831caeba6122d9eac07720217e7b Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 14:31:09 -0400 Subject: [PATCH 11/12] apply @jkeech suggestions Co-authored-by: John Keech --- proposals/devcontainer-features-distribution.md | 2 +- proposals/devcontainer-features.md | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/devcontainer-features-distribution.md b/proposals/devcontainer-features-distribution.md index 817d88b5..7ec78a6f 100644 --- a/proposals/devcontainer-features-distribution.md +++ b/proposals/devcontainer-features-distribution.md @@ -125,7 +125,7 @@ oras push ${REGISTRY}/${NAMESPACE}:latest \ ### Directly Reference Tarball -A feature can be referenced directly in a user's [`devcontainer.json`](/docs/specs/devcontainer-reference.md#devcontainerjson) file by an HTTP URI that points to the tarball from the [package step](#packaging). +A feature can be referenced directly in a user's [`devcontainer.json`](/docs/specs/devcontainer-reference.md#devcontainerjson) file by an HTTP or HTTPS URI that points to the tarball from the [package step](#packaging). ### Addendum: Locally Referenced diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index 2296c82b..5c1e3394 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -24,7 +24,7 @@ The properties of the file are as follows: | Property | Type | Description | | :--- | :--- | :--- | -| id | string | Id of the feature/definition. The id should be unique in the context of the repository/published package where the feature exists. | +| id | string | Id of the feature/definition. The id should be unique in the context of the repository/published package where the feature exists and must match the name of the directory where the `devcontainer-feature.json` resides. | | version | string | The semantic version of the feature. | | name | string | Name of the feature/definition. | | description | string | Description of the feature/definition. | @@ -152,7 +152,7 @@ All feature `id` provided in `overrideFeatureInstallOrder` must also exist in th #### (2) installsAfter -This propertry is defined in an individual feature's `devcontainer-feature.json` file by the feature author. `installsAfter` allows an author to provide the tooling hints on loose dependencies between features. +This property is defined in an individual feature's `devcontainer-feature.json` file by the feature author. `installsAfter` allows an author to provide the tooling hints on loose dependencies between features. After `overrideFeatureInstallOrder` is resolved, any remaining features that declare an `installsAfter` must be installed after the features declared in the property, provided that the features have also been declared in the `features` property. @@ -237,9 +237,9 @@ echo "Optimize? $OPTIMIZE" ...output the following: ``` -3.10 -false -true +Version is 3.10 +Pip? false +Optimize? true ``` ### Implementation Notes From 17cbfa5aa223323d869c118fac5adca8d1d29dc4 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Wed, 27 Jul 2022 18:40:05 +0000 Subject: [PATCH 12/12] note about compatibility with base images --- proposals/devcontainer-features.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/proposals/devcontainer-features.md b/proposals/devcontainer-features.md index 5c1e3394..b32dab34 100644 --- a/proposals/devcontainer-features.md +++ b/proposals/devcontainer-features.md @@ -1,7 +1,10 @@ # Dev container features reference -Dev container 'features' are self-contained units of installation code and development container configuration. Features are designed to install atop a wide-range of base container images. -Features are generally designed to be installed on top of a subset of base container images (eg: debian-based images). +Dev container 'features' are self-contained, shareable units of installation code and development container configuration. + +> While 'features' may be installed on top of any base image, the implementation of a feature might restrict it to a subset of possible base images. +> +> For example, some features may be authored to work with a certain linux distro (e.g. debian-based images that use the `apt` package manager). Feature metadata is captured by a `devcontainer-feature.json` file in the root folder of the feature.