-
Notifications
You must be signed in to change notification settings - Fork 185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: backup and restore specification #1610
base: main
Are you sure you want to change the base?
Changes from 1 commit
7e33f88
f4391e4
aedfdaf
d22a47b
69870d2
26f3f1a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,107 @@ | ||
# Backup and Restore Commands | ||
|
||
This document describes how the backup and restore commands will work and how the feature will be rolled out. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What are we backing up and restoring? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added something in a new paragraph above this. |
||
The proposal is to create the feature as a minimum viable product and follow that up with non breaking features to the commands. | ||
The initial implementation will be the easiest implementation that is useful. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a bit confusing. |
||
|
||
## Minium Viable Product | ||
TerryHowe marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The backup command will initially just support writing to a directory and the restore command will only support reading from a directory. | ||
|
||
### oras backup | ||
|
||
The backup command will initially only support reading a list of files from the command line and writing to a directory: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. List of files or list or artifacts? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed |
||
|
||
```bash | ||
oras backup --output ./mirror registry.k8s.io/kube-apiserver-arm64:v1.31.0 registry.k8s.io/kube-controller-manager-arm64:v1.31.0 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will here be an option to suppress output? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Initially, there will be no output to stdout. |
||
``` | ||
|
||
The generated directory structure is `<specified-directory>/<repository>`. | ||
The command above puts the OCI layout for the Kubernetes API server in `mirror/kube-api-server-arm64`. | ||
The directory structure with intermediate blobs removed: | ||
|
||
```bash | ||
$ find mirror | ||
mirror | ||
mirror/kube-apiserver-arm64 | ||
mirror/kube-apiserver-arm64 | ||
mirror/kube-apiserver-arm64/ingest | ||
mirror/kube-apiserver-arm64/oci-layout | ||
mirror/kube-apiserver-arm64/blobs | ||
mirror/kube-apiserver-arm64/blobs/sha256 | ||
mirror/kube-apiserver-arm64/blobs/sha256/3f4e2c5863480125882d92060440a5250766bce764fee10acdbac18c872e4dc7 | ||
... | ||
mirror/kube-apiserver-arm64/blobs/sha256/4f80fb2b9442dbecd41e68b598533dcaaf58f9d45cce2e03a715499aa9f6b676 | ||
mirror/kube-apiserver-arm64/index.json | ||
mirror/kube-controller-manager-arm64 | ||
mirror/kube-controller-manager-arm64 | ||
mirror/kube-controller-manager-arm64/ingest | ||
mirror/kube-controller-manager-arm64/oci-layout | ||
mirror/kube-controller-manager-arm64/blobs | ||
mirror/kube-controller-manager-arm64/blobs/sha256 | ||
mirror/kube-controller-manager-arm64/blobs/sha256/3f4e2c5863480125882d92060440a5250766bce764fee10acdbac18c872e4dc7 | ||
... | ||
mirror/kube-controller-manager-arm64/blobs/sha256/4f80fb2b9442dbecd41e68b598533dcaaf58f9d45cce2e03a715499aa9f6b676 | ||
mirror/kube-controller-manager-arm64/index.json | ||
$ | ||
``` | ||
|
||
The directory `mirror` must exist for the initial feature. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will it support the creation of a directory if it does not exist? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was thinking not for the initial release because later we want to support writing to a tgz file and this will help differentiate file output vs directory output. Some option could be added later. |
||
Each image will be stored in a subdirectory which matches the repository name. | ||
|
||
### oras restore | ||
|
||
Initially, the restore command will only support reading a directory and writing to a registry: | ||
TerryHowe marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```bash | ||
oras restore --input ./mirror localhost:15000/my-mirror | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will there be flags to support authentication and TLS for the remote registry? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, all the normal registry flags. I added some text about that. |
||
``` | ||
|
||
The above backup example would result in: | ||
```bash | ||
localhost:15000/my-mirror/kube-apiserver-arm64:v1.31.0 | ||
localhost:15000/my-mirror/kube-controller-manager-arm64:v1.31.0 | ||
``` | ||
|
||
A namespace in the registry will be optional. | ||
The registry in the above example could be specified as `localhost:15000`. | ||
|
||
|
||
Any directory in the input that does not contain an `index.json` shall be silently ignored. | ||
|
||
## Compressed Tar File Support | ||
|
||
After the command supports directory write and read, support will be added for write and read to a compressed tar file. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this trying to state that future enhancements after the initial release will add support for compressed archives? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes. |
||
The directory structure in the tar file would be the same as in the directory output. | ||
|
||
```bash | ||
oras backup --output ./mirror.tgz registry.k8s.io/kube-apiserver-arm64:v1.31.0 registry.k8s.io/kube-controller-manager-arm64:v1.31.0 | ||
``` | ||
|
||
There will be no validation on file name. | ||
The file name does not need to end in `tgz`, but the format will be compressed tar file. | ||
If the file exists, it will be overwritten. | ||
|
||
```bash | ||
oras restore --input ./mirror.tgz localhost:15000/my-mirror | ||
``` | ||
If the specified source is a file, the format is assumed to be a compressed tar file. | ||
|
||
## Backup file input from a file | ||
|
||
The backup command will support an `--input filename` argument which will be a file. | ||
TerryHowe marked this conversation as resolved.
Show resolved
Hide resolved
|
||
The format of the contents of the file is a list of images names separated by newlines. | ||
If the `--input` argument is specified, no images may be specified on the command line. | ||
|
||
## Backup file input from standard input | ||
|
||
The backup command will support the `--input -` argument to read a list of images from standard input. | ||
The format of the input is a list of images names separated by newlines. | ||
|
||
## Restore file input from standard input | ||
|
||
The restore command will support the `--input -` argument to read a compressed tar input from standard input. | ||
|
||
## Optimize blobs | ||
|
||
Add the feature to create blobs that are duplicated between images as hard links in the directory output and compressed tar files. | ||
TerryHowe marked this conversation as resolved.
Show resolved
Hide resolved
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should there be a problem statement that is included within this proposal that illustrates why this feature is needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added.