Introduce WG Checkpoint Restore #8508
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3724,6 +3724,45 @@ workinggroups: | |
| liaison: | ||
| github: aojea | ||
| name: Antonio Ojea | ||
| - dir: wg-checkpoint-restore | ||
| name: Checkpoint Restore | ||
| mission_statement: > | ||
| This working group aims to provide a central location for the community to discuss | ||
| the integration of Checkpoint/Restore functionality into Kubernetes. | ||
|
|
||
| charter_link: charter.md | ||
adrianreber marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| stakeholder_sigs: | ||
|
There was a problem hiding this comment. sig auth may have a big say in security of this whole restoration pipeline There was a problem hiding this comment. Thank you for pointing this out! Security is definitely an important topic that we need to discuss with sig-auth, both for the checkpoint API and the restoration pipeline. The following paper and master thesis describe our recent work on this topic: There was a problem hiding this comment. I added sig auth to the list of stakeholder sigs There was a problem hiding this comment. this showed up in the sig-auth meeting, we may have missed the discussion around this WG if this WG is contemplating taking state from a running pod / saving it / letting it be consumed on another node or from another pod or another namespace, then sig-auth is definitely interested in making sure the permissions model around that exists and is ~consistent with similar things Kubernetes does elsewhere (like PVC / snapshots) We're happy to consult on that, I'm not sure our awareness / involvement rises to the level of sponsoring the WG :) cc @kubernetes/sig-auth-leads There was a problem hiding this comment. nod.. definitely needs an extra level of security due to customer data being serialized and available in the checkpoint, esp if not encrypted, but also due to windows of opportunity to do transactions/data manipulation.. then "undo" them by restoring a checkpoint There was a problem hiding this comment. This is a valuable initiative. The charter mentions that the scope includes checkpointing and restoring 'workloads' and providing 'guidance for developers on checkpoint-friendly app design.' Given this focus, it's essential for SIG Apps to be involved as a key stakeholder. There was a problem hiding this comment. @janetkuo This is a good idea, thank you so much for suggesting it! There was a problem hiding this comment. Yes, thanks @janetkuo. I added SIG Apps to the proposal. There was a problem hiding this comment. I agree with Janet here, but please make sure to show up and present the scope of this proposal to one of the future SIG-Apps calls. |
||
| - API Machinery | ||
| - Apps | ||
| - Auth | ||
| - Node | ||
| - Scheduling | ||
| label: checkpoint-restore | ||
| leadership: | ||
| chairs: | ||
| - github: adrianreber | ||
| name: Adrian Reber | ||
| company: Red Hat | ||
| email: [email protected] | ||
| - github: haircommander | ||
| name: Peter Hunt | ||
| company: Red Hat | ||
| email: [email protected] | ||
| - github: rst0git | ||
| name: Radostin Stoyanov | ||
| company: University of Oxford | ||
| email: [email protected] | ||
| - github: viktoriaas | ||
| name: Viktória Spišaková | ||
| company: Masaryk University | ||
| email: [email protected] | ||
| meetings: [] | ||
| contact: | ||
| slack: wg-checkpoint-restore | ||
| mailing_list: https://groups.google.com/a/kubernetes.io/g/wg-checkpoint-restore | ||
| liaison: | ||
| github: BenTheElder | ||
| name: Benjamin Elder | ||
| - dir: wg-data-protection | ||
| name: Data Protection | ||
| mission_statement: > | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| <!--- | ||
| This is an autogenerated file! | ||
|
|
||
| Please do not edit this file directly, but instead make changes to the | ||
| sigs.yaml file in the project root. | ||
|
|
||
| To understand how this file is generated, see https://git.k8s.io/community/generator/README.md | ||
| ---> | ||
| # Checkpoint Restore Working Group | ||
|
|
||
| This working group aims to provide a central location for the community to discuss the integration of Checkpoint/Restore functionality into Kubernetes. | ||
|
|
||
| The [charter](charter.md) defines the scope and governance of the Checkpoint Restore Working Group. | ||
|
|
||
| ## Stakeholder SIGs | ||
| * [SIG API Machinery](/sig-api-machinery) | ||
| * [SIG Apps](/sig-apps) | ||
| * [SIG Auth](/sig-auth) | ||
| * [SIG Node](/sig-node) | ||
| * [SIG Scheduling](/sig-scheduling) | ||
|
|
||
|
|
||
|
|
||
| ## Organizers | ||
|
|
||
| * Adrian Reber (**[@adrianreber](https://github.com/adrianreber)**), Red Hat | ||
| * Peter Hunt (**[@haircommander](https://github.com/haircommander)**), Red Hat | ||
| * Radostin Stoyanov (**[@rst0git](https://github.com/rst0git)**), University of Oxford | ||
| * Viktória Spišaková (**[@viktoriaas](https://github.com/viktoriaas)**), Masaryk University | ||
|
|
||
| ## Contact | ||
| - Slack: [#wg-checkpoint-restore](https://kubernetes.slack.com/messages/wg-checkpoint-restore) | ||
| - [Mailing list](https://groups.google.com/a/kubernetes.io/g/wg-checkpoint-restore) | ||
| - [Open Community Issues/PRs](https://github.com/kubernetes/community/labels/wg%2Fcheckpoint-restore) | ||
adrianreber marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - Steering Committee Liaison: Benjamin Elder (**[@BenTheElder](https://github.com/BenTheElder)**) | ||
| <!-- BEGIN CUSTOM CONTENT --> | ||
|
|
||
| <!-- END CUSTOM CONTENT --> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
|
|
||
| # WG Checkpoint Restore Charter | ||
|
|
||
| This charter adheres to the conventions described in the [Kubernetes Charter README] and uses | ||
| the Roles and Organization Management outlined in [sig-governance]. | ||
|
|
||
| ## Scope | ||
|
|
||
| The Checkpoint/Restore Working Group aims to solve the problem of transparently | ||
| checkpointing and restoring workloads in Kubernetes, a [functionality discussed | ||
| for over five years][kep2008]. The group will deliver the design and | ||
| implementation of Checkpoint/Restore functionality in Kubernetes, serving as a | ||
| central hub for community information and discussion. This initiative addresses | ||
| a wide range of problems, including fault tolerance, improved resource | ||
| utilization, and accelerated application startup times. | ||
|
|
||
| ### In scope | ||
|
|
||
| - Identify core Kubernetes checkpoint/restore use cases (e.g., live migration, | ||
| fault tolerance, debugging, snapshotting) and gather stakeholder requirements. | ||
| - Investigate and propose Kubernetes APIs for checkpoint/restore operations. | ||
| - Work with SIGs for the best integration of checkpoint/restore functionality | ||
| and APIs. | ||
| - Provide guidance for developers on checkpoint-friendly app design and | ||
|
There was a problem hiding this comment. there may be API needed to communicate between the app and API server that the checkopoint is requested AND/OR that the app is ready for checkpoint. Something that is beyond just guidance There was a problem hiding this comment. That is actually something we discussed how to do in containers for years now (outside of Kubernetes). But we never found the right way how to do this. We were looking at kernel interfaces or systemd interfaces because for many applications it could be helpful to free temporary memory to reduce checkpoint size or even drop confidential information. Also after restore it would be good to tell the application that maybe certain cryptographic values need to be reset or regenerated. I will try to include something mentioning this. Thanks. |
||
| recommendations for operators on feature management. | ||
| - Work closely with relevant upstream projects (CRI-O, containerd, CRIU, gVisor) | ||
| for alignment and integration. | ||
| - Revisit the existing implementations to find and remedy possible inefficiencies. | ||
| One example is the existing checkpoint archive format which has already been | ||
| identified as being a major source of slowdown. | ||
|
|
||
| ### Out of scope | ||
|
|
||
| - Not focused on general OS-level checkpointing outside Kubernetes | ||
| pods/containers. | ||
| - Will not dictate internal application checkpointing logic; focuses on | ||
| Kubernetes platform orchestration of *container/pod state. | ||
|
|
||
| ## Stakeholders | ||
|
|
||
| Stakeholders in this working group span multiple SIGs that own parts of the | ||
| code in core kubernetes components and addons. | ||
|
|
||
| - SIG API Machinery | ||
| - SIG Node | ||
| - SIG Scheduling | ||
| - SIG Auth | ||
| - SIG Apps | ||
|
|
||
| ## Deliverables | ||
|
|
||
| The list of deliverables include the following high level features: | ||
|
|
||
| - In the early stage, we mainly want to offer a well-defined location for the | ||
adrianreber marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| community to find information, ask questions, and discuss the next steps of | ||
| enabling checkpoint and restore in Kubernetes. | ||
|
|
||
| Later: | ||
|
|
||
| - Ability to checkpoint and restore a container using kubectl | ||
| - Ability to checkpoint and restore a pod using kubectl | ||
| - Integration of container/pod checkpointing in scheduling decisions | ||
|
There was a problem hiding this comment. Why pod checkpointing would have anything to do with scheduling? There was a problem hiding this comment. Because, as far as I know, a pod is always scheduled on one node. It doesn't sound useful to base the scheduling on the possibility to migrate containers. Container migration is an important first step, but for automatic scheduling decisions, it would make more sense to be able to easily migrate a complete pod. There was a problem hiding this comment.
I agree that pod checkpointing needs to take scheduling into consideration. Let's assume the following scenario: after a Pod is checkpointed, it might not be deleted. However, since the memory, CPU, and GPU resources have already been dumped to a volume, the resources originally allocated to the Pod could then be reallocated to other Pods. When the Pod needs to be resumed later via restore, the scheduler should also be involved. |
||
|
|
||
| ## Roles and Organization Management | ||
|
|
||
| This WG adheres to the Roles and Organization Management outlined in [wg-governance] | ||
| and opts-in to updates and modifications to [wg-governance]. | ||
|
|
||
| [wg-governance]: /committee-steering/governance/wg-governance.md | ||
|
|
||
| Additionally, the WG commits to: | ||
|
|
||
| - maintain a solid communication line between the Kubernetes groups and the | ||
| wider CNCF community | ||
|
|
||
| ## Timelines and Disbanding | ||
|
|
||
| As a first mandate, the WG will propose a draft roadmap and identify key tasks in the first quarter of operation. | ||
|
|
||
| After that, the WG will facilitate collaboration among community members to explore possible APIs and draft proposals for their integration into Kubernetes, which will then be presented to the relevant SIGs. | ||
|
|
||
| Achieving the aforementioned deliverables, also mentioned in the `In Scope` | ||
| section, will allow us to decide when to disband this WG. There is no | ||
| expectations that the Working Group will be converted into a SIG long term. | ||
|
|
||
| [sig-governance]: https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance.md | ||
| [Kubernetes Charter README]: https://github.com/kubernetes/community/blob/master/committee-steering/governance/README.md | ||
| [kep2008]: https://github.com/kubernetes/enhancements/issues/2008 | ||
Uh oh!
There was an error while loading. Please reload this page.