gc-tag should use label, not annotation #205
Comments
We want to find all objects using the garbage-collect tag. This means we *should* have used a label for this, so we get indexing, server-side filtering, etc. Migration plan annotation->label: - phase1 (this change): write annotation+label; continue to read annotation - release v.A - phase2 (future change): read/write label only; remove annotation code - release v.B This means writers have to run v.A at least once before moving to v.B, and can't go back from v.B. Plan B (if that nice v.A->v.B track doesn't work out for a user) is to run `update` with some version off one set of configs, and then (without removing objects from configs) run `update --skip-gc` with v.B once. Then regular `update` with v.B will work with the new label. Partially-fixes vmware-archive#205
We want to find all objects using the garbage-collect tag. This means we *should* have used a label for this, so we get indexing, server-side filtering, etc. Migration plan annotation->label: - phase1 (this change): write annotation+label; continue to read annotation - release v.A - phase2 (future change): read/write label only; remove annotation code - release v.B This means writers have to run v.A at least once before moving to v.B, and can't go back from v.B. Plan B (if that nice v.A->v.B track doesn't work out for a user) is to run `update` with some version off one set of configs, and then (without removing objects from configs) run `update --skip-gc` with v.B once. Then regular `update` with v.B will work with the new label. Partially-fix for vmware-archive#205
We want to find all objects using the garbage-collect tag. This means we *should* have used a label for this, so we get indexing, server-side filtering, etc. Migration plan annotation->label: - phase1 (this change): write annotation+label; continue to read annotation - release v.A - phase2 (future change): read/write label only; remove annotation code - release v.B This means writers have to run v.A at least once before moving to v.B, and can't go back from v.B. Plan B (if that nice v.A->v.B track doesn't work out for a user) is to run `update` with some version off one set of configs, and then (without removing objects from configs) run `update --skip-gc` with v.B once. Then regular `update` with v.B will work with the new label. Partial-fix for vmware-archive#205
|
@gotwarlost points out in qbec's version of garbage collection that the Service->Endpoints controller copies labels across, but not annotations. We will need to ensure that we don't garbage collect Endpoints unnecessarily (deserves an explicit integration test case). The fix probably involves recording "kubecfg created this object as <group/kind>" annotation, and then double-checking the resource is both marked and the original kind before deleting. Note this also means Deployments will only be deleted via the original apigroup (currently kubecfg's uid logic makes this safe). |
|
nice trick with the annotation check. I'll incorporate some version of this in |
|
Hi @anguslees Any update on the progress of phase 2 for this, having kubecfg retrieve every resource for each resource type is extremely inefficient when the cluster has even a few hundred unique resources. |
Garbage collection works by listing everything with the gc-tag. In a busy cluster, we really want that filter to happen server-side and ideally using an index of some sort.
That means we should use a Kubernetes label, not an annotation.
I think this will require a two-step migration plan (write both but continue to read annotation; release; drop support for annotation; release).
The text was updated successfully, but these errors were encountered: