-
Notifications
You must be signed in to change notification settings - Fork 1.6k
v1-to-v3 CRD migration controller to enable API server removal #12012
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
coutinhop
merged 101 commits into
projectcalico:master
from
caseydavenport:caseydavenport/datastore-migration-controller-prototype
Mar 19, 2026
Merged
Changes from all commits
Commits
Show all changes
101 commits
Select commit
Hold shift + click to select a range
a14ee22
Add DatastoreMigration API type and generated files
caseydavenport 99d6759
Add datastore migration controller prototype
caseydavenport a961fb7
Refactor kind cluster setup out of node/ into shared infrastructure
caseydavenport 95a9701
Skip loading unchanged images onto kind cluster
caseydavenport 463849f
Address review feedback
caseydavenport 80f2484
Remove unused test-webserver from kind cluster setup
caseydavenport 2023975
Restore useful comments removed during refactor
caseydavenport 3222e8b
Clean up kind target naming
caseydavenport e3cddb5
Restore remaining comments in deploy script
caseydavenport 2e37544
Rename kind scripts and make VALUES_FILE overridable
caseydavenport 1fbb177
Fix kind-deploy chart dependency resolution
caseydavenport 0edf653
Pass image list to load script instead of hardcoding
caseydavenport fed3cd1
Default CALICO_API_GROUP to projectcalico.org/v3
caseydavenport cf89785
Default CALICO_API_GROUP to v3 for kind cluster setup
caseydavenport 3d856a9
Fix k8s-test to call kind-up/kind-down from repo root
caseydavenport 1cc2af6
Add OwnerReference UID remapping to migration controller
caseydavenport 2c9adab
Add migration test bed with connectivity and OwnerReference verification
caseydavenport b530e6a
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport 662f175
Fix libbpf clone failure in git worktrees
caseydavenport 4446559
Add IPAMBlock and IPAMHandle migration support
caseydavenport 5e1078b
Add bounded worker pool for concurrent resource creation
caseydavenport b03f089
Add abort/rollback support and disruption testing
caseydavenport ff7a50e
Add per-type progress reporting to migration controller
caseydavenport 57d0c29
Implement v3 resource deletion for abort cleanup
caseydavenport 5b01a9a
Add retry with exponential backoff for transient API errors
caseydavenport 7adb76f
Add Prometheus metrics for migration observability
caseydavenport 59d68e2
Generate DatastoreMigration CRD from kubebuilder markers
caseydavenport f17b96a
Fix migration controller status conflicts and RBAC for CRD cleanup
caseydavenport fa14541
Use v1beta1 for the DatastoreMigration CRD API version
caseydavenport 80a41b5
Make kind-reload detect and rebuild stale component images
caseydavenport 2d95176
Merge remote-tracking branch 'origin/master' into HEAD
caseydavenport 0a0163d
Refactor migration controller: informer, typed client, pre-validation…
caseydavenport 9f23413
Address code review feedback: cleanup TODOs, remove dead code, add ed…
caseydavenport 929873b
Add lifecycle integration tests for migration controller
caseydavenport 20ff6b7
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport 0ba77ee
Remove felix changes (moved to separate PR)
caseydavenport b31b654
Fix makefile
caseydavenport 249007a
Restore Makefile to latest master, add kind-migration-test target
caseydavenport 0407d15
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport 18ab83b
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport 25f3509
Generate files
caseydavenport e5df584
Fix vet
caseydavenport 725551b
Fix vet
caseydavenport 63793fe
Add WaitingForConflictResolution phase and conflict re-checking
caseydavenport 23016d6
Fix AlreadyExists race: read back v3 resource for UID mapping
caseydavenport 5613781
Fix updateMetadata to update dm in-place with server response
caseydavenport c9c90c0
Use %w instead of %v for error wrapping in fmt.Errorf
caseydavenport 5dd2a86
Use safe type assertions throughout migration controller
caseydavenport 70dad31
Rename receiver from c to m on migrationController
caseydavenport 8f3af0f
Validate Spec.Kind in migration reconcile
caseydavenport d646a02
Protect global registry with mutex
caseydavenport e52b0be
Only delete migration-created resources on abort
caseydavenport e7c8f87
Remove redundant re-fetches after updateMetadata fix
caseydavenport 0901695
Add dev-image and dev-push Make targets
caseydavenport 8dd2a56
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport 71b56bc
Add dev-image and dev-push Make targets
caseydavenport 43b1413
Merge branch 'casey-dev-image-targets' into caseydavenport/datastore-…
caseydavenport aaf5131
Point OPERATOR_BRANCH at migration support branch on fork
caseydavenport 99cbc50
Merge remote-tracking branch 'cd4/caseydavenport/datastore-migration-…
caseydavenport 1b3d207
Rename DatastoreMigrationSpec.Kind to Type to match CRD schema
caseydavenport 1e10440
Grant operator RBAC to read DatastoreMigration CRs
caseydavenport 08d5cef
Gate Converged → Complete on calico-node v3 rollout
caseydavenport 45349a7
Fix dev-image build and add migration RBAC to operator chart
caseydavenport c21c2de
Wait for migration RBAC before starting migration
caseydavenport 088e202
Check RBAC for all registered resource types before starting migration
caseydavenport 24deb15
Combine Types Done/Total into single printer column
caseydavenport 56c2442
Move Migrated/Skipped/Conflicts to -o wide output
caseydavenport adfb99b
Keep v1 datastore locked after migration to prevent IPAM leaks
caseydavenport 7759774
Add Message column to DatastoreMigration status output
caseydavenport 514e633
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport f21169f
Address straightforward PR review comments
caseydavenport 30a046a
Refactor migration to use controller-runtime client for v3 operations
caseydavenport 04a04a2
Add comprehensive test coverage for migration controller
caseydavenport 4136b35
Make generation
caseydavenport 76025ef
Remove dev-images and dev-push workflow
caseydavenport f30eca5
Mechanical cleanup from PR review
caseydavenport 970da09
Extract test helpers and reduce boilerplate
caseydavenport 4075ec6
Add DeferredCRDController and remove unstructured migration client
caseydavenport f392f74
Add namespace/operator detection utilities and terminal error handling
caseydavenport e83062c
Consolidate conflict detection and pre-check before migration
caseydavenport 263d13c
Add BackendAccessor interface, block rollback from Converged
caseydavenport 7e984f3
Refactor ResourceMigrator from struct to generic interface
caseydavenport bf42633
Add envtest-based integration test for migration controller
caseydavenport 81de6f4
Eliminate re-list in OwnerRef remapping, remove unnecessary read-back
caseydavenport 8819aed
Remove redundant policy name conversion from migration
caseydavenport 37834a7
Add conflict resolution FV test and requeueAfter for polling phases
caseydavenport 15a08d2
Add rollback FV test and startController helper
caseydavenport 59e40ab
Add deletion-blocked-then-completed FV test, fix Converged deletion bug
caseydavenport d51979e
Add tests for DeferredCRDController
caseydavenport c54df72
Add unit tests for defaultConvert and RemapOwnerReferences
caseydavenport 3331edd
Fix assorted bugs found during review
caseydavenport ff68318
'make generate'
coutinhop f53bfc2
'make generate'
coutinhop ab64a34
fix gen-deps conflicts
coutinhop 4dd6cdb
Fix staticcheck lint errors in kube-controllers tests
caseydavenport a4f9727
Revert operator branch/org override back to defaults
caseydavenport df9619b
Use NewSimpleClientset for apiextensions fake client in deferred CRD …
caseydavenport 6251640
Set up envtest binaries for kube-controllers migration tests
caseydavenport b893e31
Extract FindRepoRoot into shared testutils and fix envtest CRD paths
caseydavenport 298484d
Add unit tests for IPAM convert functions and backend error FV test
caseydavenport 3f3c344
Configure logrus formatting and fix rate limiter noise in migration t…
caseydavenport File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| # Connectivity test workloads for migration testing. A client pod continuously | ||
| # probes the server pod via a Service. After migration, we check the client's | ||
| # logs for any probe failures — there should be zero packet loss since the | ||
| # dataplane state is cached while the datastore is locked. | ||
|
|
||
| --- | ||
| apiVersion: v1 | ||
| kind: Namespace | ||
| metadata: | ||
| name: migration-test | ||
|
|
||
| --- | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: server | ||
| namespace: migration-test | ||
| labels: | ||
| app: migration-server | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: migration-server | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: migration-server | ||
| spec: | ||
| containers: | ||
| - name: server | ||
| image: busybox:1.36 | ||
| command: | ||
| - /bin/sh | ||
| - -c | ||
| - | | ||
| mkdir -p /www | ||
| echo "ok" > /www/index.html | ||
| httpd -f -p 8080 -h /www | ||
| ports: | ||
| - containerPort: 8080 | ||
| readinessProbe: | ||
| httpGet: | ||
| path: / | ||
| port: 8080 | ||
| initialDelaySeconds: 1 | ||
| periodSeconds: 2 | ||
|
|
||
| --- | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: server | ||
| namespace: migration-test | ||
| spec: | ||
| selector: | ||
| app: migration-server | ||
| ports: | ||
| - port: 80 | ||
| targetPort: 8080 | ||
|
|
||
| --- | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: client | ||
| namespace: migration-test | ||
| labels: | ||
| app: migration-client | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: migration-client | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: migration-client | ||
| spec: | ||
| containers: | ||
| - name: client | ||
| image: busybox:1.36 | ||
| command: | ||
| - /bin/sh | ||
| - -c | ||
| - | | ||
| echo "PROBE_START $(date +%s)" | ||
| while true; do | ||
| ts=$(date +%s) | ||
| if wget -q -T 2 -O /dev/null http://server.migration-test.svc.cluster.local 2>/dev/null; then | ||
| echo "PROBE_OK $ts" | ||
| else | ||
| echo "PROBE_FAIL $ts" | ||
| fi | ||
| sleep 1 | ||
| done |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.