Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
a14ee22
Add DatastoreMigration API type and generated files
caseydavenport Mar 6, 2026
99d6759
Add datastore migration controller prototype
caseydavenport Mar 6, 2026
a961fb7
Refactor kind cluster setup out of node/ into shared infrastructure
caseydavenport Mar 6, 2026
95a9701
Skip loading unchanged images onto kind cluster
caseydavenport Mar 6, 2026
463849f
Address review feedback
caseydavenport Mar 6, 2026
80f2484
Remove unused test-webserver from kind cluster setup
caseydavenport Mar 6, 2026
2023975
Restore useful comments removed during refactor
caseydavenport Mar 6, 2026
3222e8b
Clean up kind target naming
caseydavenport Mar 6, 2026
e3cddb5
Restore remaining comments in deploy script
caseydavenport Mar 6, 2026
2e37544
Rename kind scripts and make VALUES_FILE overridable
caseydavenport Mar 6, 2026
1fbb177
Fix kind-deploy chart dependency resolution
caseydavenport Mar 6, 2026
0edf653
Pass image list to load script instead of hardcoding
caseydavenport Mar 6, 2026
fed3cd1
Default CALICO_API_GROUP to projectcalico.org/v3
caseydavenport Mar 6, 2026
cf89785
Default CALICO_API_GROUP to v3 for kind cluster setup
caseydavenport Mar 6, 2026
3d856a9
Fix k8s-test to call kind-up/kind-down from repo root
caseydavenport Mar 6, 2026
1cc2af6
Add OwnerReference UID remapping to migration controller
caseydavenport Mar 7, 2026
2c9adab
Add migration test bed with connectivity and OwnerReference verification
caseydavenport Mar 7, 2026
b530e6a
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport Mar 8, 2026
662f175
Fix libbpf clone failure in git worktrees
caseydavenport Mar 8, 2026
4446559
Add IPAMBlock and IPAMHandle migration support
caseydavenport Mar 8, 2026
5e1078b
Add bounded worker pool for concurrent resource creation
caseydavenport Mar 8, 2026
b03f089
Add abort/rollback support and disruption testing
caseydavenport Mar 8, 2026
ff7a50e
Add per-type progress reporting to migration controller
caseydavenport Mar 8, 2026
57d0c29
Implement v3 resource deletion for abort cleanup
caseydavenport Mar 8, 2026
5b01a9a
Add retry with exponential backoff for transient API errors
caseydavenport Mar 8, 2026
7adb76f
Add Prometheus metrics for migration observability
caseydavenport Mar 8, 2026
59d68e2
Generate DatastoreMigration CRD from kubebuilder markers
caseydavenport Mar 8, 2026
f17b96a
Fix migration controller status conflicts and RBAC for CRD cleanup
caseydavenport Mar 8, 2026
fa14541
Use v1beta1 for the DatastoreMigration CRD API version
caseydavenport Mar 8, 2026
80a41b5
Make kind-reload detect and rebuild stale component images
caseydavenport Mar 8, 2026
2d95176
Merge remote-tracking branch 'origin/master' into HEAD
caseydavenport Mar 9, 2026
0a0163d
Refactor migration controller: informer, typed client, pre-validation…
caseydavenport Mar 9, 2026
9f23413
Address code review feedback: cleanup TODOs, remove dead code, add ed…
caseydavenport Mar 10, 2026
929873b
Add lifecycle integration tests for migration controller
caseydavenport Mar 10, 2026
20ff6b7
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport Mar 11, 2026
0ba77ee
Remove felix changes (moved to separate PR)
caseydavenport Mar 11, 2026
b31b654
Fix makefile
caseydavenport Mar 11, 2026
249007a
Restore Makefile to latest master, add kind-migration-test target
caseydavenport Mar 11, 2026
0407d15
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport Mar 12, 2026
18ab83b
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport Mar 12, 2026
25f3509
Generate files
caseydavenport Mar 12, 2026
e5df584
Fix vet
caseydavenport Mar 12, 2026
725551b
Fix vet
caseydavenport Mar 12, 2026
63793fe
Add WaitingForConflictResolution phase and conflict re-checking
caseydavenport Mar 13, 2026
23016d6
Fix AlreadyExists race: read back v3 resource for UID mapping
caseydavenport Mar 13, 2026
5613781
Fix updateMetadata to update dm in-place with server response
caseydavenport Mar 13, 2026
c9c90c0
Use %w instead of %v for error wrapping in fmt.Errorf
caseydavenport Mar 13, 2026
5dd2a86
Use safe type assertions throughout migration controller
caseydavenport Mar 13, 2026
70dad31
Rename receiver from c to m on migrationController
caseydavenport Mar 13, 2026
8f3af0f
Validate Spec.Kind in migration reconcile
caseydavenport Mar 13, 2026
d646a02
Protect global registry with mutex
caseydavenport Mar 13, 2026
e52b0be
Only delete migration-created resources on abort
caseydavenport Mar 13, 2026
e7c8f87
Remove redundant re-fetches after updateMetadata fix
caseydavenport Mar 13, 2026
0901695
Add dev-image and dev-push Make targets
caseydavenport Mar 16, 2026
8dd2a56
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport Mar 16, 2026
71b56bc
Add dev-image and dev-push Make targets
caseydavenport Mar 16, 2026
43b1413
Merge branch 'casey-dev-image-targets' into caseydavenport/datastore-…
caseydavenport Mar 16, 2026
aaf5131
Point OPERATOR_BRANCH at migration support branch on fork
caseydavenport Mar 16, 2026
99cbc50
Merge remote-tracking branch 'cd4/caseydavenport/datastore-migration-…
caseydavenport Mar 16, 2026
1b3d207
Rename DatastoreMigrationSpec.Kind to Type to match CRD schema
caseydavenport Mar 16, 2026
1e10440
Grant operator RBAC to read DatastoreMigration CRs
caseydavenport Mar 16, 2026
08d5cef
Gate Converged → Complete on calico-node v3 rollout
caseydavenport Mar 16, 2026
45349a7
Fix dev-image build and add migration RBAC to operator chart
caseydavenport Mar 17, 2026
c21c2de
Wait for migration RBAC before starting migration
caseydavenport Mar 17, 2026
088e202
Check RBAC for all registered resource types before starting migration
caseydavenport Mar 17, 2026
24deb15
Combine Types Done/Total into single printer column
caseydavenport Mar 17, 2026
56c2442
Move Migrated/Skipped/Conflicts to -o wide output
caseydavenport Mar 17, 2026
adfb99b
Keep v1 datastore locked after migration to prevent IPAM leaks
caseydavenport Mar 17, 2026
7759774
Add Message column to DatastoreMigration status output
caseydavenport Mar 17, 2026
514e633
Merge remote-tracking branch 'origin/master' into caseydavenport/data…
caseydavenport Mar 17, 2026
f21169f
Address straightforward PR review comments
caseydavenport Mar 17, 2026
30a046a
Refactor migration to use controller-runtime client for v3 operations
caseydavenport Mar 17, 2026
04a04a2
Add comprehensive test coverage for migration controller
caseydavenport Mar 17, 2026
4136b35
Make generation
caseydavenport Mar 17, 2026
76025ef
Remove dev-images and dev-push workflow
caseydavenport Mar 17, 2026
f30eca5
Mechanical cleanup from PR review
caseydavenport Mar 17, 2026
970da09
Extract test helpers and reduce boilerplate
caseydavenport Mar 17, 2026
4075ec6
Add DeferredCRDController and remove unstructured migration client
caseydavenport Mar 17, 2026
f392f74
Add namespace/operator detection utilities and terminal error handling
caseydavenport Mar 17, 2026
e83062c
Consolidate conflict detection and pre-check before migration
caseydavenport Mar 17, 2026
263d13c
Add BackendAccessor interface, block rollback from Converged
caseydavenport Mar 17, 2026
7e984f3
Refactor ResourceMigrator from struct to generic interface
caseydavenport Mar 18, 2026
bf42633
Add envtest-based integration test for migration controller
caseydavenport Mar 18, 2026
81de6f4
Eliminate re-list in OwnerRef remapping, remove unnecessary read-back
caseydavenport Mar 18, 2026
8819aed
Remove redundant policy name conversion from migration
caseydavenport Mar 18, 2026
37834a7
Add conflict resolution FV test and requeueAfter for polling phases
caseydavenport Mar 18, 2026
15a08d2
Add rollback FV test and startController helper
caseydavenport Mar 18, 2026
59e40ab
Add deletion-blocked-then-completed FV test, fix Converged deletion bug
caseydavenport Mar 18, 2026
d51979e
Add tests for DeferredCRDController
caseydavenport Mar 18, 2026
c54df72
Add unit tests for defaultConvert and RemapOwnerReferences
caseydavenport Mar 18, 2026
3331edd
Fix assorted bugs found during review
caseydavenport Mar 18, 2026
ff68318
'make generate'
coutinhop Mar 18, 2026
f53bfc2
'make generate'
coutinhop Mar 18, 2026
ab64a34
fix gen-deps conflicts
coutinhop Mar 18, 2026
4dd6cdb
Fix staticcheck lint errors in kube-controllers tests
caseydavenport Mar 18, 2026
a4f9727
Revert operator branch/org override back to defaults
caseydavenport Mar 18, 2026
df9619b
Use NewSimpleClientset for apiextensions fake client in deferred CRD …
caseydavenport Mar 18, 2026
6251640
Set up envtest binaries for kube-controllers migration tests
caseydavenport Mar 19, 2026
b893e31
Extract FindRepoRoot into shared testutils and fix envtest CRD paths
caseydavenport Mar 19, 2026
298484d
Add unit tests for IPAM convert functions and backend error FV test
caseydavenport Mar 19, 2026
3f3c344
Configure logrus formatting and fix rate limiter noise in migration t…
caseydavenport Mar 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ builder.coverprofile

/* Image build stamp files for kind testing */
.stamp.*
hack/test/kind/infra/operator

/* Created by local kind cluster */
hack/test/kind/kind
Expand All @@ -40,4 +41,4 @@ pod2daemon/node-driver-registrar/
/networking-calico/upper-constraints*.txt

/* Created by libcalico-go UTs */
ut-tmp-dir
ut-tmp-dir
8 changes: 4 additions & 4 deletions .semaphore/semaphore.yml

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,13 @@ kind-up: kind-build-images
$(MAKE) kind-cluster-create CALICO_API_GROUP=$(KIND_CALICO_API_GROUP)
$(MAKE) kind-deploy

## Build images, create a kind cluster with v1 CRDs, deploy Calico, and run the
## v1-to-v3 migration test.
Comment thread
caseydavenport marked this conversation as resolved.
.PHONY: kind-migration-test
kind-migration-test:
KIND_CALICO_API_GROUP=crd.projectcalico.org/v1 $(MAKE) kind-up
$(REPO_ROOT)/hack/test/kind/migration/run_test.sh

## Create a kind cluster and run all e2e tests.
e2e-test:
$(MAKE) -C e2e build
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -696,3 +696,13 @@ rules:
- update
resourceNames:
- istiod
# The operator checks DatastoreMigration CRs at startup and during reconciliation
# to determine whether a v1-to-v3 CRD migration is in progress or complete.
- apiGroups:
- migration.projectcalico.org
resources:
- datastoremigrations
verbs:
- get
- list
- watch
96 changes: 96 additions & 0 deletions hack/test/kind/migration/connectivity.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Connectivity test workloads for migration testing. A client pod continuously
# probes the server pod via a Service. After migration, we check the client's
# logs for any probe failures — there should be zero packet loss since the
# dataplane state is cached while the datastore is locked.

---
apiVersion: v1
kind: Namespace
metadata:
name: migration-test

---
apiVersion: apps/v1
kind: Deployment
metadata:
name: server
namespace: migration-test
labels:
app: migration-server
spec:
replicas: 1
selector:
matchLabels:
app: migration-server
template:
metadata:
labels:
app: migration-server
spec:
containers:
- name: server
image: busybox:1.36
command:
- /bin/sh
- -c
- |
mkdir -p /www
echo "ok" > /www/index.html
httpd -f -p 8080 -h /www
ports:
- containerPort: 8080
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 1
periodSeconds: 2

---
apiVersion: v1
kind: Service
metadata:
name: server
namespace: migration-test
spec:
selector:
app: migration-server
ports:
- port: 80
targetPort: 8080

---
apiVersion: apps/v1
kind: Deployment
metadata:
name: client
namespace: migration-test
labels:
app: migration-client
spec:
replicas: 1
selector:
matchLabels:
app: migration-client
template:
metadata:
labels:
app: migration-client
spec:
containers:
- name: client
image: busybox:1.36
command:
- /bin/sh
- -c
- |
echo "PROBE_START $(date +%s)"
while true; do
ts=$(date +%s)
if wget -q -T 2 -O /dev/null http://server.migration-test.svc.cluster.local 2>/dev/null; then
echo "PROBE_OK $ts"
else
echo "PROBE_FAIL $ts"
fi
sleep 1
done
Loading
Loading