You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Stop worker agents from watching cluster-wide gateway topology they don't use.
Gateway informers. The agent starts the GatewayPool, GatewayPoolNode and SiteGatewayPoolAssignment informers only while its role or its Site's assignments need them. It stops them when they no longer do.
MTU.
MTU is resolved from actual forwarding paths rather than from undirected topology components.
SiteGatewayPoolAssignment MTUs no longer apply to ordinary worker mesh links. They still apply on the gateway path to an assigned Site.
Watch timeout. The randomized Node informer watch timeout is doubled, so watches are re-established half as often. LIST requests are unchanged.
Startup race. The gateway event callback is installed before the reconcile worker starts, and events go through a synchronized lookup. Events that arrive during startup are no longer lost.
Testing
Unit tests for:
informer start and stop across role and assignment changes
registering the callback after events start arriving (add, update and delete for all three resources)
MTU along forwarding paths
the watch timeout wrapper
Tested on an isolated AKS cluster with two gateways and two remote workers:
ICMP and HTTP succeeded in both directions, 10 of 10.
Removing the remote Site from the assignment and adding it back stopped and restarted exactly the three gateway informers per remote. No agents restarted.
Demoting a gateway and promoting it again replaced only that gateway's agent. The other gateway carried traffic with no loss.
Stack
This is one of five stacked PRs that reduce API server load from unbounded-net-controller and unbounded-net-node. Kubernetes remains the source of truth. Review them bottom to top:
Motivation. A large cluster running v0.8.0 had the controller continuously writing objects that hadn't changed. Every node agent also watched the whole cluster's gateway topology. On a canary cluster, the full stack cut controller API calls by 54.4% and node agent API calls by 73.8%.
Use informer cache comparisons and normalized server defaults to avoid no-op API writes, while narrowing DaemonSet and Site event requeues.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab9b78ee-42b8-4841-8d74-cd78d002d48b
Generations are only comparable within one UID, so treat a same-name
replacement observed as an update as relevant.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab9b78ee-42b8-4841-8d74-cd78d002d48b
SiteNodeSlice membership is derived from internal IPs, so Node updates and
deletions dirty both the labeled and IP-matched Sites. Slice updates dirty
both the old and new siteName so the original Site can repair a slice that
the new Site deletes as extra.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab9b78ee-42b8-4841-8d74-cd78d002d48b
Every adjacent pool hop scans all GatewayPoolPeerings through gatewayPoolPeeringTunnelMTU (reconciliation_helpers.go:264-296). With R advertised paths, H pool hops per path, and P two-pool peerings, this adds O(RHP) work per gateway peer. CIDRs sharing paths repeat the scan, and it runs before the unchanged-state check (site_watch_reconcile.go:1895,2168-2197). Index enabled MTU minima by unordered pool pair once per reconciliation, as already done for assignment and SitePeering edges, and use that lookup here.
Avoid quadratic gateway cache scans during informer resync
This full-cache dependency check runs for every assignment update, including informer resyncs. For a worker with no matching assignment, gatewayCachesRequired parses all N objects (gateway_informer_lifecycle.go:381-388), with a JSON marshal/unmarshal for each (reconciliation_helpers.go:670-677). A resync delivering N updates therefore performs N² JSON round trips per worker outside the coalesced reconcile queue; the default resync is 600 seconds (main.go:299). Index assignments by Site, or restrict synchronous checks to events affecting the pinned Site and leave full scans to the coalesced reconcile. Preserve immediate cancellation when the last relevant assignment disappears.
Bypass unchanged-state reconciliation after a partial programming failure, and propagate gateway policy installation errors. Index enabled pool-peering MTU caps once per reconcile. Avoid synchronous assignment-cache scans for unchanged resyncs and events unrelated to the adopted Site, preserving immediate local dependency cancellation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab9b78ee-42b8-4841-8d74-cd78d002d48b
Addressed the two additional scaling findings from review 5553902702: pool-peering MTU minima are indexed by canonical unordered pool pair once per reconciliation; assignment event callbacks no longer scan the full cache for remote-Site events or unchanged informer resyncs. Local membership changes, including tombstone deletion and moves away from the adopted Site, still refresh dependency requirements synchronously. Added enabled/disabled/duplicate edge-minimum tests and event tests exercising 1,000 resyncs plus local/remote membership transitions. Node/netlink race tests and scoped lint pass.
Compute a common configured Site and mesh-peering MTU cap from existing caches. Shared pod-facing and tunnel interfaces require this constraint even without transitive routing. Exclude disabled, network-only, missing-Site and disconnected relationships; do not join components through gateway paths. Apply the cap to peers and the local fabric without adding API writes or watches, and cover the three-Site asymmetric case and upward recovery.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab9b78ee-42b8-4841-8d74-cd78d002d48b
This eagerly builds every pair for each capped SitePeering: 1,000 distinct Sites require 1,000,000 comparisons and 499,500 map entries. The caller rebuilds the index before selecting gateway peers and before the unchanged-state check (site_watch_reconcile.go:1820,2187), so even workers without gateway assignments pay this quadratic CPU and allocation cost on periodic reconciles. Skip gateway-only indexing when no gateway peers need it, and use Site-to-peering memberships with cached lookups for pairs needed by accepted gateway paths instead of expanding every pair.
Replace eager all-pairs expansion with linear Site and GatewayPool memberships and memoized lookups for pairs actually used by accepted gateway paths. Build gateway-only indexes lazily so direct-mesh workers do not pay for them. Cover 1,000-member peerings and repeated reverse-pair cache hits without quadratic storage.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab9b78ee-42b8-4841-8d74-cd78d002d48b
Fixed the previously missed quadratic Site-pair indexing finding from review 5554640820 in ff09134. SitePeering and GatewayPoolPeering now share a linear membership index with memoized pair lookups, rather than eagerly expanding every pair. Gateway-only indexes are created lazily after accepting an actual gateway/path, so workers without gateway peers skip them entirely. Tests verify 1,000 members create 1,000 memberships and zero cached pairs initially, then repeated/reversed requested-pair lookups retain one cache entry. Scoped lint and node/netlink race suites pass.
This reference omits the direct-mesh component cap applied in cmd/unbounded-net-node/site_watch_reconcile.go:2101-2112. For example, A-B=1200 and A-C=1400 also constrain C to 1200, even though C does not forward through B. Document this exception and its effect on CNI MTU, matching docs/net/configuration.md:307-315, so operators do not expect independent per-path limits.
Fixed the documentation finding from review 5554696112 in 3659bf5. The public networking configuration reference now documents the direct-mesh component cap, the A-B=1200/A-C=1400 example, the shared CNI MTU impact, excluded relationships, and that constraints propagate without adding transitive routes. It now matches docs/net/configuration.md.
If an existing gateway tunnel's MTU drifts below the committed fabric MTU, this verification fails even on the unchanged-state path (site_watch_reconcile.go:2198-2203). That path leaves dataplaneDirty false, so later reconciles skip tunnel programming again and repeatedly fail route/MTU publication instead of repairing the interface. Mark the dataplane dirty on verification failure so the next reconcile retries programming.
Retain the attempted fabric MTU independently from committed routing state and retry CNI/pod MTU writes after any incomplete reconciliation. Mark all error exits dirty, including unchanged-state verification failures. Preserve tighten-before-tunnel and tunnel-before-relax ordering through failed writes and topology reversions. Extend production-path tests for failed status publication, partial pod MTU updates, verification drift, and both reversion directions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab9b78ee-42b8-4841-8d74-cd78d002d48b
Also fixed the previously missed MTU-verification recovery finding from review 5554744809 in 4b04cae: all error exits, including verification failure on the unchanged-state path, mark dataplaneDirty. The regression test injects MTU drift, verifies no status PATCH occurs, and confirms the next reconcile actually reprograms the dataplane rather than repeating the no-op path.
Preserve scoped gateway informers and forwarding MTU handling while integrating the parent's startup fixes and automatic neighbor sizing. Align neighbor-sizing regression coverage with the existing failed-reconciliation retry fence.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6b70e00f-eec0-4611-97c6-1b6200092b62
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stop worker agents from watching cluster-wide gateway topology they don't use.
Testing
Stack
This is one of five stacked PRs that reduce API server load from unbounded-net-controller and unbounded-net-node. Kubernetes remains the source of truth. Review them bottom to top:
Motivation. A large cluster running v0.8.0 had the controller continuously writing objects that hadn't changed. Every node agent also watched the whole cluster's gateway topology. On a canary cluster, the full stack cut controller API calls by 54.4% and node agent API calls by 73.8%.