Repository navigation
feat(proxy): opt-in latency-aware upstream selection - #194
Merged
Merged
Conversation
Container image availableMulti-arch image (amd64 + arm64): podman pull ttl.sh/extractedprism:pr-194-1d
|
Latency-aware upstream selection needs to expose the smoothed connect time it ranks upstreams by and the tier each upstream is grouped into. Both series are removed together with the other per-upstream series when an upstream is removed. Signed-off-by: Aleksei Sviridkin <f@lex.la> Assisted-by: LLM
Control plane nodes can sit in different sites, and uniform random picking sends a share of new connections to an upstream tens of milliseconds away while one under a millisecond away is available. With latency selection enabled, successful health check dials feed their connect time into a per-upstream EWMA. A sample is capped at four times the current average, so one retransmitted SYN moves the average by a bounded step instead of sending a near upstream into a far tier for minutes, and a sample below a quarter of the average replaces it, so a slow first sample is undone by the next normal one. Sampled upstreams are grouped into tiers by latency alone: a tier holds every remaining upstream whose average is at most max(best+2ms, 2*best), best being the lowest remaining average. A hysteresis band keeps a member near the boundary from flapping; it follows the group rather than the tier number, so tiers appearing or disappearing above do not split a group. Because health does not move upstreams between tiers, the tiers change only with new samples and with the endpoint set. New connections are picked at random among the pickable upstreams of the lowest tier that has one, plus pickable upstreams without a sample, so unhealthy or draining members of a tier do not let a lower tier in, and a tier without a pickable member hands over to the next one. Random selection keeps its pick semantics and no latency state. proxy.New now takes the config by pointer: the new field pushed the struct over gocritic's hugeParam limit. Signed-off-by: Aleksei Sviridkin <f@lex.la> Assisted-by: LLM
Expose latency-aware upstream selection as an opt-in mode. The default stays random; any value other than random or latency fails validation. Closes: #190 Signed-off-by: Aleksei Sviridkin <f@lex.la> Assisted-by: LLM
Prose in this repository is written one line per paragraph and left to the renderer to wrap; hard-wrapping it breaks narrow viewports and list rendering. MD013 flags every such paragraph, so it is turned off rather than satisfied. The README opens with a centered logo block, which needs inline <p> and <img> and cannot start with a heading: MD033 allows those two elements and MD041 is off. Signed-off-by: Aleksei Sviridkin <f@lex.la> Assisted-by: LLM
lexfrei
force-pushed
the
feat/latency-selection
branch
from
October 9, 2026 16:18
e412972 to
d2d7d8e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This adds an opt-in mode that sends new connections to the closest control plane endpoints. It is enabled with
--upstream-selection=latency(envEP_UPSTREAM_SELECTION). The default staysrandomand picks exactly as before.In latency mode, the connect time of every successful health check goes into a per-upstream EWMA. Upstreams are grouped into tiers by that average. A tier holds every remaining upstream whose average is at most
max(best + 2ms, 2 × best). A new connection goes to a random healthy, non-draining upstream from the lowest tier that has one. Upstreams with no sample yet are picked together with that tier.Why
Control plane nodes can sit in different countries. With random picking, a node that has an API server a fraction of a millisecond away still sends some of its new connections to one 50-100 ms away.
I use tiers and not "always the fastest" on purpose. In a single-site cluster all upstreams land in one tier, so it behaves like
randomand every node does not pile onto the same API server.Behaviour worth knowing
extractedprism_upstream_rtt_secondsandextractedprism_upstream_latency_tier. They exist only in latency mode and are deleted together with the upstream.Closes #190
Notes
proxy.Newnow takes*Config, because the new field pushed the struct over gocritic'shugeParamlimit. The flag is wired through a newproxyConfighelper ininternal/server..markdownlint.yamlcomes in a separate chore commit. README prose is one line per paragraph, and the README opens with a centered logo block. markdownlint does not run in CI.lexfrei/chartsPR after this is released.