Skip to content

[client][server] Preserve fetch bucket order to prevent starvation - #4785

Open
fxbing wants to merge 1 commit into
apache:mainfrom
fxbing:fix/20261009-fetch-bucket-fairness
Open

fxbing wants to merge 1 commit into
apache:mainfrom
fxbing:fix/20261009-fetch-bucket-fairness

Conversation

@fxbing

@fxbing fxbing commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #4784

Preserve the client's round-robin bucket order throughout fetch processing. With a small shared fetch budget, losing this order can allow continuously backlogged buckets to repeatedly consume the budget before other buckets are served.

Brief change log

  • Build per-server requests in bucket order, grouping only consecutive buckets belonging to the same table. This preserves interleaved orders such as A0 → B0 → A1.
  • Use LinkedHashMap when decoding fetch requests and filtering authorized buckets.
  • Preserve order through log-read results, delayed-fetch status creation, and delayed re-fetching.
  • Add regression coverage for request ordering and progress under a small fetch budget.

Tests

Added:

  • LogFetcherTest.testFetchRequestPreservesBucketOrderAcrossTables
  • ServerRpcMessageUtilsTest.testFetchLogRequestPreservesInterleavedBucketOrder
  • FlussAuthorizationITCase.testFetchAuthorizationPreservesBucketOrder
  • DelayedFetchLogTest.testRefetchRespectsBucketOrderWithSmallBudget
  • LogFetcherITCase.testSmallFetchBudgetDoesNotStarveInterleavedTables

The end-to-end test uses an empty bucket and two backlogged buckets from interleaved tables on the same leader. With a one-byte fetch budget, it verifies alternating progress over successive fetch rounds while both non-empty buckets remain backlogged.

Local validation completed on Java 11: 100 tests across eight focused test classes, with 98 passed and two existing disabled tests skipped. Spotless, Checkstyle, and license checks passed. The full project test suite was not run.

Mutation checks also confirmed that the regression tests fail when restoring either table-wide grouping with LinkedHashMap or unordered authorization filtering.

API and Format

No public API, protocol schema, or storage-format changes.

A request may contain multiple entries for the same table when its buckets are interleaved with another table's buckets. The existing repeated-field protocol and server decoder support this representation.

Documentation

No documentation changes.

Generative AI disclosure

Generative AI tools were used: OpenAI Codex.

Group only consecutive same-table buckets within each leader request, and preserve their order through decoding, authorization, and delayed reads.

Add regressions for interleaved tables, sustained backlog with a tiny fetch budget, ACL filtering, and delayed fetch ordering.
@fxbing fxbing closed this Oct 22, 2026
@fxbing fxbing reopened this Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[client][server] Fetch request ordering can cause bucket starvation

1 participant