Skip to content

Simplify backfill window to always return full projection - #68

Merged
JaeseungYeom merged 1 commit into
mainfrom
exp/multiclusters
Sep 30, 2026
Merged

JaeseungYeom merged 1 commit into
mainfrom
exp/multiclusters

Conversation

@JaeseungYeom

Copy link
Copy Markdown
Collaborator
  • Remove the full_horizon option across the C++, Python, and gRPC APIs.
  • Always include projected releases for all running jobs and update the dispatcher, tests, and documentation accordingly.

Changes in the current worktree:

  • Scheduler API: adds on-demand waiting_resource_area() and shared prediction_horizon() logic.
  • Existing FCFS implementations: scan their existing queue entries only when prediction is requested.
  • Backfill window: always returns all projected running-job releases; removes full_horizon.
  • gRPC/Python: exposes prediction-horizon queries.
  • Dispatcher: adds performance-aware multi-system routing.
  • Tests/CMake: adds dispatcher, standard-FCFS, Python, and gRPC coverage.
  • Documentation: describes the new APIs and dispatch workflow.

Runtime qualifications:

  • Normal scheduling has no new ongoing bookkeeping.
  • Prediction queries cost roughly O(Q + R log R) with temporary O(R) storage.
  • get_backfill_window() also constructs its release map only when queried.
  • FCFS-alt now writes its reservation into the already-existing m_fcfs_reservation_time field instead of a local variable. That is a correctness fix, not new state.

- Remove the full_horizon option across the C++, Python, and gRPC APIs.
- Always include projected releases for all running jobs and update the
  dispatcher, tests, and documentation accordingly.

Changes in the current worktree:
  - Scheduler API: adds on-demand waiting_resource_area() and shared prediction_horizon() logic.
  - Existing FCFS implementations: scan their existing queue entries only when prediction is requested.
  - Backfill window: always returns all projected running-job releases; removes full_horizon.
  - gRPC/Python: exposes prediction-horizon queries.
  - Dispatcher: adds performance-aware multi-system routing.
  - Tests/CMake: adds dispatcher, standard-FCFS, Python, and gRPC coverage.
  - Documentation: describes the new APIs and dispatch workflow.

Runtime qualifications:
  - Normal scheduling has no new ongoing bookkeeping.
  - Prediction queries cost roughly O(Q + R log R) with temporary O(R) storage.
  - get_backfill_window() also constructs its release map only when queried.
  - FCFS-alt now writes its reservation into the already-existing m_fcfs_reservation_time field instead of a local variable. That is a correctness fix, not new state.
@JaeseungYeom JaeseungYeom self-assigned this Sep 30, 2026
@JaeseungYeom JaeseungYeom added the enhancement New feature or request label Sep 30, 2026
@JaeseungYeom
JaeseungYeom merged commit b1334a0 into main Sep 30, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant