Scheduler::schedule() breaks out of the packing loop on the first group that exceeds the remaining token budget. When the highest-priority group's num_seqs is greater than max_tokens_per_batch, it breaks on the very first iteration and returns an empty ScheduleBatch - so that group never makes forward progress (starvation/deadlock), even though it is the only runnable work.
Expected: schedule() should never return an empty batch while a runnable group exists; the highest-priority group should be admitted even if it alone overflows the soft token budget.
No test currently covers the scheduler.
Scheduler::schedule()breaks out of the packing loop on the first group that exceeds the remaining token budget. When the highest-priority group'snum_seqsis greater thanmax_tokens_per_batch, it breaks on the very first iteration and returns an emptyScheduleBatch- so that group never makes forward progress (starvation/deadlock), even though it is the only runnable work.Expected:
schedule()should never return an empty batch while a runnable group exists; the highest-priority group should be admitted even if it alone overflows the soft token budget.No test currently covers the scheduler.