remove Model field from LLMRequest

[nirrozenbaum]

This PR removes the Model field from Scheduling.LLMRequest struct.
from the scheduler point of view, it doesn't care about the original requested model name, only about the resolved model after traffic splitting that was done in a higher level.
we can see that Model field was used in unit-tests only and that it's always set to be identical to ResolvedTargetModel. Scheduler doesn't use this field.

In addition to removing Model field from LLMRequest struct, this PR renames ResolvedTargetModel to TargetModel in LLMRequest from the same reasons. scheduler plugins don't care about traffic splitting and "resolved" model, only about what is target model.

unit-tests and other usage places were updated accordingly.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nirrozenbaum
Once this PR has been reviewed and has the lgtm label, please assign ahg-g for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[nirrozenbaum]

cc @kfswain

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	4313476
🔍 Latest deploy log	https://app.netlify.com/sites/gateway-api-inference-extension/deploys/6818557264dd2500086f8779
😎 Deploy Preview	https://deploy-preview-782--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[LukeAVanDrie]

I am working on the Flow Controller POC. We currently enforce the dispatch policy (for fairness) by model, not resolved target model. I will either need to use a different type when enqueuing a request than LLMRequest or will need model to remain as a field on this struct.

The Flow Controller is not tightly coupled to the Scheduler, so either approach is fine.

[nirrozenbaum]

I am working on the Flow Controller POC. We currently enforce the dispatch policy (for fairness) by model, not resolved target model. I will either need to use a different type when enqueuing a request than LLMRequest or will need model to remain as a field on this struct.

The Flow Controller is not tightly coupled to the Scheduler, so either approach is fine.

@LukeAVanDrie thanks for bringing this up.
can you elaborate on what other fields you have in Flow Controller? (do you have a link to your git branch?)
I mean, do you have a subset of the fields that are used in both? do you have additional fields that are not used in Scheduler and you need in Flow Control?

Ideally, each layer should get the data it needs for its mission, and only that.
if there is a complete match (with the exception of Model field) then sure, let's leave it as is.
but I suspect you might need only parts of the fields or additional fields

[kfswain]

but I suspect you might need only parts of the fields or additional fields

To my understanding, @LukeAVanDrie needs the Model name, as that is the unique identifier we currently use to separate one 'use case' from another.

Ideally, each layer should get the data it needs for its mission, and only that.
Completely agreed here. That is the endstate

[kfswain]

I'd like to hold off on any changes to this file until: #781 merges, I do quite a bit of this same refactoring

[nirrozenbaum]

/hold until #781 is merged

[LukeAVanDrie]

@LukeAVanDrie thanks for bringing this up. can you elaborate on what other fields you have in Flow Controller? (do you have a link to your git branch?) I mean, do you have a subset of the fields that are used in both? do you have additional fields that are not used in Scheduler and you need in Flow Control?

A subset (model, criticality) and some new fields: a reference to the request prompt size in bytes and a reference to the request context.

Ideally, each layer should get the data it needs for its mission, and only that. if there is a complete match (with the exception of Model field) then sure, let's leave it as is. but I suspect you might need only parts of the fields or additional fields

This is a good point. I have removed my dependency on scheduling.LLMRequest and am defining my own type for the FlowController input. My POC's fate is no longer tied to this PR.