Repository navigation
Support ADK for RL training #641
Description
Is your feature request related to a problem? Please describe.
I would like to use repo to train an agent
Describe the solution you'd like
ADK seems to be the right way for agentic framework. It will be nice to support ADK during GRPO training.
Describe alternatives you've considered
Alternative approaches will inevitably introduce gap between training and inference/serving, so they are not ideal.
Activity
snowmanwwg commentedon Jul 21, 2025
Hi Jialei, my brief read on ADK tells me that it is actually orthogonal to what Nemo RL does - Nemo RL is a training FW that outputs a model that has reasoning capabilities. That model can be used with ADK to build and deploy as an agent. As long as ADK is model checkpoint agnostic, it should work. Let me know if my understanding is correct.
Alternatively, what do you see as the "to-dos" for Nemo RL to support ADK?
jialei777 commentedon Jul 21, 2025
Thank you @snowmanwwg for the feedback. My understanding is ADK is becoming the standard way to deploy model as agent. While we can train model with Nemo RL and then deploy to ADK, there is gap as the rollout in nemo rl is different from the inference time with ADK (e.g., different way of determine stop criteria for multi-turn convo). So quality might be sub-optimal when the usecase is with ADK. This is the main motivation for the request.
As the todo items, I am thinking a 2-step procedure.
for turn in range(max_turn):loop) will be replaced by ADK logic (exactly the same as in inference time) and ADK returns the whole conversation for reward computation.I understand the potential huge code change and re-design of the repo structure. That is why I want to point this out and start discussion as early as possible.