Skip to content

Add native TileRT P/D orchestration - #10

Closed
cquil11 wants to merge 13 commits into
agent/atom-engine-atomeshfrom
agent/tilert-pd-runtime
Closed

cquil11 wants to merge 13 commits into
agent/atom-engine-atomeshfrom
agent/tilert-pd-runtime

Conversation

@cquil11

@cquil11 cquil11 commented Aug 27, 2026 •

Copy link
Copy Markdown

Summary

  • add a native TileRT backend for vLLM prefill plus TileRT decode
  • add the official tilert.pd_vllm.pd_router frontend with exact static P/D topology
  • support role-specific worker images without shell-level Slurm orchestration
  • materialize converted decode weights once under a shared lock, with atomic publication and cache reuse
  • provide an idempotent role-aware runtime setup for the published TileRT and NIXL versions
  • validate the single-router/single-prefill topology and managed command ownership

Stack

This PR is stacked directly on #7, which is stacked on #1. PR #3 remains separate.

Current validation

  • 244 focused schema/frontend/backend tests pass
  • targeted ty type checking passes
  • ruff, formatting, Bash syntax, and diff checks pass
  • draft while the InferenceX AgentX port is completed and exercised on B200 infrastructure

@cquil11
cquil11 force-pushed the agent/tilert-pd-runtime branch from 0eddbb7 to 4244140 Compare August 27, 2026 06:08
@cquil11
cquil11 force-pushed the agent/tilert-pd-runtime branch from 2da56a7 to 5821eb2 Compare August 27, 2026 12:06
@cquil11

cquil11 commented Aug 27, 2026

Copy link
Copy Markdown
Author

Closing because the project scope is now limited to porting active, nondeprecated AMD multi-node configurations to srt-slurm. This PR is outside that scope.

@cquil11 cquil11 closed this Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant