refactor: move backend deploy, launch and slurm files from components to examples #3849
Conversation
Signed-off-by: Anant Sharma <[email protected]>
Signed-off-by: Anant Sharma <[email protected]>
Signed-off-by: Anant Sharma <[email protected]>
Signed-off-by: Anant Sharma <[email protected]>
Signed-off-by: Anant Sharma <[email protected]>
Signed-off-by: Anant Sharma <[email protected]>
… to examples Signed-off-by: Anant Sharma <[email protected]>
Signed-off-by: Anant Sharma <[email protected]>
Signed-off-by: Anant Sharma <[email protected]>
|
/ok to test b1da6c4 |
Signed-off-by: Anant Sharma <[email protected]>
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
|
/ok to test e2655ed |
WalkthroughUpdates across the repository replace backend path references from Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Poem
Pre-merge checks❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (4)
examples/backends/trtllm/performance_sweeps/README.md (1)
52-52: Improve link text descriptiveness.The link text should be more descriptive than "here" to comply with markdown best practices. Replace generic "here" links with descriptive anchor text that indicates the link destination.
-2. Container image of Dynamo with TensorRT-LLM built using instructions from [here](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/trtllm#build-docker). +2. Container image of Dynamo with TensorRT-LLM built using instructions from [TensorRT-LLM Docker build guide](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/trtllm#build-docker).-# NOTE: To build an iamge, see the steps here: -# https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/trtllm#build-docker +# NOTE: To build an image, see the [TensorRT-LLM Docker build guide](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/trtllm#build-docker).Also note: "iamge" on line 71 should be "image".
Also applies to: 72-72
examples/deployments/ECS/task_definition_prefillworker.json (1)
18-18: Use absolute path for consistency and clarity.The ECS task definition uses a relative path
cd examples/backends/vllm, while all Kubernetes manifests and other deployment files consistently use the absolute path/workspace/examples/backends/vllm. This inconsistency could cause issues if the container's working directory differs from assumptions.For consistency and to avoid potential runtime failures, use the absolute path:
- "cd examples/backends/vllm && python3 -m dynamo.vllm --model Qwen/Qwen3-0.6B --enforce-eager --is-prefill-worker" + "cd /workspace/examples/backends/vllm && python3 -m dynamo.vllm --model Qwen/Qwen3-0.6B --enforce-eager --is-prefill-worker"docs/reference/cli.md (2)
296-296: Minor: Add hyphen for compound adjective.Line 296 uses "production grade" which should be hyphenated as "production-grade" when used as a compound adjective before "engines".
Apply this diff to improve style:
-`vllm`, `sglang` and `trtllm` production grade engines are available in `examples/backends`. +`vllm`, `sglang` and `trtllm` production-grade engines are available in `examples/backends`.
323-323: Minor: Simplify phrasing for clarity.Line 323 could be more concise by removing the redundant "of".
Apply this diff to improve readability:
-All of the main backend components in `examples/backends/` work like this. +All main backend components in `examples/backends/` work like this.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (104)
.github/filters.yaml(1 hunks).github/workflows/container-validation-backends.yml(1 hunks)README.md(1 hunks)benchmarks/README.md(1 hunks)benchmarks/profiler/utils/config_modifiers/sglang.py(1 hunks)benchmarks/profiler/utils/config_modifiers/trtllm.py(1 hunks)benchmarks/profiler/utils/config_modifiers/vllm.py(1 hunks)components/README.md(1 hunks)components/src/dynamo/router/README.md(1 hunks)container/Dockerfile.sglang-wideep(1 hunks)deploy/helm/README.md(2 hunks)deploy/inference-gateway/README.md(2 hunks)deploy/tracing/README.md(3 hunks)docs/backends/sglang/README.md(5 hunks)docs/backends/sglang/dsr1-wideep-h100.md(1 hunks)docs/backends/sglang/multimodal_epd.md(2 hunks)docs/backends/trtllm/README.md(6 hunks)docs/backends/trtllm/gemma3_sliding_window_attention.md(4 hunks)docs/backends/trtllm/gpt-oss.md(1 hunks)docs/backends/trtllm/llama4_plus_eagle.md(1 hunks)docs/backends/trtllm/multimodal_epd.md(1 hunks)docs/backends/trtllm/multinode/multinode-examples.md(2 hunks)docs/backends/trtllm/multinode/multinode-multimodal-example.md(1 hunks)docs/backends/vllm/LMCache_Integration.md(2 hunks)docs/backends/vllm/README.md(3 hunks)docs/benchmarks/benchmarking.md(2 hunks)docs/benchmarks/kv-router-ab-testing.md(2 hunks)docs/benchmarks/pre_deployment_profiling.md(3 hunks)docs/design_docs/distributed_runtime.md(2 hunks)docs/design_docs/dynamo_flow.md(1 hunks)docs/development/backend-guide.md(1 hunks)docs/kubernetes/README.md(2 hunks)docs/kubernetes/deployment/create_deployment.md(4 hunks)docs/kubernetes/deployment/multinode-deployment.md(1 hunks)docs/kubernetes/installation_guide.md(1 hunks)docs/kubernetes/observability/logging.md(1 hunks)docs/kubernetes/observability/metrics.md(1 hunks)docs/kvbm/vllm-setup.md(1 hunks)docs/performance/aiconfigurator.md(1 hunks)docs/planner/sla_planner_quickstart.md(2 hunks)docs/reference/cli.md(3 hunks)docs/router/README.md(1 hunks)examples/README.md(1 hunks)examples/backends/sglang/deploy/README.md(1 hunks)examples/backends/sglang/deploy/agg.yaml(1 hunks)examples/backends/sglang/deploy/agg_logging.yaml(1 hunks)examples/backends/sglang/deploy/agg_router.yaml(1 hunks)examples/backends/sglang/deploy/disagg-multinode.yaml(2 hunks)examples/backends/sglang/deploy/disagg.yaml(2 hunks)examples/backends/sglang/deploy/disagg_planner.yaml(2 hunks)examples/backends/trtllm/deploy/README.md(2 hunks)examples/backends/trtllm/deploy/agg-with-config.yaml(1 hunks)examples/backends/trtllm/deploy/disagg-multinode.yaml(1 hunks)examples/backends/trtllm/deploy/disagg_planner.yaml(1 hunks)examples/backends/trtllm/performance_sweeps/README.md(2 hunks)examples/backends/vllm/deploy/README.md(2 hunks)examples/backends/vllm/deploy/agg.yaml(1 hunks)examples/backends/vllm/deploy/agg_kvbm.yaml(1 hunks)examples/backends/vllm/deploy/agg_router.yaml(1 hunks)examples/backends/vllm/deploy/disagg-multinode.yaml(3 hunks)examples/backends/vllm/deploy/disagg.yaml(2 hunks)examples/backends/vllm/deploy/disagg_kvbm.yaml(2 hunks)examples/backends/vllm/deploy/disagg_kvbm_2p2d.yaml(2 hunks)examples/backends/vllm/deploy/disagg_kvbm_tp2.yaml(2 hunks)examples/backends/vllm/deploy/disagg_planner.yaml(2 hunks)examples/backends/vllm/deploy/disagg_router.yaml(2 hunks)examples/basics/kubernetes/Distributed_Inference/README.md(1 hunks)examples/basics/kubernetes/Distributed_Inference/agg_router.yaml(1 hunks)examples/basics/kubernetes/Distributed_Inference/disagg_router.yaml(2 hunks)examples/basics/kubernetes/shared_frontend/shared_frontend.yaml(1 hunks)examples/basics/multinode/trtllm/srun_aggregated.sh(1 hunks)examples/basics/multinode/trtllm/srun_disaggregated.sh(1 hunks)examples/deployments/ECS/README.md(1 hunks)examples/deployments/ECS/task_definition_frontend.json(1 hunks)examples/deployments/ECS/task_definition_prefillworker.json(1 hunks)examples/deployments/EKS/Deploy_Dynamo_Cloud.md(1 hunks)examples/deployments/EKS/Deploy_VLLM_example.md(1 hunks)examples/deployments/GKE/sglang/disagg.yaml(2 hunks)examples/deployments/GKE/vllm/disagg.yaml(2 hunks)launch/dynamo-run/src/main.rs(1 hunks)lib/bindings/python/examples/metrics/README.md(2 hunks)pyproject.toml(1 hunks)recipes/deepseek-r1/sglang-wideep/tep16p-dep16d-disagg.yaml(2 hunks)recipes/deepseek-r1/sglang-wideep/tep8p-dep8d-disagg.yaml(2 hunks)recipes/gpt-oss-120b/trtllm/agg/deploy.yaml(1 hunks)recipes/llama-3-70b/vllm/agg/deploy.yaml(2 hunks)recipes/llama-3-70b/vllm/disagg-multi-node/deploy.yaml(3 hunks)recipes/llama-3-70b/vllm/disagg-single-node/deploy.yaml(3 hunks)tests/fault_tolerance/deploy/scenarios.py(1 hunks)tests/fault_tolerance/deploy/templates/vllm/moe_agg.yaml(1 hunks)tests/fault_tolerance/deploy/templates/vllm/moe_disagg.yaml(2 hunks)tests/planner/README.md(2 hunks)tests/planner/perf_test_configs/agg_8b.yaml(2 hunks)tests/planner/perf_test_configs/disagg_8b_2p2d.yaml(3 hunks)tests/planner/perf_test_configs/disagg_8b_3p1d.yaml(3 hunks)tests/planner/perf_test_configs/disagg_8b_planner.yaml(3 hunks)tests/planner/perf_test_configs/disagg_8b_tp2.yaml(3 hunks)tests/planner/profiling_results/H200_TP1P_TP1D/disagg.yaml(3 hunks)tests/profiler/test_profile_sla_aiconfigurator.py(1 hunks)tests/profiler/test_profile_sla_dryrun.py(3 hunks)tests/serve/test_sglang.py(1 hunks)tests/serve/test_trtllm.py(1 hunks)tests/serve/test_vllm.py(1 hunks)tests/utils/managed_deployment.py(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/reference/cli.md
[grammar] ~296-~296: Use a hyphen to join words.
Context: ...vllm, sglang and trtllm production grade engines are available in `examples...
(QB_NEW_EN_HYPHEN)
[style] ~323-~323: Consider removing “of” to be more concise
Context: ...our own engine and attach it to Dynamo. All of the main backend components in `examples/ba...
(ALL_OF_THE)
docs/backends/sglang/README.md
[grammar] ~229-~229: Use a hyphen to join words.
Context: ...h ### Disaggregated Serving with KV Aware Prefill Routing bash cd $DYNAM...
(QB_NEW_EN_HYPHEN)
deploy/tracing/README.md
[uncategorized] ~182-~182: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...on tracing environment variables at the top level and service-specific names in each comp...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🪛 markdownlint-cli2 (0.18.1)
examples/backends/trtllm/performance_sweeps/README.md
52-52: Link text should be descriptive
(MD059, descriptive-link-text)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: tests (launch/dynamo-run)
- GitHub Check: tests (.)
- GitHub Check: Build and Test - dynamo
🔇 Additional comments (108)
docs/kvbm/vllm-setup.md (1)
42-43: Documentation path updates align with backend restructuring.The changes correctly redirect users to the new
examples/backends/vllmlocation for launch scripts (agg_kvbm.sh, disagg_kvbm.sh, disagg_kvbm_2p2d.sh). The updates are consistent across all three occurrences and properly use$DYNAMO_HOMEfor portability.Please verify that the launch scripts referenced in this documentation exist at the new
examples/backends/vllm/launch/directory paths and that users following this guide will successfully locate them. If any of these scripts have been renamed, moved to subdirectories, or deprecated, the documentation paths should be adjusted accordingly.Also applies to: 50-51, 55-56
recipes/gpt-oss-120b/trtllm/agg/deploy.yaml (1)
88-88: Path migration looks correct; verify target directory exists in deployment.The workingDir update from
/workspace/components/backends/trtllmto/workspace/examples/backends/trtllmaligns with the PR's restructuring objectives. This change is appropriately isolated to a single line and the YAML syntax is valid.Please verify that:
- The target directory
/workspace/examples/backends/trtllmexists in the container image or is mounted/created during deployment.- Any initialization scripts or tools that depend on this working directory path have been updated consistently across the repository.
To help confirm this, you may want to check if similar deployment files (e.g., other backend deployment YAMLs in
recipes/orexamples/backends/) have been updated with the same path migration.docs/backends/vllm/LMCache_Integration.md (1)
46-46: Path references correctly updated to reflect directory restructuring.The two launch script paths have been properly updated from
components/backends/vllm/launch/toexamples/backends/vllm/launch/, aligning with the PR's objective to move backend deployment configurations out of the components directory.Please confirm that the referenced launch scripts (
agg_lmcache.shanddisagg_lmcache.sh) exist at their new location inexamples/backends/vllm/launch/to ensure the documentation paths remain functional.Also applies to: 72-72
docs/kubernetes/deployment/multinode-deployment.md (1)
284-286: Documentation paths correctly updated to reflect new backend deployment structure.The links have been updated consistently across all three backends with proper relative path syntax. The relative paths correctly navigate from the nested documentation directory (three levels up to repository root) and then into the new examples structure.
examples/basics/multinode/trtllm/srun_disaggregated.sh (1)
36-42: Correctly updated error message URL to reflect new backend location.The error message now points to the new
examples/backends/trtllmdocumentation path, consistent with the PR objective to move backends out of thecomponentsdirectory. The URL format and anchor reference (#build-docker) are correct.recipes/llama-3-70b/vllm/disagg-multi-node/deploy.yaml (1)
22-22: Path refactoring is comprehensive and consistently applied across all deployment files.Verification confirms that all four
deploy.yamlfiles in the repository have been updated with the new/workspace/examples/backends/path pattern. The oldcomponents/backendspath no longer exists anywhere in the codebase. The three updates in the file under review (lines 22, 46, 75) are part of a complete and consistent refactoring with no dangling references or inconsistencies.examples/backends/vllm/deploy/disagg_kvbm_2p2d.yaml (1)
28-28: Path migration to examples/backends looks good.The workingDir updates in both worker services are consistent and correctly reflect the new examples directory structure. The path will serve as the execution context for the vLLM backend module.
Also applies to: 60-60
examples/backends/vllm/deploy/disagg_kvbm.yaml (1)
28-28: Path migration is consistent with other vLLM variants.Both worker services correctly reference the new examples/backends path.
Also applies to: 60-60
examples/backends/vllm/deploy/agg_kvbm.yaml (1)
35-35: Path migration applied correctly to aggregated deployment.Aggregated architecture variant correctly updated to use the new examples/backends path.
docs/performance/aiconfigurator.md (1)
105-105: Documentation example correctly updated.The Fast Profiling Integration example now references the correct examples/backends path for trtllm, consistent with the cross-framework migration.
docs/kubernetes/observability/metrics.md (1)
57-57: Deployment instruction updated to reflect new path.The pushd command now correctly references examples/backends/vllm/deploy, enabling users to apply the Kubernetes manifests from the new location.
examples/basics/kubernetes/shared_frontend/shared_frontend.yaml (1)
49-49: vllm-agg workingDir correctly updated.The vllm-agg worker service now references the new examples/backends/vllm path. Other services in the same manifest correctly retain their own paths (multimodal workflows), which is appropriate for their distinct purposes.
examples/backends/sglang/deploy/disagg.yaml (1)
29-29: SGLang backend path migration is consistent.Both decode and prefill worker services correctly reference the new examples/backends/sglang path, confirming the migration spans multiple backend frameworks systematically.
Also applies to: 65-65
docs/design_docs/dynamo_flow.md (1)
20-20: Documentation reference updated to new path.The architecture flow diagram reference now correctly points to examples/backends/vllm, allowing readers to locate the reference implementation.
examples/basics/kubernetes/Distributed_Inference/disagg_router.yaml (1)
39-39: LGTM.Path updates to workingDir are consistent and correctly reflect the migration from components to examples directory structure.
Also applies to: 67-67
recipes/llama-3-70b/vllm/disagg-single-node/deploy.yaml (1)
22-22: LGTM.All three workingDir updates are consistent and correctly migrate to the examples/backends path structure.
Also applies to: 56-56, 95-95
examples/backends/sglang/deploy/agg_router.yaml (1)
31-31: LGTM.SGLang workingDir path correctly updated to examples/backends directory.
tests/planner/perf_test_configs/disagg_8b_3p1d.yaml (1)
42-42: LGTM.All workingDir paths correctly updated for performance test configuration. Consistent with production deployment manifests.
Also applies to: 92-92, 142-142
docs/benchmarks/benchmarking.md (1)
103-103: LGTM.Documentation links correctly updated to point to examples/backends. Both references use consistent path structure.
Also applies to: 335-335
docs/development/backend-guide.md (1)
80-80: LGTM.Documentation reference correctly updated to point to examples/backends for code examples.
examples/backends/vllm/deploy/disagg_router.yaml (1)
31-31: LGTM.vLLM workingDir paths consistently updated to examples/backends structure in both container specs.
Also applies to: 50-50
examples/backends/sglang/deploy/agg.yaml (1)
28-28: LGTM.SGLang workingDir path correctly updated to examples/backends directory.
examples/backends/sglang/deploy/README.md (1)
65-65: Path update is accurate and consistent.The workingDir path correctly reflects the new location under examples/backends/sglang.
docs/benchmarks/kv-router-ab-testing.md (1)
166-166: Path updates maintain parity between benchmark configurations.Both router-off and router-on deployments now correctly reference the same backend path under examples/backends/vllm, ensuring fair A/B comparison as intended by the benchmark guide.
Also applies to: 237-237
examples/backends/vllm/deploy/README.md (1)
73-73: Path updates are accurate in both example configuration and deployment instructions.Line 73 updates the container workingDir example, and line 144 corrects the deployment directory path in the shell command. Both changes ensure users follow the new examples/backends structure.
Also applies to: 144-144
docs/kubernetes/observability/logging.md (1)
129-129: Deployment path correctly updated for logging guide.The kubectl apply command now correctly references the logging deployment from examples/backends/sglang, ensuring users can follow this observability guide without path errors.
container/Dockerfile.sglang-wideep (1)
90-90: Container WORKDIR correctly updated for new backend location.The Dockerfile now sets the correct working directory in the sglang-wideep runtime image to reflect the examples/backends/sglang structure.
deploy/helm/README.md (1)
36-36: All Helm command examples updated consistently to new path structure.The three installation variants (basic, Grove, and customizable) now all reference examples/backends/vllm/deploy/agg.yaml, providing accurate and consistent documentation for users deploying with Helm.
Also applies to: 44-44, 53-53
examples/backends/sglang/deploy/agg_logging.yaml (1)
31-31: Kubernetes deployment manifest workingDir correctly updated.The agg_logging.yaml manifest now specifies the correct working directory for the SGLang worker pod runtime, reflecting the new examples/backends/sglang location.
tests/planner/perf_test_configs/agg_8b.yaml (1)
42-42: Test configuration paths updated consistently for both Frontend and Worker services.Both mainContainer workingDir entries in the performance test config now correctly reference examples/backends/vllm, ensuring the planner tests execute with the correct backend paths.
Also applies to: 92-92
examples/backends/vllm/deploy/disagg-multinode.yaml (1)
17-17: Path migration looks good.All three workingDir fields consistently updated from
/workspace/components/backends/vllmto/workspace/examples/backends/vllmacross Frontend, decode, and prefill services. No logic changes.Also applies to: 38-38, 61-61
examples/basics/kubernetes/Distributed_Inference/agg_router.yaml (1)
42-42: Path migration is correct.Updated workingDir from
/workspace/components/backends/vllmto/workspace/examples/backends/vllmfor the VllmDecodeWorker service.examples/backends/vllm/deploy/agg_router.yaml (1)
31-31: Path migration is consistent.Updated workingDir from
/workspace/components/backends/vllmto/workspace/examples/backends/vllmfor the VllmDecodeWorker service.recipes/llama-3-70b/vllm/agg/deploy.yaml (1)
22-22: Path migration is consistent.Both workingDir fields updated from
/workspace/components/backends/vllmto/workspace/examples/backends/vllmacross Frontend and VllmPrefillWorker services.Also applies to: 46-46
recipes/deepseek-r1/sglang-wideep/tep8p-dep8d-disagg.yaml (1)
49-49: Path migration is consistent.Both workingDir fields updated from
/workspace/components/backends/sglangto/workspace/examples/backends/sglangacross decode and prefill services.Also applies to: 98-98
tests/planner/perf_test_configs/disagg_8b_2p2d.yaml (1)
42-42: Path migration is consistent across test config.All three workingDir fields consistently updated from
/workspace/components/backends/vllmto/workspace/examples/backends/vllmacross Frontend, VllmDecodeWorker, and VllmPrefillWorker services in the test configuration.Also applies to: 92-92, 142-142
examples/backends/trtllm/deploy/agg-with-config.yaml (1)
54-54: Path migration is correct and complete for this file.The workingDir has been properly updated to point to the examples directory structure as intended by this refactor.
examples/backends/trtllm/deploy/disagg_planner.yaml (1)
20-20: Frontend workingDir correctly updated to examples backend path.The Frontend service workingDir has been updated appropriately. The Planner and worker components correctly retain their original paths (/workspace/components/src for Planner, /workspace/ for workers) as they serve different roles and are not backend-specific.
examples/backends/sglang/deploy/disagg-multinode.yaml (1)
39-39: Both decode and prefill worker workingDirs correctly updated.The workingDir paths for both the decode and prefill services have been consistently updated to the examples backend structure.
Also applies to: 76-76
examples/backends/trtllm/deploy/disagg-multinode.yaml (1)
99-99: Frontend workingDir correctly updated; worker paths appropriately unchanged.The Frontend service workingDir has been updated to reference the examples backend path. The decode and prefill worker services correctly remain at /workspace/ as they reference relative paths via their arguments.
examples/backends/vllm/deploy/disagg.yaml (1)
29-29: Both decode and prefill worker workingDirs correctly updated.The workingDir paths for both VllmDecodeWorker and VllmPrefillWorker services have been consistently migrated to the examples backend structure.
Also applies to: 49-49
examples/basics/kubernetes/Distributed_Inference/README.md (2)
22-22: Documentation reference correctly updated to examples path.The link to the deployment example has been properly updated to point to the examples backend directory.
26-27: Documentation enhancements provide clearer deployment guidance.The added bullets documenting replica configuration and debug flag environment variable help users understand the customizations available in the referenced deployment example.
docs/design_docs/distributed_runtime.md (2)
31-31: Deployment example references correctly updated to examples backend path.The references to deployment configurations (agg.yaml, agg_router.yaml) have been properly updated to point to the examples directory structure.
78-78: Python bindings reference appropriately retained at components/src.The reference to
components/src/dynamofor Python implementation details remains correct, as this documents the actual source code location rather than external examples.examples/deployments/GKE/vllm/disagg.yaml (1)
31-31: Both decode and prefill worker workingDirs correctly updated in GKE variant.The workingDir paths for both VllmDecodeWorker and VllmPrefillWorker services in the GKE deployment have been consistently migrated to the examples backend structure.
Also applies to: 53-53
launch/dynamo-run/src/main.rs (1)
95-99: LGTM! Error message path updated correctly.The error message now correctly directs users to the new
examples/backends/directory location for Python interface examples.tests/planner/profiling_results/H200_TP1P_TP1D/disagg.yaml (1)
42-42: LGTM! Working directory paths updated consistently.All three container workingDir paths have been correctly updated to reference the new
examples/backends/vllmlocation.Also applies to: 92-92, 142-142
examples/deployments/ECS/task_definition_frontend.json (1)
26-26: LGTM! Command path updated correctly.The working directory in the container command has been correctly updated to the new
examples/backends/vllmlocation.examples/backends/vllm/deploy/disagg_kvbm_tp2.yaml (1)
30-30: LGTM! Working directory paths updated consistently.Both worker containers now correctly reference the new
examples/backends/vllmpath.Also applies to: 64-64
examples/backends/vllm/deploy/disagg_planner.yaml (1)
52-52: LGTM! Worker paths updated correctly.Both vLLM worker containers now correctly reference the new
examples/backends/vllmlocation. The Planner component correctly remains incomponents/src/dynamo/planneras it's not part of the backends being moved.Also applies to: 72-72
docs/backends/trtllm/gpt-oss.md (1)
131-131: LGTM! Documentation paths updated correctly.The documentation now correctly instructs users to navigate to
examples/backends/trtllmfor both the launch script (Option A) and manual launch (Option B) approaches.Also applies to: 139-139
docs/backends/trtllm/llama4_plus_eagle.md (1)
42-42: LGTM! Setup path updated correctly.The setup instructions now correctly direct users to
examples/backends/trtllmfor the deployment configuration.docs/backends/sglang/multimodal_epd.md (1)
50-50: LGTM! Documentation paths updated correctly.Both the aggregated and disaggregated serving examples now correctly reference the new
examples/backends/sglanglocation.Also applies to: 136-136
examples/deployments/EKS/Deploy_Dynamo_Cloud.md (1)
89-89: Shell prompt path updated to reflect new examples/backends location.The kubectl command output now shows the correct working directory path (
examples/backends/vllm/deploy). Verification confirms no remaining references to the old path exist in the file.pyproject.toml (1)
154-154: Test ignore glob correctly updated for SGLang slurm_jobs relocation.The pytest
--ignore-globpattern now properly referencesexamples/backends/sglang/slurm_jobs/*, ensuring test discovery excludes these files from their new location. Verification confirms the directory exists and no stale references to the old path remain.examples/deployments/EKS/Deploy_VLLM_example.md (1)
6-6: Deployment paths correctly updated for vLLM example.Both the navigation command (line 6) and shell output prompt (line 14) now reflect the
examples/backends/vllm/deploylocation, ensuring consistency for users following the deployment guide. Verification confirms the directory and deployment files exist.examples/backends/trtllm/deploy/README.md (1)
93-93: Path references correctly updated in deployment guide.Both the workingDir example (line 93) and deployment navigation command (line 171) now reference the correct
examples/backends/trtllmpaths. Verification confirms no remaining references to the oldcomponents/backends/trtllmstructure exist in this file.docs/kubernetes/deployment/create_deployment.md (1)
3-4: Deployment documentation paths verified and correctly migrated.All referenced files exist in their new locations, documentation at the specified lines (3, 4, 28, 45, 64, 72, 75, 76) contains the correct
examples/backends/paths, and no oldcomponents/backends/references remain. The migration is complete and consistent.examples/README.md (1)
37-40: Documentation links correctly updated to point to Examples Backends.The framework support section now accurately references the
examples/backends/directory for vLLM, SGLang, and TensorRT-LLM specific deployment patterns. All three backend directories exist with proper subdirectories for deployment, launch configurations, and framework-specific optimizations.recipes/deepseek-r1/sglang-wideep/tep16p-dep16d-disagg.yaml (1)
51-51: Path verification complete—no issues found.The directory
examples/backends/sglang/exists in the repository with proper structure. Both lines 51 and 104 are correctly and consistently updated to/workspace/examples/backends/sglang. The directory contains the necessary launch scripts and deployment configurations that will execute from this working directory when the container starts.docs/kubernetes/installation_guide.md (1)
236-236: Documentation paths correctly migrated to examples directory.Path updates are consistent and properly formatted for the new directory structure.
Also applies to: 244-246
docs/benchmarks/pre_deployment_profiling.md (1)
45-45: Profiling configuration paths consistently updated across documentation and examples.All default config path references now point to examples/backends directory.
Also applies to: 134-134, 294-294
docs/reference/cli.md (1)
399-399: Path updates to examples/backends are correct and consistent.Documentation references to Python engine examples are properly updated.
docs/planner/sla_planner_quickstart.md (1)
71-71: SLA Planner guide documentation paths correctly updated to examples directory.Both inject_manifest command and error example reference the new examples/backends path.
Also applies to: 246-246
docs/backends/trtllm/gemma3_sliding_window_attention.md (1)
30-30: Deployment guide working directory paths consistently updated across all TensorRT-LLM configurations.All cd commands correctly reference examples/backends/trtllm with proper launch script references maintained.
Also applies to: 39-39, 48-48, 58-58
deploy/inference-gateway/README.md (1)
88-88: Inference Gateway deployment guide paths correctly updated with proper relative path structure.References to model deployment guide and working directories are consistently updated and maintain correct path traversal from deployment location.
Also applies to: 93-93, 119-119
docs/backends/trtllm/README.md (1)
131-131: TensorRT-LLM backend documentation comprehensively updated with consistent path migration.All deployment mode examples, Kubernetes guide links, test references, and performance sweep documentation correctly reference examples/backends directory structure.
Also applies to: 137-137, 147-147, 157-157, 163-163, 189-189, 273-273, 319-319
docs/backends/trtllm/multimodal_epd.md (1)
24-24: EPD multimodal documentation working directory path correctly updated to examples directory.Path update is consistent with other TensorRT-LLM backend documentation changes.
examples/deployments/ECS/README.md (1)
87-87: ✓ Path updates are consistent and correct.The Docker command path changes from
cd components/backends/vllmtocd examples/backends/vllmmaintain the proper relative structure for container execution. Both the Frontend/Decoding Worker and PrefillWorker tasks are updated consistently.Also applies to: 93-93
deploy/tracing/README.md (1)
94-94: ✓ Path updates are consistent and properly formatted.All path references have been updated consistently from
components/backends/vllmtoexamples/backends/vllm. Both the shell script directory navigation (line 94) and Kubernetes deployment references (lines 182, 231) are correct.Also applies to: 182-182, 231-231
docs/kubernetes/README.md (1)
70-72: ✓ Documentation links and deployment paths are correctly updated.All relative paths (lines 70-72) from
docs/kubernetes/to example deployment files are properly calculated. The kubectl apply command (line 87) correctly references the new location.Also applies to: 87-87
.github/filters.yaml (1)
31-31: ✓ GitHub Actions filter globs are correctly updated.All three backend filter globs have been consistently updated to point to
examples/backends/<backend>/**. The glob syntax is valid and will properly trigger CI workflows when files in these directories change.Also applies to: 38-38, 45-45
README.md (1)
174-174: ✓ Main README link is correctly updated.The "Backends" link in the "Deploying Dynamo" section now points to the correct
examples/backendsdirectory.docs/backends/trtllm/multinode/multinode-examples.md (1)
51-51: ✓ Documentation links to TensorRT-LLM README are correct.Both references now point to the
#build-containeranchor in the TensorRT-LLM README, which is the appropriate section for container build instructions.Also applies to: 90-90
tests/fault_tolerance/deploy/templates/vllm/moe_disagg.yaml (1)
55-55: ✓ Container working directory paths are correctly updated.Both VllmDecodeWorker and VllmPrefillWorker have their
workingDirproperly updated to/workspace/examples/backends/vllm. This maintains consistency across fault-tolerance test templates.Also applies to: 120-120
benchmarks/README.md (1)
23-23: ✓ Benchmarks README now links to appropriate deployment documentation.The Quick Start section correctly directs users to the Kubernetes deployment guide for deployment instructions before benchmarking.
components/src/dynamo/router/README.md (1)
50-50: LGTM! Documentation link updated correctly.The link now points to the new location of the launch script in the
examples/backendsdirectory, consistent with the repository reorganization..github/workflows/container-validation-backends.yml (1)
451-451: LGTM! Path updated in commented code for future consistency.The path update in this commented-out deployment test maintains consistency with the repository-wide path migration, even though this code is currently disabled.
tests/fault_tolerance/deploy/templates/vllm/moe_agg.yaml (1)
52-52: LGTM! Container working directory updated correctly.The
workingDirfield now points to the new location in the container filesystem, consistent with the backend files migration toexamples/backends.examples/backends/vllm/deploy/agg.yaml (1)
28-28: LGTM! Deployment working directory updated correctly.The working directory path has been updated to reflect the new location of backend files in the
examplesdirectory.docs/backends/sglang/dsr1-wideep-h100.md (1)
47-47: LGTM! Documentation updated with correct container path.The container directory reference has been updated to reflect the new location. The
/sgl-workspaceprefix is specific to the SGLang WideEP container setup.tests/planner/perf_test_configs/disagg_8b_planner.yaml (1)
45-45: LGTM! All container working directories updated consistently.All three service containers (Frontend, VllmDecodeWorker, and VllmPrefillWorker) now correctly reference the new backend location in
examples/backends.Also applies to: 142-142, 199-199
tests/profiler/test_profile_sla_aiconfigurator.py (1)
31-31: LGTM! Test fixture configuration path updated correctly.The test fixture now points to the deployment configuration in the new
examples/backendslocation, ensuring tests use the correct files.tests/serve/test_trtllm.py (1)
32-34: LGTM! Test directory path updated correctly.The default trtllm directory now points to the new location in
examples/backends, while still allowing override via theTRTLLM_DIRenvironment variable.examples/deployments/GKE/sglang/disagg.yaml (1)
28-28: LGTM! Consistent path migration for SGLang workingDir.The workingDir paths have been correctly updated from
components/backends/sglangtoexamples/backends/sglangfor both decode and prefill workers, aligning with the repository restructuring.Also applies to: 51-51
tests/serve/test_vllm.py (1)
33-35: LGTM! Default vLLM directory path correctly updated.The default vLLM directory has been updated to use the new
examples/backends/vllmpath structure, maintaining consistency with the repository reorganization.docs/backends/trtllm/multinode/multinode-multimodal-example.md (1)
55-55: LGTM! Documentation link correctly updated.The build container reference has been updated to point to the appropriate documentation location in
docs/backends/trtllm/README.md.docs/router/README.md (1)
63-65: LGTM! K8s example links correctly migrated.All three router example links have been consistently updated to reference the new
examples/backends/directory structure for TRT-LLM, vLLM, and SGLang.benchmarks/profiler/utils/config_modifiers/trtllm.py (1)
40-40: LGTM! Default TRT-LLM config path correctly updated.The default configuration path has been updated to reference the new
examples/backends/trtllm/directory structure.tests/utils/managed_deployment.py (1)
825-827: LGTM! Example deployment spec path correctly updated.The example DeploymentSpec path in the main function has been updated to reference the new
examples/backends/vllm/directory structure.tests/planner/perf_test_configs/disagg_8b_tp2.yaml (1)
42-42: LGTM! All workingDir paths consistently migrated.The workingDir has been correctly updated from
components/backends/vllmtoexamples/backends/vllmacross all three service definitions (Frontend, VllmDecodeWorker, and VllmPrefillWorker).Also applies to: 92-92, 142-142
benchmarks/profiler/utils/config_modifiers/sglang.py (1)
38-38: LGTM! Path migration is correct.The default config path has been properly updated to reference the new location in the examples directory.
examples/backends/sglang/deploy/disagg_planner.yaml (2)
53-53: LGTM! Working directory path updated correctly.The decode worker's working directory has been updated to reference the new examples location.
89-89: LGTM! Consistent path update for prefill worker.The prefill worker's working directory matches the decode worker's path update.
docs/backends/sglang/README.md (2)
185-186: LGTM! Documentation paths updated consistently.All deployment script paths in the quick start section have been correctly updated to reference the examples directory.
Also applies to: 192-193, 201-202, 225-226, 232-233, 242-243
288-288: LGTM! Deployment guide references updated.The Kubernetes and SLURM deployment guide links now correctly point to the examples directory.
Also applies to: 291-291
benchmarks/profiler/utils/config_modifiers/vllm.py (1)
36-36: LGTM! vLLM config path updated consistently.The default config path has been updated to match the new examples directory structure, consistent with the sglang and trtllm changes.
docs/backends/vllm/README.md (2)
109-110: LGTM! Deployment script paths updated consistently.All deployment examples now correctly reference the examples directory structure.
Also applies to: 117-118, 125-126, 133-134, 143-144
156-156: LGTM! Kubernetes deployment guide link updated.The deployment guide reference now points to the correct location in the examples directory.
tests/planner/README.md (2)
13-13: LGTM! Backend README reference updated.The setup instructions now correctly point to the examples directory for backend-specific READMEs.
173-179: LGTM! Deployment manifest path updated.The test preparation instructions now reference the correct location of the disagg_planner.yaml file in the examples directory.
tests/profiler/test_profile_sla_dryrun.py (1)
49-49: Config files verified at all new paths.All three
disagg.yamlfiles have been confirmed to exist at their updated locations:
examples/backends/vllm/deploy/disagg.yaml✓examples/backends/sglang/deploy/disagg.yaml✓examples/backends/trtllm/deploy/disagg.yaml✓The test fixture updates are complete and consistent across all backends.
tests/fault_tolerance/deploy/scenarios.py (1)
180-181: Deployment YAML files verified at new paths.All agg.yaml and disagg.yaml files exist for vllm, sglang, and trtllm backends at
examples/backends/{backend}/deploy/.components/README.md (3)
32-32: No issues found—path and referenced content are correct.The relative path
../examples/backends/correctly resolves fromcomponents/README.mdto the repository root'sexamples/backends/directory. Each engine (sglang, trtllm, vllm) contains verifiedlaunchanddeploysubdirectories with deployment scripts (agg.sh, disagg.sh, etc.).
28-30: No changes needed—documentation links are valid and resolve correctly.The review comment raised concerns about absolute path resolution for the documentation links, but verification confirms all three target files exist at their specified locations:
docs/backends/vllm/README.md,docs/backends/sglang/README.md, anddocs/backends/trtllm/README.md. GitHub markdown natively supports root-relative paths (those starting with/), which is a standard and widely-adopted convention in repository documentation. The links will resolve correctly in all supported contexts including GitHub web UI, static documentation generators, and GitHub-aware markdown processors.Likely an incorrect or invalid review comment.
26-32: Review comment verified as addressing valid concerns with positive results.The consolidation successfully preserves necessary context:
Documentation links verified: All three engines have README files at
/docs/backends/vllm/README.md,/docs/backends/sglang/README.md, and/docs/backends/trtllm/README.mdwith detailed capability documentation.Deployment examples verified:
examples/backends/contains launch and deploy scripts for all three engines (vLLM, SGLang, TensorRT-LLM), each withdeploy/andlaunch/subdirectories.Inline descriptions remain accurate: The consolidated summaries correctly capture each engine's key characteristics while directing readers to detailed docs and deployment examples.
Readers can access:
- Quick feature overviews in components/README.md (consolidated)
- Detailed backend capabilities via linked /docs/backends/ READMEs
- Deployment instructions and scripts via ../examples/backends/
tests/serve/test_sglang.py (1)
35-35: Path update is correct and all required scripts are present.The change properly updates the SGLang directory from the old
components/location to the newexamples/location. The test framework automatically appends thelaunch/subdirectory when constructing script paths, and all required scripts (agg.sh, disagg.sh, agg_router.sh, multimodal_agg.sh, agg_embed.sh) are present at./examples/backends/sglang/launch/. The old components path no longer exists, confirming the migration is complete.
Signed-off-by: Anant Sharma <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
examples/basics/multinode/trtllm/srun_aggregated.sh (1)
35-35: Duplicate: The URL anchor#build-dockerremains broken.This critical issue was already flagged in the previous review. The URL still references a non-existent anchor in
examples/backends/trtllm#build-dockerbecause no README.md exists in that directory. The actual build documentation is indocs/backends/trtllm/README.md.Please verify whether a README.md was added to
examples/backends/trtllm/as part of this PR restructuring, or update the URL to point to the correct documentation location:#!/bin/bash # Verify if README.md now exists in the new examples/backends/trtllm/ location ls -la examples/backends/trtllm/README.md 2>/dev/null && echo "README.md exists" || echo "README.md does not exist" # Check the build section anchors in docs/backends/trtllm/README.md grep -n "^##.*[Bb]uild" docs/backends/trtllm/README.md 2>/dev/null || echo "No build sections found"If no README.md was added, either:
- Create
examples/backends/trtllm/README.mdwith a# Build Dockersection, or- Update the URL to:
https://github.com/ai-dynamo/dynamo/tree/main/docs/backends/trtllm#build-container(or the correct anchor from docs/backends/trtllm/README.md)
🧹 Nitpick comments (1)
examples/deployments/EKS/Deploy_VLLM_example.md (1)
5-9: Add language specifier to fenced code block.The shell script block should explicitly specify the language for proper syntax highlighting.
Apply this diff to add the language identifier:
-``` +```bash cd dynamo/examples/backends/vllm/deploy
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (104)
.github/filters.yaml(1 hunks).github/workflows/container-validation-backends.yml(1 hunks)README.md(1 hunks)benchmarks/README.md(1 hunks)benchmarks/profiler/utils/config_modifiers/sglang.py(1 hunks)benchmarks/profiler/utils/config_modifiers/trtllm.py(1 hunks)benchmarks/profiler/utils/config_modifiers/vllm.py(1 hunks)components/README.md(1 hunks)components/src/dynamo/router/README.md(1 hunks)container/Dockerfile.sglang-wideep(1 hunks)deploy/helm/README.md(2 hunks)deploy/inference-gateway/README.md(2 hunks)deploy/tracing/README.md(3 hunks)docs/backends/sglang/README.md(5 hunks)docs/backends/sglang/dsr1-wideep-h100.md(1 hunks)docs/backends/sglang/multimodal_epd.md(2 hunks)docs/backends/trtllm/README.md(6 hunks)docs/backends/trtllm/gemma3_sliding_window_attention.md(4 hunks)docs/backends/trtllm/gpt-oss.md(1 hunks)docs/backends/trtllm/llama4_plus_eagle.md(1 hunks)docs/backends/trtllm/multimodal_epd.md(1 hunks)docs/backends/trtllm/multinode/multinode-examples.md(2 hunks)docs/backends/trtllm/multinode/multinode-multimodal-example.md(1 hunks)docs/backends/vllm/LMCache_Integration.md(2 hunks)docs/backends/vllm/README.md(3 hunks)docs/benchmarks/benchmarking.md(2 hunks)docs/benchmarks/kv-router-ab-testing.md(2 hunks)docs/benchmarks/pre_deployment_profiling.md(3 hunks)docs/design_docs/distributed_runtime.md(2 hunks)docs/design_docs/dynamo_flow.md(1 hunks)docs/development/backend-guide.md(1 hunks)docs/kubernetes/README.md(2 hunks)docs/kubernetes/deployment/create_deployment.md(4 hunks)docs/kubernetes/deployment/multinode-deployment.md(1 hunks)docs/kubernetes/installation_guide.md(1 hunks)docs/kubernetes/observability/logging.md(1 hunks)docs/kubernetes/observability/metrics.md(1 hunks)docs/kvbm/vllm-setup.md(1 hunks)docs/performance/aiconfigurator.md(1 hunks)docs/planner/sla_planner_quickstart.md(2 hunks)docs/reference/cli.md(3 hunks)docs/router/README.md(1 hunks)examples/README.md(1 hunks)examples/backends/sglang/deploy/README.md(1 hunks)examples/backends/sglang/deploy/agg.yaml(1 hunks)examples/backends/sglang/deploy/agg_logging.yaml(1 hunks)examples/backends/sglang/deploy/agg_router.yaml(1 hunks)examples/backends/sglang/deploy/disagg-multinode.yaml(2 hunks)examples/backends/sglang/deploy/disagg.yaml(2 hunks)examples/backends/sglang/deploy/disagg_planner.yaml(2 hunks)examples/backends/trtllm/deploy/README.md(2 hunks)examples/backends/trtllm/deploy/agg-with-config.yaml(1 hunks)examples/backends/trtllm/deploy/disagg-multinode.yaml(1 hunks)examples/backends/trtllm/deploy/disagg_planner.yaml(1 hunks)examples/backends/trtllm/performance_sweeps/README.md(2 hunks)examples/backends/vllm/deploy/README.md(2 hunks)examples/backends/vllm/deploy/agg.yaml(1 hunks)examples/backends/vllm/deploy/agg_kvbm.yaml(1 hunks)examples/backends/vllm/deploy/agg_router.yaml(1 hunks)examples/backends/vllm/deploy/disagg-multinode.yaml(3 hunks)examples/backends/vllm/deploy/disagg.yaml(2 hunks)examples/backends/vllm/deploy/disagg_kvbm.yaml(2 hunks)examples/backends/vllm/deploy/disagg_kvbm_2p2d.yaml(2 hunks)examples/backends/vllm/deploy/disagg_kvbm_tp2.yaml(2 hunks)examples/backends/vllm/deploy/disagg_planner.yaml(2 hunks)examples/backends/vllm/deploy/disagg_router.yaml(2 hunks)examples/basics/kubernetes/Distributed_Inference/README.md(1 hunks)examples/basics/kubernetes/Distributed_Inference/agg_router.yaml(1 hunks)examples/basics/kubernetes/Distributed_Inference/disagg_router.yaml(2 hunks)examples/basics/kubernetes/shared_frontend/shared_frontend.yaml(1 hunks)examples/basics/multinode/trtllm/srun_aggregated.sh(1 hunks)examples/basics/multinode/trtllm/srun_disaggregated.sh(1 hunks)examples/deployments/ECS/README.md(1 hunks)examples/deployments/ECS/task_definition_frontend.json(1 hunks)examples/deployments/ECS/task_definition_prefillworker.json(1 hunks)examples/deployments/EKS/Deploy_Dynamo_Cloud.md(1 hunks)examples/deployments/EKS/Deploy_VLLM_example.md(1 hunks)examples/deployments/GKE/sglang/disagg.yaml(2 hunks)examples/deployments/GKE/vllm/disagg.yaml(2 hunks)launch/dynamo-run/src/main.rs(1 hunks)lib/bindings/python/examples/metrics/README.md(2 hunks)pyproject.toml(1 hunks)recipes/deepseek-r1/sglang-wideep/tep16p-dep16d-disagg.yaml(2 hunks)recipes/deepseek-r1/sglang-wideep/tep8p-dep8d-disagg.yaml(2 hunks)recipes/gpt-oss-120b/trtllm/agg/deploy.yaml(1 hunks)recipes/llama-3-70b/vllm/agg/deploy.yaml(2 hunks)recipes/llama-3-70b/vllm/disagg-multi-node/deploy.yaml(3 hunks)recipes/llama-3-70b/vllm/disagg-single-node/deploy.yaml(3 hunks)tests/fault_tolerance/deploy/scenarios.py(1 hunks)tests/fault_tolerance/deploy/templates/vllm/moe_agg.yaml(1 hunks)tests/fault_tolerance/deploy/templates/vllm/moe_disagg.yaml(2 hunks)tests/planner/README.md(2 hunks)tests/planner/perf_test_configs/agg_8b.yaml(2 hunks)tests/planner/perf_test_configs/disagg_8b_2p2d.yaml(3 hunks)tests/planner/perf_test_configs/disagg_8b_3p1d.yaml(3 hunks)tests/planner/perf_test_configs/disagg_8b_planner.yaml(3 hunks)tests/planner/perf_test_configs/disagg_8b_tp2.yaml(3 hunks)tests/planner/profiling_results/H200_TP1P_TP1D/disagg.yaml(3 hunks)tests/profiler/test_profile_sla_aiconfigurator.py(1 hunks)tests/profiler/test_profile_sla_dryrun.py(3 hunks)tests/serve/test_sglang.py(1 hunks)tests/serve/test_trtllm.py(1 hunks)tests/serve/test_vllm.py(1 hunks)tests/utils/managed_deployment.py(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/reference/cli.md
[grammar] ~296-~296: Use a hyphen to join words.
Context: ...vllm, sglang and trtllm production grade engines are available in `examples...
(QB_NEW_EN_HYPHEN)
[style] ~323-~323: Consider removing “of” to be more concise
Context: ...our own engine and attach it to Dynamo. All of the main backend components in `examples/ba...
(ALL_OF_THE)
deploy/tracing/README.md
[uncategorized] ~182-~182: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...on tracing environment variables at the top level and service-specific names in each comp...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
docs/backends/sglang/README.md
[grammar] ~229-~229: Use a hyphen to join words.
Context: ...h ### Disaggregated Serving with KV Aware Prefill Routing bash cd $DYNAM...
(QB_NEW_EN_HYPHEN)
🪛 markdownlint-cli2 (0.18.1)
examples/deployments/EKS/Deploy_VLLM_example.md
13-13: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: trtllm (arm64)
- GitHub Check: sglang
- GitHub Check: trtllm (amd64)
- GitHub Check: vllm (arm64)
- GitHub Check: vllm (amd64)
- GitHub Check: Build and Test - dynamo
🔇 Additional comments (121)
examples/basics/multinode/trtllm/srun_disaggregated.sh (1)
36-42: ✓ Error message URL correctly updated to reflect backend path restructuring.The documentation link in the error message has been properly updated from
components/backends/trtllmtoexamples/backends/trtllm, aligning with the PR's objective to move backend files to the examples directory. The message remains clear and helpful for users who encounter the missingIMAGEenvironment variable.docs/development/backend-guide.md (1)
80-80: Path reference update looks good.The change correctly updates the documentation reference from
components/backendstoexamples/backends, aligning with the PR's refactoring objectives to move backend files from components to examples.Please verify that the
examples/backendsdirectory is properly populated with the full code examples referenced in this documentation, and that this is the sole reference to the old path location in this file.recipes/deepseek-r1/sglang-wideep/tep8p-dep8d-disagg.yaml (1)
49-49: Path migrations are consistent and aligned with PR objectives.Both
workingDirpaths in the decode and prefill service specs are correctly updated from/workspace/components/backends/sglangto/workspace/examples/backends/sglang, maintaining consistency across the manifest. No YAML structure or configuration logic is altered.Please verify that the target working directory
/workspace/examples/backends/sglangexists and contains the sglang backend code in the deployed environment where this manifest is applied.Also applies to: 98-98
examples/basics/kubernetes/Distributed_Inference/disagg_router.yaml (1)
39-39: Paths updated correctly for backend migration.Both workingDir references have been consistently updated from
components/backends/vllmtoexamples/backends/vllm, aligning with the repository refactoring objectives. The YAML syntax remains valid.Verify that the corresponding backend files now exist at
/workspace/examples/backends/vllmand that no other deployment manifests or configuration files in this directory reference the oldcomponents/backendspaths.Also applies to: 67-67
recipes/llama-3-70b/vllm/disagg-multi-node/deploy.yaml (1)
22-22: Path updates are consistent and correctly aligned with the refactoring objective.All three
workingDirfields have been updated from/workspace/components/backends/vllmto/workspace/examples/backends/vllmacross the Frontend, VllmPrefillWorker, and VllmDecodeWorker services. The YAML structure and syntax remain valid.Please confirm that the backend files have been moved to
/workspace/examples/backends/vllmand that this deployment manifest will correctly locate the required backend code at runtime. You may wish to verify this by checking that all corresponding backend files exist in the target examples directory as part of the same PR changes.Also applies to: 46-46, 75-75
recipes/llama-3-70b/vllm/agg/deploy.yaml (2)
22-22: Path migration looks correct.The workingDir reference has been properly updated from the old
componentspath to the newexamplespath, aligning with the backend restructuring outlined in the PR.
46-46: Path migration consistent across both services.The VllmPrefillWorker workingDir has been updated to match the Frontend service, maintaining consistency. Both services now reference the consolidated
examples/backends/vllmlocation.Please verify that any scripts or executables referenced in this deployment (e.g., entry points, model loading scripts) are present at the new
/workspace/examples/backends/vllmpath and that the working directory change does not affect the execution of the commands specified in lines 40–44.launch/dynamo-run/src/main.rs (1)
97-97: LGTM! Path reference updated correctly.The error message now correctly points users to
examples/backends/instead ofcomponents/backends/, which aligns with the directory restructuring objectives of this PR.recipes/gpt-oss-120b/trtllm/agg/deploy.yaml (1)
88-88: workingDir path update verified and correct.The new
/workspace/examples/backends/trtllmdirectory exists with all expected files including deployment manifests (agg.yaml, disagg.yaml, etc.), launch scripts, and performance sweep utilities. The oldcomponents/backends/trtllmpath has been properly removed. The path update is correct.examples/deployments/EKS/Deploy_Dynamo_Cloud.md (1)
89-89: Path update is correct and complete.The prompt path at line 89 has been properly updated to
examples/backends/vllm/deploy, reflecting the refactoring of backend files fromcomponents/toexamples/. Verification confirms this is the only backend path reference in the file and no oldcomponents/backendspaths remain.recipes/llama-3-70b/vllm/disagg-single-node/deploy.yaml (1)
22-22: Path migration verified and complete.All three
workingDirupdates are consistent and properly aligned with the repository's refactoring:
- Target
examples/backends/vllmdirectory exists with expected structure (deploy/andlaunch/subdirectories)- Old
components/backends/vllmpath has been removed- No stale references to the old path exist anywhere in the codebase
- Changes affect only path configuration with no logic modifications
examples/basics/kubernetes/shared_frontend/shared_frontend.yaml (1)
47-54: ✓ Backend path refactored correctly.The workingDir change from
components/backends/vllmtoexamples/backends/vllm(line 49) is valid. The directory exists and contains the deployment configs and launch scripts. Thepython3 -m dynamo.vllmmodule is installed in the container imagenvcr.io/nvidia/ai-dynamo/vllm-runtime:0.5.0, not sourced from the working directory, so the path restructuring has no impact on module availability.docs/kubernetes/observability/metrics.md (1)
57-57: Documentation path update is correct.The command now points users to the correct backend deployment location. This aligns with the directory restructuring and is consistent with changes across other documentation files.
benchmarks/README.md (1)
23-23: Verify the link destination is intentional.Line 23 changes the Quick Start link from
../components/backends/to../docs/kubernetes/. While other files in the PR update toexamples/backends, this link points to documentation instead. Please confirm if directing users to Kubernetes deployment docs is the intended behavior, or if it should point to../examples/backendsfor consistency with the rest of the refactoring.README.md (1)
174-174: Link reference correctly updated to match new backend location.The link now correctly directs to the
examples/backendsdirectory where deployment configurations have been moved.docs/kubernetes/observability/logging.md (1)
129-129: Deployment configuration path correctly updated.The kubectl apply command now references the correct path where SGLang deployment configurations have been relocated.
examples/backends/vllm/deploy/agg_router.yaml (1)
31-31: Container working directory correctly updated for aggregated router deployment.The VllmDecodeWorker now references the examples backend path consistently with other deployment configurations.
tests/planner/perf_test_configs/agg_8b.yaml (1)
42-42: Test configuration paths updated to match actual deployment locations.Both Frontend and VllmDecodeWorker containers in this performance test configuration now use the updated examples backend path, ensuring test realism.
Also applies to: 92-92
examples/backends/vllm/deploy/disagg-multinode.yaml (1)
17-17: All container working directories successfully updated and verified.Verification confirms no old paths remain in the file. Frontend, decode, and prefill workers all correctly reference
/workspace/examples/backends/vllm.examples/backends/vllm/deploy/disagg_router.yaml (1)
31-31: Working directory paths consistently updated across all deployment manifests.The path changes from
/workspace/components/backends/vllmto/workspace/examples/backends/vllmare applied consistently throughout the YAML file with no remaining references to the old location. The code changes are correct and complete.examples/backends/vllm/deploy/disagg_kvbm_2p2d.yaml (1)
28-28: Path migration applied correctly.Both
workingDirreferences have been updated fromcomponents/backends/vllmtoexamples/backends/vllm, aligning with the PR's refactoring objective.Also applies to: 60-60
tests/fault_tolerance/deploy/templates/vllm/moe_agg.yaml (1)
52-52: Test template path updated consistently.The fault-tolerance test template correctly uses the new
examples/backends/vllmpath for consistency with actual deployments.container/Dockerfile.sglang-wideep (1)
90-90: WORKDIR updated for examples path.The Dockerfile's working directory has been updated to point to the new
examples/backends/sglanglocation.Verify that SGLang backend files and scripts exist at the new
examples/backends/sglang/location in this repository, as the Dockerfile'sWORKDIRdirective relies on these files being present.examples/deployments/GKE/vllm/disagg.yaml (1)
31-31: GKE deployment paths updated consistently.Both decode and prefill worker
workingDirpaths have been correctly migrated toexamples/backends/vllm.Also applies to: 53-53
docs/performance/aiconfigurator.md (1)
105-105: Documentation example path updated correctly.The bash example in the Fast Profiling Integration section now correctly references the new
examples/backends/trtllm/deploy/disagg.yamlpath.tests/planner/perf_test_configs/disagg_8b_2p2d.yaml (1)
42-42: Test config paths updated comprehensively.All three service types (Frontend, VllmDecodeWorker, VllmPrefillWorker) have their
workingDirreferences correctly updated toexamples/backends/vllm.Also applies to: 92-92, 142-142
docs/backends/vllm/LMCache_Integration.md (1)
46-46: Documentation launch scripts paths updated.Both aggregated and disaggregated launch script paths have been updated from
components/backends/vllm/launch/toexamples/backends/vllm/launch/.Verify that the launch scripts (
agg_lmcache.shanddisagg_lmcache.sh) exist at the new paths inexamples/backends/vllm/launch/so users can follow the documentation.Also applies to: 72-72
examples/basics/kubernetes/Distributed_Inference/agg_router.yaml (1)
42-42: LGTM: Path updated correctly.The working directory has been successfully migrated from the components path to the examples path, consistent with the repository restructuring.
docs/kubernetes/deployment/multinode-deployment.md (1)
284-286: LGTM: Documentation links updated consistently.All three backend reference links have been correctly updated to point to the new examples directory structure.
examples/deployments/GKE/sglang/disagg.yaml (2)
28-28: LGTM: Decode worker path updated correctly.The working directory for the decode worker has been properly migrated to the examples path.
51-51: LGTM: Prefill worker path updated correctly.The working directory for the prefill worker has been properly migrated to the examples path, maintaining consistency with the decode worker.
examples/backends/trtllm/performance_sweeps/README.md (2)
52-52: LGTM: Documentation URL updated correctly.The build documentation reference has been updated to point to the new examples path.
72-72: LGTM: Documentation URL updated consistently.The second build documentation reference has been correctly updated to match the repository restructuring.
docs/benchmarks/benchmarking.md (2)
103-103: LGTM: Deployment documentation link updated correctly.The reference to backend deployment documentation has been properly updated to point to the examples directory.
335-335: LGTM: Deployment documentation link updated consistently.The second deployment documentation reference has been correctly updated to match the repository restructuring.
benchmarks/profiler/utils/config_modifiers/trtllm.py (1)
40-40: Config file verified; all checks passed.The TensorRT-LLM configuration file exists at
examples/backends/trtllm/deploy/disagg.yaml. The constant at line 40 is correctly defined and properly used byload_default_config()at line 46. All three backend config modifiers (trtllm, vllm, sglang) have been consistently updated to point to their new example locations, and all corresponding YAML files are present in the repository. No runtime failures will occur.tests/utils/managed_deployment.py (1)
825-827: This review comment is incorrect.The file referenced (
agg.yaml) does exist in the repository atexamples/backends/vllm/deploy/agg.yaml. Additionally, themain()function in this file (lines 814-849) is not a pytest test—it's a standalone script meant to be executed directly viaif __name__ == "__main__", not through the test runner. The hardcoded/workspace/path assumes a Docker/container environment where the repository is mounted at that location, which is a runtime environment detail rather than a missing file issue.If this script is intended to be portable across different environments (local development, CI/CD, containers), consider using relative paths or environment variables instead of hardcoded absolute paths.
Likely an incorrect or invalid review comment.
components/src/dynamo/router/README.md (1)
50-50: LGTM! Documentation reference updated correctly.The path reference has been consistently updated to reflect the new
examples/backends/vllmlocation.examples/backends/vllm/deploy/disagg_kvbm_tp2.yaml (1)
30-30: LGTM! Working directory paths updated consistently.Both decode and prefill worker containers now use the correct
examples/backends/vllmworking directory.Also applies to: 64-64
examples/backends/vllm/deploy/disagg_kvbm.yaml (1)
28-28: LGTM! Container paths updated correctly.The working directory paths have been updated consistently for both worker types.
Also applies to: 60-60
tests/serve/test_sglang.py (1)
35-35: LGTM! Test configuration updated to new path.The default SGLANG_DIR fallback now correctly points to the
examples/backends/sglangdirectory.recipes/deepseek-r1/sglang-wideep/tep16p-dep16d-disagg.yaml (1)
51-51: LGTM! Working directories updated for both worker types.The path migration to
examples/backends/sglanghas been applied consistently to decode and prefill containers.Also applies to: 104-104
examples/backends/vllm/deploy/disagg.yaml (1)
29-29: LGTM! Path refactoring applied correctly.Both worker containers now reference the updated
examples/backends/vllmdirectory.Also applies to: 49-49
pyproject.toml (1)
154-154: LGTM! Pytest configuration updated for new directory structure.The ignore-glob pattern correctly reflects the relocation of slurm_jobs to the examples directory.
docs/benchmarks/kv-router-ab-testing.md (1)
166-166: LGTM! Documentation examples updated with correct paths.The YAML configuration examples in the documentation now reference the updated
examples/backends/vllmdirectory, ensuring users have accurate deployment templates.Also applies to: 237-237
examples/basics/kubernetes/Distributed_Inference/README.md (1)
22-22: Path reference correctly updated to align with examples backend structure.The documentation now correctly references the deployment example from its new location under
examples/backends/vllm/deploy/, maintaining consistency with the repository restructuring.examples/backends/trtllm/deploy/README.md (2)
93-93: Container working directory correctly updated to reflect new backend location.The
workingDirnow points to the correct path underexamples/backends/trtllmwhere the TensorRT-LLM runtime will execute.
171-171: Deployment navigation path correctly updated.The shell command now directs users to the new backend location at
dynamo/examples/backends/trtllm/deployfor applying Kubernetes manifests.docs/backends/sglang/dsr1-wideep-h100.md (1)
47-47: Working directory path correctly updated for SGLang backend location.The documentation now correctly indicates that containers should be in the new
examples/backends/sglangdirectory, ensuring users deploy from the right location.docs/backends/sglang/multimodal_epd.md (2)
50-50: Multimodal aggregated serving path correctly updated.The deployment instruction now references the correct backend path for launching the aggregated multimodal pipeline.
136-136: Multimodal disaggregated serving path correctly updated.The deployment instruction now references the correct backend path for launching the disaggregated multimodal pipeline.
examples/deployments/EKS/Deploy_VLLM_example.md (1)
6-6: Deployment directory path correctly updated to examples location.Users are now directed to the correct backend path for vLLM deployment configurations.
examples/backends/vllm/deploy/README.md (2)
73-73: vLLM container working directory correctly updated to new backend location.The
workingDirconfiguration ensures the container executes in the correct path underexamples/backends/vllm.
144-144: Deployment path correctly updated in usage instructions.Users are now directed to navigate to the new backend location at
examples/backends/vllm/deployfor applying Kubernetes manifests.tests/planner/perf_test_configs/disagg_8b_tp2.yaml (3)
42-42: Frontend service working directory updated to new backend location.The container will now execute from the correct path under
examples/backends/vllm.
92-92: vLLM decode worker working directory updated to new backend location.The decode worker container will now execute from the correct path under
examples/backends/vllm.
142-142: vLLM prefill worker working directory updated to new backend location.The prefill worker container will now execute from the correct path under
examples/backends/vllm.tests/planner/perf_test_configs/disagg_8b_3p1d.yaml (3)
42-42: Frontend service working directory updated to new backend location.The container will now execute from the correct path under
examples/backends/vllm.
92-92: vLLM decode worker working directory updated to new backend location.The decode worker container will now execute from the correct path under
examples/backends/vllm.
142-142: vLLM prefill worker working directory updated to new backend location.The prefill worker container will now execute from the correct path under
examples/backends/vllm.tests/planner/profiling_results/H200_TP1P_TP1D/disagg.yaml (1)
42-42: LGTM! Consistent path migration across all services.The
workingDirupdates consistently reflect the new directory structure for all three service definitions (Frontend, VllmDecodeWorker, VllmPrefillWorker).Also applies to: 92-92, 142-142
benchmarks/profiler/utils/config_modifiers/vllm.py (1)
36-36: LGTM! Default config path correctly updated.The constant update aligns with the repository-wide path migration from
components/backendstoexamples/backends.docs/router/README.md (1)
63-65: LGTM! Documentation links correctly updated.All three backend example links (TRT-LLM, vLLM, SGLang) consistently reference the new
examples/backendsdirectory structure.docs/reference/cli.md (1)
296-296: LGTM! Documentation references correctly updated.All references to production engine locations consistently point to the new
examples/backendsdirectory.Also applies to: 323-323, 399-399
docs/kvbm/vllm-setup.md (1)
42-42: LGTM! Working directory paths correctly updated.All KVBM setup instructions consistently reference the new
examples/backends/vllmdirectory for aggregated and disaggregated serving.Also applies to: 50-50, 55-55
docs/planner/sla_planner_quickstart.md (1)
71-71: LGTM! Configuration paths correctly updated.Both the default manifest injection path and the error message example consistently reference the new
examples/backends/vllm/deploy/disagg.yamllocation.Also applies to: 246-246
deploy/tracing/README.md (1)
94-94: LGTM! Tracing documentation paths correctly updated.All references to vLLM deployment and configuration files consistently point to the new
examples/backends/vllmdirectory structure.Also applies to: 182-182, 231-231
docs/kubernetes/deployment/create_deployment.md (1)
3-4: LGTM! Deployment guide links comprehensively updated.All references to deployment templates (agg.yaml, agg_router.yaml, disagg_router.yaml) and launch scripts consistently point to the new
examples/backends/vllmdirectory structure.Also applies to: 28-28, 45-45, 64-64, 72-72, 74-74, 76-76
examples/README.md (1)
37-40: LGTM! Path migration correctly applied.The path references have been consistently updated from
components/backends/toexamples/backends/for all three supported backends (vLLM, SGLang, and TensorRT-LLM).examples/deployments/ECS/task_definition_frontend.json (1)
26-26: LGTM! Working directory path correctly updated.The Docker command now references the new backend location at
examples/backends/vllm, consistent with the repository restructuring.examples/deployments/ECS/README.md (2)
87-87: LGTM! Documentation correctly updated.The Docker command examples now reference
examples/backends/vllm, consistent with the backend relocation.
93-93: LGTM! Prefill worker command correctly updated.The prefill worker Docker command now references the new backend location.
benchmarks/profiler/utils/config_modifiers/sglang.py (1)
38-38: LGTM! Default config path correctly updated.The
DEFAULT_SGLANG_CONFIG_PATHconstant now points to the new backend location atexamples/backends/sglang/deploy/disagg.yaml, consistent with the repository restructuring.docs/benchmarks/pre_deployment_profiling.md (3)
45-45: LGTM! Documentation paths correctly updated.The profiling results path reference and example command paths have been consistently updated to use
examples/backends/instead ofcomponents/backends/.
134-134: LGTM! Auto-configuration documentation updated.The default config file path reference correctly points to the new location.
294-294: LGTM! Example command updated.The TensorRT-LLM command example now references the correct backend path.
components/README.md (1)
24-32: LGTM! Components README correctly updated.The Backends section has been appropriately restructured to reflect that launch and deploy scripts now reside in the
../examples/backends/directory. The documentation links to backend-specific READMEs are preserved.docs/backends/trtllm/multinode/multinode-examples.md (2)
51-51: LGTM! Documentation link anchor updated.The link anchor has been updated from
#build-dockerto#build-container, aligning with the updated anchor name in the target documentation.
90-90: LGTM! Second link anchor updated consistently.The second reference to the build instructions also correctly uses the new
#build-containeranchor.docs/backends/trtllm/llama4_plus_eagle.md (1)
42-42: LGTM! Setup instructions correctly updated.The working directory path has been updated from
$DYNAMO_HOME/components/backends/trtllmto$DYNAMO_HOME/examples/backends/trtllm, consistent with the backend relocation.examples/backends/sglang/deploy/disagg.yaml (1)
29-29: Consistent path migration.Both workingDir references updated correctly to point to the new examples location. Changes are straightforward and align with the broader refactoring.
Also applies to: 65-65
.github/workflows/container-validation-backends.yml (1)
451-451: Path reference in commented deployment test is updated correctly.The commented deployment test section now reflects the new examples-based path structure. When these deployment tests are enabled (see OPS-1140 TODO comments), they will target the correct backend locations.
examples/backends/sglang/deploy/agg.yaml (1)
28-28: Path migration is consistent with aggregated deployment pattern.Working directory correctly points to
/workspace/examples/backends/sglangroot (without/deploy/subdirectory), consistent with disagg.yaml and other deployment manifests. The module command (python3 -m dynamo.sglang) will execute from this root directory.tests/fault_tolerance/deploy/templates/vllm/moe_disagg.yaml (1)
55-55: Test fixture working directories updated consistently.Both decode and prefill worker workingDir references correctly migrated to
examples/backends/vllm. Test fixture now points to the new backend location for fault tolerance deployment testing.Also applies to: 120-120
examples/backends/sglang/deploy/README.md (1)
65-65: Documentation example updated to match new deployment structure.The YAML configuration example in the Container Configuration section now shows the correct working directory path. This ensures users following the deployment guide will reference the correct backend location.
docs/backends/trtllm/multimodal_epd.md (1)
24-24: Setup documentation path corrected.The
cdcommand in the "How to use" section now references the correct examples-based path. Users following this guide will navigate to the correct backend directory before executing launch scripts.examples/backends/sglang/deploy/agg_router.yaml (1)
31-31: Path migration applied to router variant consistently.The aggregated router deployment workingDir now points to the new examples location. Routing configuration remains unchanged; only the backend path has been updated.
examples/backends/vllm/deploy/agg_kvbm.yaml (1)
35-35: Path migration applied to KVBM variant without affecting optimization settings.The vLLM KV Block Manager deployment workingDir now references the new examples path. KVBM-specific configuration (
DYN_KVBM_CPU_CACHE_GB) remains intact.docs/backends/trtllm/multinode/multinode-multimodal-example.md (1)
55-55: LGTM! Documentation reference correctly updated.The URL reference has been properly updated to reflect the new path structure.
examples/backends/sglang/deploy/agg_logging.yaml (1)
31-31: LGTM! Working directory correctly updated.The workingDir field has been properly updated to reflect the new examples directory structure.
examples/backends/trtllm/deploy/disagg_planner.yaml (1)
20-20: LGTM! Frontend working directory correctly updated.The Frontend service's workingDir has been properly updated to the new examples path. Note that the Planner service (line 48) still references
/workspace/components/src/dynamo/planner, which appears intentional as the planner is a separate infrastructure component, not a backend.tests/serve/test_vllm.py (1)
33-35: LGTM! Test path correctly updated.The default vLLM directory path has been properly updated to reference the new examples directory structure.
examples/backends/trtllm/deploy/agg-with-config.yaml (1)
54-54: LGTM! Working directory correctly updated.The TRTLLMWorker's workingDir has been properly updated to reflect the new examples directory structure.
docs/design_docs/dynamo_flow.md (1)
20-20: LGTM! Documentation reference correctly updated.The implementation path reference has been properly updated to point to the new examples directory structure.
examples/backends/vllm/deploy/agg.yaml (1)
28-28: LGTM! Working directory correctly updated.The VllmDecodeWorker's workingDir has been properly updated to reflect the new examples directory structure.
examples/backends/sglang/deploy/disagg-multinode.yaml (1)
39-39: LGTM! Working directories correctly updated.Both the decode and prefill worker services have been properly updated to use the new examples directory structure. The changes are consistent across both services.
Also applies to: 76-76
examples/backends/trtllm/deploy/disagg-multinode.yaml (1)
99-99: LGTM! Path migration applied correctly.The working directory path has been updated consistently with the broader repository restructuring from
components/backendstoexamples/backends.docs/backends/trtllm/gemma3_sliding_window_attention.md (1)
30-30: LGTM! Documentation paths updated consistently.All four example commands now reference the new
examples/backends/trtllmpath, ensuring users follow the updated directory structure.Also applies to: 39-39, 48-48, 58-58
deploy/helm/README.md (1)
36-36: LGTM! Helm deployment examples updated correctly.All references to the vLLM deployment configuration now point to the new
examples/backends/vllm/deploy/agg.yamlpath.Also applies to: 44-44, 53-53
tests/planner/perf_test_configs/disagg_8b_planner.yaml (1)
45-45: LGTM! Test configuration paths updated consistently.All three service containers (Frontend, VllmDecodeWorker, VllmPrefillWorker) now use the correct working directory path under
examples/backends/vllm.Also applies to: 142-142, 199-199
tests/profiler/test_profile_sla_aiconfigurator.py (1)
31-31: LGTM! Test fixture updated to reference new path.The test configuration now correctly points to the deployment YAML at its new location in
examples/backends/trtllm/deploy/.examples/backends/vllm/deploy/disagg_planner.yaml (1)
52-52: LGTM! Worker paths updated correctly.Both worker containers now use the correct working directory path. Note that the Planner container at line 30 retains its path at
/workspace/components/src/dynamo/planner, which appears intentional as the planner code itself wasn't moved.Also applies to: 72-72
docs/backends/trtllm/gpt-oss.md (1)
131-131: LGTM! Documentation paths updated correctly.Both launch script and frontend start directory references now use the new
examples/backends/trtllmpath structure. The different path prefixes (with and without/dynamo/) appear context-appropriate for their respective use cases.Also applies to: 139-139
docs/kubernetes/README.md (1)
70-72: LGTM! Path migration fully verified and complete.All backend reference links in
docs/kubernetes/README.md(lines 70-72, 87) correctly point to the newexamples/backendsstructure. Verification confirms:
- All three backend deployment READMEs exist at the new locations
- No remaining references to the old
components/backendspath in the codebase- Documentation links and deployment example command use correct relative and absolute paths
examples/backends/sglang/deploy/disagg_planner.yaml (1)
53-53: LGTM! Path updates are consistent.The workingDir updates correctly reflect the relocation of backend files from
components/backends/sglangtoexamples/backends/sglang.Also applies to: 89-89
tests/profiler/test_profile_sla_dryrun.py (1)
49-49: LGTM! Test configuration paths updated consistently.All three backend test fixtures (vllm, sglang, trtllm) now reference the correct deployment configurations in the
examples/backends/directory.Also applies to: 83-83, 131-131
docs/kubernetes/installation_guide.md (1)
236-246: LGTM! Documentation links updated correctly.The deployment example and backend guide references now point to the
examples/backends/directory, maintaining consistency with the repository restructuring.docs/design_docs/distributed_runtime.md (1)
31-31: LGTM! Documentation references updated correctly.The deployment configuration examples and Python bindings references now accurately reflect the reorganized directory structure.
Also applies to: 78-78
tests/fault_tolerance/deploy/scenarios.py (1)
180-181: LGTM! Deployment path templates updated correctly.The yaml file path templates now correctly reference the
examples/backends/directory for all backend types, maintaining consistency with the repository restructure.docs/backends/sglang/README.md (2)
185-242: LGTM! Quick Start paths updated consistently.All example commands in the Quick Start section now correctly reference the
examples/backends/sglangdirectory for aggregated serving, embedding, disaggregation, and DP attention examples.
288-291: LGTM! Deployment documentation links updated.The Kubernetes and SLURM deployment references now correctly point to the
examples/backends/sglang/directory.deploy/inference-gateway/README.md (1)
88-119: LGTM! Inference Gateway documentation updated correctly.All VLLM deployment references, including model deployment links and working directory examples, now correctly point to the
examples/backends/vllm/directory.tests/serve/test_trtllm.py (1)
32-34: LGTM! Backend path migration verified as complete.The
trtllm_dircorrectly references the relocatedexamples/backends/trtllmdirectory. Verification confirms:
- No remaining references to old
components/backendspaths in the codebase- New path exists with expected subdirectories (deploy, launch, performance_sweeps)
- Migration successfully applied across all backend references
docs/backends/vllm/README.md (2)
109-109: Working directory updates for vLLM backend deployment examples.The shell script examples correctly reference the new
examples/backends/vllmdirectory. These changes are consistent across all deployment patterns (aggregated, router variants, and dep).Also applies to: 117-117, 125-125, 133-133, 143-143
156-156: Relative path is correct — no action required.The verification confirms the relative path
../../../examples/backends/vllm/deploy/README.mdfromdocs/backends/vllm/README.mdresolves correctly toexamples/backends/vllm/deploy/README.md, and the target file exists.tests/planner/README.md (2)
13-13: Backend README references updated consistently.The text reference to backend documentation paths is correctly updated across all three backends (vllm, sglang, trtllm).
173-173: Source file verified—no changes required.The file
examples/backends/vllm/deploy/disagg_planner.yamlexists at the specified location, confirming that users can successfully execute the copy operation referenced in the documentation..github/filters.yaml (1)
31-31: CI filter glob patterns correctly updated for all backends.All three backend filter patterns (vllm, sglang, trtllm) are consistently updated from
components/backends/<name>/**toexamples/backends/<name>/**. The glob syntax is correct and will properly match changes in the new directory structure.Also applies to: 38-38, 45-45
docs/backends/trtllm/README.md (3)
131-131: Working directory updates for TensorRT-LLM backend deployment examples.All shell script examples consistently reference the new
examples/backends/trtllmdirectory. The use of$DYNAMO_HOMEvariable in the cd commands is clear and follows the documentation's established pattern.Also applies to: 137-137, 147-147, 157-157, 163-163, 273-273
319-319: The documentation link has been verified as valid. The performance_sweeps directory and README.md file both exist at the pathexamples/backends/trtllm/performance_sweeps/README.mdas referenced in line 319.
189-189: Relative path is correct and resolves properly.The verification confirms that the relative path
../../../examples/backends/trtllm/deploy/README.mdfromdocs/backends/trtllm/README.mdresolves correctly toexamples/backends/trtllm/deploy/README.md, and the target file exists.
Signed-off-by: Anant Sharma <[email protected]>
|
/ok to test 9120955 |
Signed-off-by: Anant Sharma <[email protected]>
|
/ok to test d55dc3d |
|
/ok to test be35a04 |
2 participants
Overview:
Restructure the codebase by moving backend deployment, launch, and Slurm configuration files from the components/ directory to a centralized examples/backends/ location. Changes:
examples/backends/<backend>/ref: components dir restructing DEP.
closes: OPS-1363
Summary by CodeRabbit