Skip to content

Commit 95311bd

Browse files
committed
docs: condense fabric reference guidance
1 parent 57b3e95 commit 95311bd

3 files changed

Lines changed: 5 additions & 6 deletions

File tree

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ Recipes do not hardcode host DRAM sizes either. A DRAM point's budget, the matri
7272

7373
References resolve to integers, or to decimal strings in env values and argument lists (e.g. after `--l1-size-gb`), including whole values inside JSON object strings such as `kv-transfer-config`. Per-rank pools take the `per-gpu` sizes, and Mooncake `global_segment_size` takes bytes because it reads `GB` as GiB. Unknown names, embedded references and references on a point without a DRAM budget fail. SimpleCPU, LMCache CPU and `--l1-size-gb` sizes must be references; measured HiCache, TRT-LLM host-cache and Mooncake sizes may stay literal if they fit the node's share. When a backend allocates several pools from the budget, such as the KV and Mamba HiCache pools of a hybrid model, the master's `dram-utilization` sizes one of them.
7474

75-
Recipes do not hardcode cluster hardware facts: a value `'@fabric.<name>'`, anywhere in the recipe, becomes the job cluster's `srt-slurm.fabric` field after binding (lists comma-joined; fields in [CONFIGS.md](../../../configs/CONFIGS.md#runners)). A name the schema does not define, a reference inside a longer string, or a field the cluster does not set fails before submission; matrix generation already fails the variant a row selects when any cluster its runner label reaches lacks the field. Nothing resolves references before launch: the recipe fingerprint hashes them as written, so it stays cluster-independent. A value a recipe chooses on purpose, such as a permuted rail order or a subset of the devices, stays literal.
75+
Recipes do not hardcode cluster hardware facts: a value `'@fabric.<name>'`, anywhere in the recipe, becomes the job cluster's `srt-slurm.fabric` field after binding (lists comma-joined; fields in [CONFIGS.md](../../../configs/CONFIGS.md#runners)). Matrix generation fails a selected variant whose reference is embedded in a longer string, names no field, or names one that a cluster its runner label reaches does not set. The recipe fingerprint hashes references as written, so it stays cluster-independent. A value a recipe chooses on purpose, such as a permuted rail order or a subset of the devices, stays literal.
7676

7777
Inspect what the launcher submits without a cluster:
7878

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES_zh.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ TileRT 使用固定版本的上游 srt-slurm 子模块。配置指定 `roles.pre
7272

7373
引用替换为整数;作为环境变量值或参数列表项(例如 `--l1-size-gb` 之后的一项)时替换为十进制字符串,JSON 对象字符串(例如 `kv-transfer-config`)中的完整取值同样适用。按 rank 分配的内存池使用 `per-gpu` 取值;Mooncake `global_segment_size` 使用字节,因为 Mooncake 将 `GB` 视为 GiB。未知名称、嵌在更长字符串中的引用,以及没有 DRAM 预算的测试点上的引用都会失败。SimpleCPU、LMCache CPU 和 `--l1-size-gb` 的大小必须使用引用;实测得到的 HiCache、TRT-LLM 主机缓存和 Mooncake 大小只要不超过节点份额,可以保留字面值。若后端从预算中分配多个内存池(例如混合模型的 HiCache KV 池和 Mamba 池),由主配置的 `dram-utilization` 确定其中一个池的大小。
7474

75-
配置不硬编码集群硬件信息:配置中任意位置的 `'@fabric.<name>'` 值在绑定后替换为作业所在集群的 `srt-slurm.fabric` 字段(列表以逗号连接;字段见 [CONFIGS.md](../../../configs/CONFIGS.md#runners))。schema 未定义的名称、嵌在更长字符串中的引用,或集群未设置的字段,都会在提交前失败;生成矩阵时,若某行选中的变体所引用的字段在其运行器标签可达的任一集群上未设置,也已失败。启动前不会解析引用:配置指纹按原样对引用计算哈希,因此与集群无关。配置有意选择的值(例如重新排列的 rail 顺序或设备子集)保持字面值。
75+
配置不硬编码集群硬件信息:配置中任意位置的 `'@fabric.<name>'` 值在绑定后替换为作业所在集群的 `srt-slurm.fabric` 字段(列表以逗号连接;字段见 [CONFIGS.md](../../../configs/CONFIGS.md#runners))。若选中变体的引用嵌在更长字符串中、名称未定义,或其运行器标签可达的某个集群未设置该字段,矩阵生成即失败。配置指纹按原样对引用计算哈希,因此与集群无关。配置有意选择的值(例如重新排列的 rail 顺序或设备子集)保持字面值。
7676

7777
无需集群即可查看启动器实际提交的内容:
7878

‎inferencex-e2e/configs/CONFIGS.md‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -239,7 +239,6 @@ schema; unknown keys fail.
239239
`'@fabric.<name>'` values: the lists `ucx-net-devices`, `rdma-devices` (the RDMA HCAs
240240
that NCCL, Mooncake, SGLang `disaggregation-ib-device` and `IBDEVICES` use) and
241241
`socket-ifname` (NCCL and Gloo), rendered comma-joined, and the numbers
242-
`mooncake-gid-index`, `mori-rdma-tc` and `mori-io-tc`. Matrix generation fails a recipe
243-
that references an undefined field, or one a cluster its runner label reaches does not
244-
set. The cluster's own `env` and `srt-slurm.host-setup.env` values may take a field the
245-
same way; they resolve when the record loads, and `env` still rejects the commas of a list.
242+
`mooncake-gid-index`, `mori-rdma-tc` and `mori-io-tc`. The cluster's own `env` and
243+
`srt-slurm.host-setup.env` values may take a field the same way; they resolve when the
244+
record loads, and `env` still rejects the commas of a list.

0 commit comments

Comments
 (0)