Skip to content

Commit 8cd65b1

Browse files
TroyGardenmeta-codesync[bot]
authored andcommitted
benchmark for comms used in TorchRec (#3435)
Summary: Pull Request resolved: #3435 # context * add benchmark to `torch.distributed.all_to_all_single`. * in this very first edition, this all_to_all_single is running with sync on the same stream. * basic operations: pre-comms compute (GPU compute heavy) ==> all_to_all_single comms ==> irrelevant compute (GPU compute heavy, does not depend on comms data) ==> post-comms compute (GPU compute heavy, uses the comms data) # other changs * extend the cmd_conf decorator so that it can support multiple-program selection in command line ``` python -m torchrec.distributed.benchmark.benchmark_comms \ a2a_single --name=a2a_sync_base-$(git rev-parse --short HEAD || echo $USER) ``` * add a config (dataclass) class **`BenchmarkFunc`** for `benchmark_func`, which includes most common arguments used in `benchmark_func` Reviewed By: spmex Differential Revision: D83900855 fbshipit-source-id: 06279945cf382dbbbab8482b47c2f82976dbeef7
1 parent 3faf5e5 commit 8cd65b1

File tree

4 files changed

+415
-151
lines changed

4 files changed

+415
-151
lines changed

torchrec/distributed/benchmark/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,15 @@ python -m torchrec.distributed.benchmark.benchmark_train_pipeline \
1212
--yaml_config=fbcode/torchrec/distributed/benchmark/yaml/sparse_data_dist_base.yml \
1313
--name=sparse_data_dist_base_$(git rev-parse --short HEAD || echo $USER) # overrides the yaml config
1414
```
15+
16+
## benchmark_comms usage
17+
- internal:
18+
```
19+
buck2 run @fbcode//mode/opt fbcode//torchrec/distributed/benchmark:benchmark_comms -- \
20+
a2a_single --name=a2a_sync_base-$(hg whereami | cut -c 1-10)
21+
```
22+
- oss:
23+
```
24+
python -m torchrec.distributed.benchmark.benchmark_comms \
25+
a2a_single --name=a2a_sync_base-$(git rev-parse --short HEAD || echo $USER)
26+
```

0 commit comments

Comments
 (0)