Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cross layer overlapping for domino #7178

Merged
merged 2 commits into from
Mar 28, 2025
Merged

Cross layer overlapping for domino #7178

merged 2 commits into from
Mar 28, 2025

Conversation

hwchen2017
Copy link
Contributor

  1. Add implementation for cross layer communication overlapping to achieve communication "free".
  2. Optimize the implementation for communication overlapping within transformer layer.

@hwchen2017 hwchen2017 requested a review from tjruwase March 27, 2025 00:59
@hwchen2017 hwchen2017 requested a review from GuanhuaWang as a code owner March 27, 2025 00:59
@tjruwase tjruwase added this pull request to the merge queue Mar 28, 2025
Merged via the queue into master with commit f355b9e Mar 28, 2025
10 checks passed
@tjruwase tjruwase deleted the hongwei_domino_cross_layer_overlap branch March 28, 2025 22:21
deepcharm pushed a commit to deepcharm/DeepSpeed that referenced this pull request Mar 31, 2025
@hwchen2017 @loadams @deepcharm
Cross layer overlapping for domino (deepspeedai#7178)
1d218c8 
1. Add implementation for cross layer communication overlapping to
achieve communication "free".
2. Optimize the implementation for communication overlapping within
transformer layer.

Signed-off-by: Hongwei Chen <[email protected]>
Co-authored-by: Logan Adams <[email protected]>
Signed-off-by: Max Kovalenko <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tjruwase tjruwase tjruwase approved these changes

@GuanhuaWang GuanhuaWang Awaiting requested review from GuanhuaWang GuanhuaWang is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@hwchen2017 @tjruwase @loadams