Your home for deploying high performance clusters to Train, Fine Tune, and Host models on Amazon SageMaker HyperPod.
This is the home for all things related to Amazon Sagemaker Hyperpod, built by the ML Frameworks team. We strive to release content and assets that are based on our customer's feedback and help them to improve their operational efficiency.
Explore practical examples, architectural patterns, troubleshooting, and many other contents. Work through running large distributed training jobs, fine tuning, distillation, and preference alignment, using frameworks such as PyTorch, JAX, NeMo, Ray, etc. We provide examples for Meta's Llama, Amazon Nova, Mistral, DeepSeek, and others.
There are troubleshooting advice on specific problems you may find, best practices when integrating with other AWS services and open source projects, and code snippets that you may find useful to incorporate on your workloads.
Note: AI On Sagemaker Hyperpod is an active development. For upcoming features and enhancements, please check out the issues section.
Those are some examples you can find on this project, using the EKS orchestration:
- Running a Fully Sharded Data Parallel training example on multiple GPUs
- Running a Distributed Data Parallel training example using CPU only
- Using AWS Trainium chips to train your generative AI models
- Using Ray to run your large distributed training job
- Using the Training Operator with HyperPod
- Creating a hybrid infrastructure using SkyPilot
- Setting up Task Governance and Task Affinity for improved cluster governance and utilization
- Deploying your models for inference
You can also find examples using the SLURM orchestration, on the website.
Before delighting yourself with the features and examples provided here, we suggest you work through the setup of your Sagemaker Hyperpod cluster. On that initial step, we provide examples on how to do it using different methods (GUI, CLI scripts, Infrastructure as a Code - IaC, etc). After the deploying, we recommend running a few basic tests to validate you have a working cluster running as expected.
Then you can select which of the scenarios you want to work on. On every scenario we have two possible orchestration choices: using SLURM or EKS. You should select the specific example you want to go through and the specific orchestration engine you are using on your cluster.
Amazon Sagemaker Hyperpod is part of the Amazon Sagemaker AI family of AI focused managed services on AWS. The documentation focuses on helping customers setup their clusters and AWS accounts.
This repository strives to go further and help customers setup the additional software stack required to quickly conduct proof-of-concepts and build production-ready clusters.
AI on SageMaker HyperPod is maintained by the AWS ML Frameworks team and is not an AWS service. Support is provided on a best effort basis by the AI on Sagemaker Hyperpod community. If you have feedback, feature ideas, or wish to report bugs, please use the Issues section of this GitHub.
See CONTRIBUTING for more information.
This library is licensed under the Apache 2.0 License.
We're building an open-source community focused on Development and Inference of Generative AI models on ML Frameworks.
Come join us and contribute to shaping the future of AI on Amazon Sagemaker Hyperpod.
Built with ❤️ at AWS.