README.md

Easy Model Deployer: Simple, Efficient, and Easy-to-Integrate

Documentation · Changelog

🔥 Latest News

• 2025-03-06: Deploy QwQ-32B with one command line.

Introduction

Easy Model Deployer is a lightweight tool designed to simplify the deployment of Open-Source LLMs (Supported Models) and Custom Models. It provides OpenAI's Completions API and LangChain Interface. Built for developers who need reliable and scalable model serving without complex setup, it seamlessly integrates with AWS services for efficient model deployment.

Key Features

• One-click deployment of models to the cloud (Amazon SageMaker, Amazon ECS, Amazon EC2)
• Diverse model types (LLMs, VLMs, Embeddings, Vision, etc.)
• Rich inference engine (vLLM, TGI, Lmdeploy, etc.)
• Different instance types (CPU/GPU/AWS Inferentia)
• Convenient integration (OpenAI Compatible API, LangChain client, etc.)

🚀 Quick Install

Install EMD with pip, currently support for Python 3.9 and above:

pip install https://github.com/aws-samples/easy-model-deployer/releases/download/emd-0.7.1/emd-0.7.1-py3-none-any.whl

Visit our documentation to learn more.

🔧 Usage

(Optional) Configure AWS Profile

If you have already configured your AWS credentials using the AWS CLI, you can skip this step.

You can configure the AWS profile by EMD. If you don't set any AWS profile, the EMD will use the default credentials in your terminal, you can also configure your credentials by aws configure.

emd config set-default-profile-name

Bootstrap

Prepare the essential resources required for model deployment.

For more information, please refer to Architecture.

emd bootstrap

💡 Tip Once you upgrade the EMD by pip, you need to run this command again.

Deploy Model

Deploy models with an interactive CLI or one command.

emd deploy

💡 Tip To view all available parameters, run emd deploy --help. When you see the message "Waiting for model: ...", it means the deployment task has started and you can stop the terminal output by pressing Ctrl+C.

• For more information on deployment parameters, please refer to the Deployment parameters.
• For best practice examples of using command line parameters, please refer to the Best Deployment Practices.

Show Status

Check the status of the model deployment task.

emd status

💡 Tip The EMD allows launch multiple deployment tasks simultaneously.

Quick Invocation

Invoke the deployed model for testing by CLI.

emd invoke DeepSeek-R1-Distill-Qwen-1.5B

💡 Tip You can find the ModelId in the output by emd status.

• Integration examples
• EMD client
• Langchain interface
• OpenAI compatible interface.

💡 Tip OpenAI Compatible API is supported only for Amazon ECS and Amazon EC2 deployment types.

List Supported Models

Quickly see what models are supported, this command will output all information related to deployment. (Plese browse Supported Models for more information.)

emd list-supported-models

The following command is recommended to just list the model types.

emd list-supported-models | jq -r '.[] | "\(.model_id)\t\(.model_type)"' | column -t -s $'\t' | sort

Delete Model

Delete the deployed model.

emd destroy DeepSeek-R1-Distill-Qwen-1.5B

💡 Tip You can find the ModelId in the output by emd status.

📖 Documentation

For advanced configurations and detailed guides, visit our documentation site.

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.