Skip to content

Commit 718629b

Browse files
committed
docs: update document
1 parent 144d72d commit 718629b

File tree

8 files changed

+30
-156
lines changed

8 files changed

+30
-156
lines changed

docs/api/cli.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ pip install serverless-llm
2424

2525
Before using the `sllm-cli` commands, you need to start the ServerlessLLM cluster. Follow the guides below to set up your cluster:
2626

27-
- [Single Machine Deployment](../stable/gettting_started.md)
27+
- [Single Machine Deployment](../stable/getting_started.md)
2828
- [Single Machine Deployment (From Scratch)](../stable/deployment/single_machine.md)
2929
- [Multi-Machine Deployment](../stable/deployment/multi_machine.md)
3030
- [SLURM Cluster Deployment](../stable/deployment/slurm_cluster.md)

docs/stable/deployment/single_machine.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
This guide provides instructions for setting up ServerlessLLM from scratch on a single machine. This 'from scratch' approach means you will manually initialize and manage the Ray cluster components. It involves using multiple terminal sessions, each configured with a distinct Conda environment, to run the head and worker processes on the same physical machine, effectively simulating a multi-node deployment locally.
88

99
:::note
10-
We strongly recommend using Docker (Compose) as detailed in the [Docker Compose guide](../gettting_started.md). Docker provides a smoother and generally easier setup process. Follow this guide only if Docker is not a suitable option for your environment.
10+
We strongly recommend using Docker (Compose) as detailed in the [Docker Compose guide](../getting_started.md). Docker provides a smoother and generally easier setup process. Follow this guide only if Docker is not a suitable option for your environment.
1111
:::
1212

1313
## Installation

docs/stable/deployment/slurm_cluster.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ sidebar_position: 3
44

55
# SLURM cluster
66

7-
This guide will help you get started with running ServerlessLLM on SLURM cluster. It provides two deployment methods, based on `sbatch` and `srun`. If you are in development, we recommend using `srun`, as it is easier to debug than `sbatch`, and if you are in production mode, `sbatch` is recommended. Please make sure you have installed the ServerlessLLM following the [installation guide](./installation.md) on all machines.
7+
This guide will help you get started with running ServerlessLLM on SLURM cluster. It provides two deployment methods, based on `sbatch` and `srun`. If you are in development, we recommend using `srun`, as it is easier to debug than `sbatch`, and if you are in production mode, `sbatch` is recommended. Please make sure you have installed the ServerlessLLM following the [installation guide](./single_machine.md#installation) on all machines.
88

99
## Pre-requisites
1010
Before you begin, make sure you have checked the following:

docs/stable/features/live_migration.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ This example illustrates the live migration of inference instances in a Serverle
1111

1212
## Prerequisites
1313

14-
To run this example, we will use Docker Compose to set up a ServerlessLLM cluster. Before proceeding, please ensure you have read the [Docker Quickstart Guide](../getting_started/docker_quickstart.md).
14+
To run this example, we will use Docker Compose to set up a ServerlessLLM cluster. Before proceeding, please ensure you have read the [Quickstart Guide](../getting_started.md).
1515

1616
**Requirements:**
1717

docs/stable/features/peft_lora_serving.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ sidebar_position: 2
55

66
## Pre-requisites
77

8-
To run this example, we will use Docker Compose to set up a ServerlessLLM cluster. Before proceeding, please ensure you have read the [Docker Quickstart Guide](../getting_started/docker_quickstart.md).
8+
To run this example, we will use Docker Compose to set up a ServerlessLLM cluster. Before proceeding, please ensure you have read the [Quickstart Guide](../getting_started.md).
99

1010
We will use the following example base model & LoRA adapter
1111
- Base model: `facebook/opt-125m`

docs/stable/features/storage_aware_scheduling.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ sidebar_position: 0
66

77
## Pre-requisites
88

9-
We will use Docker Compose to run a ServerlessLLM cluster in this example. Therefore, please make sure you have read the [Docker Quickstart Guide](../getting_started/docker_quickstart.md) before proceeding.
9+
We will use Docker Compose to run a ServerlessLLM cluster in this example. Therefore, please make sure you have read the [Quickstart Guide](../getting_started.md) before proceeding.
1010

1111
## Usage
1212

docs/stable/gettting_started.md

Lines changed: 0 additions & 136 deletions
This file was deleted.

docs/stable/store/quickstart.md

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -110,21 +110,31 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
110110

111111
## Usage with vLLM
112112

113-
:::tip
114-
To use ServerlessLLM as the load format for vLLM, you need to apply our patch `sllm_store/vllm_patch/sllm_load.patch` to the installed vLLM library. Therefore, please ensure you have applied our `vLLM Patch` as instructed in [installation guide](../getting_started/installation.md).
113+
ServerlessLLM integrates with vLLM to provide fast model loading capabilities. Follow these steps to set up and use ServerlessLLM with vLLM.
115114

116-
You may check the patch status by running the following command:
117-
``` bash
118-
./sllm_store/vllm_patch/check_patch.sh
119-
```
120-
If the patch is not applied, you can apply it by running the following command:
121-
```bash
122-
./sllm_store/vllm_patch/patch.sh
123-
```
124-
To remove the applied patch, you can run the following command:
125-
```bash
126-
./sllm_store/vllm_patch/remove_patch.sh
127-
```
115+
### Prerequisites
116+
117+
Before using ServerlessLLM with vLLM, you need to apply a compatibility patch to your vLLM installation. This patch has been tested with vLLM version `0.6.6`.
118+
119+
### Apply the vLLM Patch
120+
121+
1. **Check patch status** (optional):
122+
```bash
123+
./sllm_store/vllm_patch/check_patch.sh
124+
```
125+
126+
2. **Apply the patch**:
127+
```bash
128+
./sllm_store/vllm_patch/patch.sh
129+
```
130+
131+
3. **Remove the patch** (if needed):
132+
```bash
133+
./sllm_store/vllm_patch/remove_patch.sh
134+
```
135+
136+
:::note
137+
The patch file is located at `sllm_store/vllm_patch/sllm_load.patch` in the ServerlessLLM repository.
128138
:::
129139

130140

0 commit comments

Comments
 (0)