Skip to content

Commit fd95cfb

Browse files
authored
[HWORKS-1885] Improve vLLM-related docs (#445)
1 parent 60498d8 commit fd95cfb

File tree

13 files changed

+230
-59
lines changed

13 files changed

+230
-59
lines changed
Loading

docs/user_guides/mlops/registry/frameworks/llm.md

+5-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
# How To Export an LLM Model
1+
---
2+
description: Documentation on how to export a Large Language Model (LLM) to the model registry
3+
---
4+
5+
# How To Export a Large Language Model (LLM)
26

37
## Introduction
48

docs/user_guides/mlops/registry/frameworks/python.md

+4
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
---
2+
description: Documentation on how to export a Python model to the model registry
3+
---
4+
15
# How To Export a Python Model
26

37
## Introduction

docs/user_guides/mlops/registry/frameworks/skl.md

+4
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
---
2+
description: Documentation on how to export a Scikit-learn model to the model registry
3+
---
4+
15
# How To Export a Scikit-learn Model
26

37
## Introduction
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
description: Documentation on how to export a Pytorch model to the model registry
3+
---
4+
5+
# How To Export a Torch Model
6+
7+
## Introduction
8+
9+
In this guide you will learn how to export a Torch model and register it in the Model Registry.
10+
11+
12+
## Code
13+
14+
### Step 1: Connect to Hopsworks
15+
16+
=== "Python"
17+
```python
18+
import hopsworks
19+
20+
project = hopsworks.login()
21+
22+
# get Hopsworks Model Registry handle
23+
mr = project.get_model_registry()
24+
```
25+
26+
### Step 2: Train
27+
28+
Define your Torch model and run the training loop.
29+
30+
=== "Python"
31+
```python
32+
# Define the model architecture
33+
class Net(nn.Module):
34+
def __init__(self):
35+
super().__init__()
36+
self.conv1 = nn.Conv2d(3, 6, 5)
37+
...
38+
39+
def forward(self, x):
40+
x = self.pool(F.relu(self.conv1(x)))
41+
...
42+
return x
43+
44+
# Instantiate the model
45+
net = Net()
46+
47+
# Run the training loop
48+
for epoch in range(n):
49+
...
50+
```
51+
52+
### Step 3: Export to local path
53+
54+
Export the Torch model to a directory on the local filesystem.
55+
56+
=== "Python"
57+
```python
58+
model_dir = "./model"
59+
60+
torch.save(net.state_dict(), model_dir)
61+
```
62+
63+
### Step 4: Register model in registry
64+
65+
Use the `ModelRegistry.torch.create_model(..)` function to register a model as a Torch model. Define a name, and attach optional metrics for your model, then invoke the `save()` function with the parameter being the path to the local directory where the model was exported to.
66+
67+
=== "Python"
68+
```python
69+
# Model evaluation metrics
70+
metrics = {'accuracy': 0.92}
71+
72+
tch_model = mr.torch.create_model("tch_model", metrics=metrics)
73+
74+
tch_model.save(model_dir)
75+
```
76+
77+
## Going Further
78+
79+
You can attach an [Input Example](../input_example.md) and a [Model Schema](../model_schema.md) to your model to document the shape and type of the data the model was trained on.

docs/user_guides/mlops/registry/frameworks/tf.md

+4
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
---
2+
description: Documentation on how to export a Tensorflow model to the model registry
3+
---
4+
15
# How To Export a TensorFlow Model
26

37
## Introduction

docs/user_guides/mlops/registry/index.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,13 @@ Follow these framework-specific guides to export a Model to the Model Registry.
1111

1212
* [TensorFlow](frameworks/tf.md)
1313

14+
* [Torch](frameworks/tch.md)
15+
1416
* [Scikit-learn](frameworks/skl.md)
1517

1618
* [LLM](frameworks/llm.md)
1719

18-
* [Other frameworks](frameworks/python.md)
20+
* [Other Python frameworks](frameworks/python.md)
1921

2022

2123
## Model Schema

docs/user_guides/mlops/serving/deployment.md

+19-10
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
1-
# How To Create A Deployment
1+
---
2+
description: Documentation on how to deployment Machine Learning (ML) models and Large Language Models (LLMs)
3+
---
4+
5+
# How To Create A Model Deployment
26

37
## Introduction
48

59
In this guide, you will learn how to create a new deployment for a trained model.
610

711
!!! warning
8-
This guide assumes that a model has already been trained and saved into the Model Registry. To learn how to create a model in the Model Registry, see [Model Registry Guide](../registry/frameworks/tf.md)
12+
This guide assumes that a model has already been trained and saved into the Model Registry. To learn how to create a model in the Model Registry, see [Model Registry Guide](../registry/index.md#exporting-a-model)
913

1014
Deployments are used to unify the different components involved in making one or more trained models online and accessible to compute predictions on demand. For each deployment, there are four concepts to consider:
1115

@@ -41,8 +45,8 @@ After selecting the model, the rest of fields are filled automatically. We pick
4145
!!! notice "Deployment name validation rules"
4246
A valid deployment name can only contain characters a-z, A-Z and 0-9.
4347

44-
!!! info "Predictor script for Python models and LLMs"
45-
For Python models and LLMs, you must select a custom [predictor script](#predictor) that loads and runs the trained model by clicking on `From project` or `Upload new file`, to choose an existing script in the project file system or upload a new script, respectively.
48+
!!! info "Predictor script for Python models"
49+
For Python models, you must select a custom [predictor script](#predictor) that loads and runs the trained model by clicking on `From project` or `Upload new file`, to choose an existing script in the project file system or upload a new script, respectively.
4650

4751
If you prefer, change the name of the deployment, model version or [artifact version](#model-artifact). Then, click on `Create new deployment` to create the deployment for your model.
4852

@@ -76,10 +80,10 @@ You will be redirected to a full-page deployment creation form where you can see
7680
!!! info "Deployment advanced options"
7781
1. [Predictor](#predictor)
7882
2. [Transformer](#transformer)
79-
3. [Inference logger](#inference-logger)
80-
4. [Inference batcher](#inference-batcher)
81-
5. [Resources](#resources)
82-
6. [API protocol](#api-protocol)
83+
3. [Inference logger](predictor.md#inference-logger)
84+
4. [Inference batcher](predictor.md#inference-batcher)
85+
5. [Resources](predictor.md#resources)
86+
6. [API protocol](predictor.md#api-protocol)
8387

8488
Once you are done with the changes, click on `Create new deployment` at the bottom of the page to create the deployment for your model.
8589

@@ -174,7 +178,12 @@ Inside a model deployment, the local path to the model files is stored in the `M
174178

175179
## Artifact Files
176180

177-
Artifact files are files involved in the correct startup and running of the model deployment. The most important files are the **predictor** and **transformer scripts**. The former is used to load and run the model for making predictions. The latter is typically used to transform model inputs at inference time.
181+
Artifact files are files involved in the correct startup and running of the model deployment. The most important files are the **predictor** and **transformer scripts**. The former is used to load and run the model for making predictions. The latter is typically used to apply transformations on the model inputs at inference time before making predictions. Predictor and transformer scripts run on separate components and, therefore, scale independently of each other.
182+
183+
!!! tip
184+
Whenever you provide a predictor script, you can include the transformations of model inputs in the same script as far as they don't need to be scaled independently from the model inference process.
185+
186+
Additionally, artifact files can also contain a **server configuration file** that helps detach configuration used within the model deployment from the model server or the implementation of the predictor and transformer scripts. Inside a model deployment, the local path to the configuration file is stored in the `CONFIG_FILE_PATH` environment variable (see [environment variables](../serving/predictor.md#environment-variables)).
178187

179188
Every model deployment runs a specific version of the artifact files, commonly referred to as artifact version. ==One or more model deployments can use the same artifact version== (i.e., same predictor and transformer scripts). Artifact versions are unique for the same model version.
180189

@@ -189,7 +198,7 @@ Inside a model deployment, the local path to the artifact files is stored in the
189198
All files under `/Models` are managed by Hopsworks. Changes to artifact files cannot be reverted and can have an impact on existing model deployments.
190199

191200
!!! tip "Additional files"
192-
Currently, the artifact files only include predictor and transformer scripts. Support for additional files (e.g., configuration files or other resources) is coming soon.
201+
Currently, the artifact files can only include predictor and transformer scripts, and a configuration file. Support for additional files (e.g., other resources) is coming soon.
193202

194203
## Predictor
195204

0 commit comments

Comments
 (0)