forked from keephq/keep
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
11 changed files
with
393 additions
and
47 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
--- | ||
title: "vLLM Provider" | ||
description: "The vLLM Provider enables integration with vLLM-deployed language models into Keep." | ||
--- | ||
|
||
<Tip> | ||
The vLLM Provider supports querying language models deployed with vLLM for prompt-based interactions. | ||
</Tip> | ||
|
||
## Inputs | ||
|
||
The vLLM Provider supports the following parameters: | ||
|
||
- `prompt`: Interact with vLLM-deployed models by sending prompts and receiving responses | ||
- `model`: The model to be used, defaults to `Qwen/Qwen1.5-1.8B-Chat` | ||
- `temperature`: Controls randomness in the response, defaults to 0.7 | ||
- `max_tokens`: Limit amount of tokens returned by the model, defaults to 1024 | ||
- `structured_output_format`: Optional JSON schema for structured output formatting | ||
|
||
## Outputs | ||
|
||
The vLLM Provider returns the model's response based on the provided prompt. When using structured output format, the response will be formatted according to the provided JSON schema. | ||
|
||
## Authentication Parameters | ||
|
||
To use the vLLM Provider, you'll need to configure the following authentication parameters: | ||
|
||
- **api_url** (required): The endpoint URL where your vLLM service is deployed | ||
- **api_key** (optional): API key if your vLLM deployment requires authentication | ||
|
||
## Connecting with the Provider | ||
|
||
To connect to a vLLM deployment: | ||
|
||
1. Deploy your vLLM instance or obtain the API endpoint of an existing deployment | ||
2. Configure the API URL in your provider configuration | ||
3. If your deployment requires authentication, configure the API key | ||
|
||
## Structured Output | ||
|
||
Structure output for vLLM should follow Json Schema notation. Example: | ||
|
||
```yaml | ||
steps: | ||
- name: get-enrichments | ||
provider: | ||
config: "{{ providers.my_vllm }}" | ||
type: vllm | ||
with: | ||
prompt: "You received such an alert {{alert}}, generate missing fields." | ||
model: "Qwen/Qwen1.5-1.8B-Chat" # This model supports structured output | ||
structured_output_format: # We limit what model could return | ||
type: object | ||
properties: | ||
environment: | ||
type: string | ||
enum: | ||
- production | ||
- debug | ||
- pre-prod | ||
impacted_customer_name: | ||
type: string | ||
required: | ||
- environment | ||
- impacted_customer_name | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
41 changes: 41 additions & 0 deletions
41
examples/workflows/enrich_using_structured_output_from_vllm_qwen.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
id: enrich-using-vllm-qwen | ||
description: Enrich alerts using structured output from vLLM & Qwen | ||
triggers: | ||
- type: alert | ||
filters: | ||
- key: source | ||
value: prometheus | ||
|
||
steps: | ||
- name: get-enrichments | ||
provider: | ||
config: "{{ providers.my_vllm }}" | ||
type: vllm | ||
with: | ||
prompt: "You received such an alert {{alert}}, generate missing fields." | ||
model: "Qwen/Qwen1.5-1.8B-Chat" # This model supports structured output | ||
structured_output_format: # We limit what model could return | ||
type: object | ||
properties: | ||
environment: | ||
type: string | ||
enum: | ||
- production | ||
- debug | ||
- pre-prod | ||
impacted_customer_name: | ||
type: string | ||
required: | ||
- environment | ||
- impacted_customer_name | ||
|
||
actions: | ||
- name: enrich-alert | ||
provider: | ||
type: mock | ||
with: | ||
enrich_alert: | ||
- key: environment | ||
value: "{{ steps.get-enrichments.results.response.environment }}" | ||
- key: impacted_customer_name | ||
value: "{{ steps.get-enrichments.results.response.impacted_customer_name }}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.