Skip to content

Commit 8d353f1

Browse files
dminnear-rhdminnear-rh
committed
add docs for lemonade stand AI Quickstart pattern
1 parent 2a86ac1 commit 8d353f1

9 files changed

Lines changed: 920 additions & 0 deletions

File tree

‎content/patterns/lemonade-stand-quickstart/_index.adoc‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
title: Lemonade Stand AI Quickstart
3+
date: 2026-06-25
4+
tier: sandbox
5+
summary: This pattern deploys an AI guardrails demonstration with a multi-layered safety pipeline, interactive chatbot, and real-time monitoring on OpenShift.
6+
rh_products:
7+
- Red Hat OpenShift Container Platform
8+
- Red Hat OpenShift AI
9+
industries:
10+
- General
11+
focus_areas:
12+
- AI
13+
- Safety
14+
- AI Quickstart
15+
aliases: /lemonade-stand-quickstart/
16+
links:
17+
github: https://github.com/validatedpatterns-sandbox/ai-quickstart-lemonade-stand
18+
install: getting-started
19+
bugs: https://github.com/validatedpatterns-sandbox/ai-quickstart-lemonade-stand/issues
20+
feedback: https://docs.google.com/forms/d/e/1FAIpQLScI76b6tD1WyPu2-d_9CCVDr3Fu5jYERthqLKJDUGwqBg7Vcg/viewform
21+
---
22+
:toc:
23+
:imagesdir: /images
24+
:_content-type: ASSEMBLY
25+
include::modules/comm-attributes.adoc[]
26+
27+
include::modules/lemonade-stand-quickstart-about.adoc[leveloffset=+1]
28+
29+
include::modules/lemonade-stand-quickstart-architecture.adoc[leveloffset=+1]
30+
31+
[id="next-steps-lemonade-stand-quickstart"]
32+
== Next steps
33+
34+
* link:getting-started[Install this pattern]
35+
* link:cluster-sizing[Cluster sizing]
36+
* link:customizing-this-pattern[Customizing this pattern]
37+
* link:troubleshooting[Troubleshooting]

‎content/patterns/lemonade-stand-quickstart/cluster-sizing.adoc‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
title: Cluster sizing
3+
weight: 30
4+
aliases: /lemonade-stand-quickstart/cluster-sizing/
5+
---
6+
7+
:toc:
8+
:imagesdir: /images
9+
:_content-type: ASSEMBLY
10+
include::modules/comm-attributes.adoc[]
11+
include::modules/ai-quickstart-lemonade-stand/metadata-ai-quickstart-lemonade-stand.adoc[]
12+
13+
include::modules/cluster-sizing-template.adoc[]
14+
15+
[id="lemonade-stand-quickstart-gpu-node-requirements"]
16+
== GPU node requirements
17+
18+
In addition to the worker nodes listed above, this pattern requires at least 1 GPU-equipped node for LLM inference. On AWS, the pattern automatically provisions a `g5.2xlarge` instance with an NVIDIA A10G GPU. On other providers and bare metal, a GPU node must already be part of the cluster before deploying the pattern.
19+
20+
.GPU node minimum requirements
21+
[cols="<,^,<,<"]
22+
|===
23+
| Cloud provider | Node type | Number of nodes | Instance type
24+
25+
| Amazon Web Services
26+
| GPU Worker
27+
| 1
28+
| g5.2xlarge
29+
|===

‎content/patterns/lemonade-stand-quickstart/customizing-this-pattern.adoc‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
---
2+
title: Customizing this pattern
3+
weight: 20
4+
aliases: /lemonade-stand-quickstart/customizing/
5+
---
6+
7+
:toc:
8+
:imagesdir: /images
9+
:_content-type: ASSEMBLY
10+
include::modules/comm-attributes.adoc[]
11+
12+
[id="customizing-lemonade-stand-quickstart"]
13+
== Customizing the Lemonade Stand AI Quickstart pattern
14+
15+
This pattern deploys an AI chatbot with a multi-layered guardrails pipeline, including model-based detectors, a rule-based language detector, and regex-based competitor filtering. You can customize the LLM model, detector configuration, and monitoring settings.
16+
17+
[id="changing-model-lemonade-stand"]
18+
=== Changing the LLM model
19+
20+
The pattern serves Llama 3.2 3B Instruct (FP8-quantized) by default through vLLM on KServe. The model is defined in the lemonade-stand-assistant Helm chart's `values.yaml`.
21+
22+
To change the locally served model, update the model configuration in the Helm chart values. The model must be compatible with vLLM and fit within the available GPU VRAM on the provisioned node (NVIDIA A10G with 24 GB VRAM on `g5.2xlarge`).
23+
24+
[id="using-external-model-lemonade-stand"]
25+
=== Using an external model endpoint (BYOM)
26+
27+
Instead of serving a model locally on GPU, you can configure the pattern to use an external Model-as-a-Service endpoint. This eliminates the GPU node requirement for inference.
28+
29+
. Make a local copy of the secrets template outside of your repository:
30+
+
31+
[WARNING]
32+
====
33+
Do not add, commit, or push this file to your repository. Doing so might expose personal credentials to GitHub.
34+
====
35+
+
36+
[source,terminal]
37+
----
38+
$ cp values-secret.yaml.template ~/values-secret-ai-quickstart-lemonade-stand.yaml
39+
----
40+
41+
. Edit the secrets file and set the API key for your external model endpoint:
42+
+
43+
[source,terminal]
44+
----
45+
$ vim ~/values-secret-ai-quickstart-lemonade-stand.yaml
46+
----
47+
+
48+
[source,yaml]
49+
----
50+
- name: lemonade-stand
51+
vaultPrefixes:
52+
- global
53+
fields:
54+
- name: vllm-api-key
55+
value: <your-external-api-key>
56+
----
57+
58+
. Set the `model` section in the Helm chart values to point to your external endpoint:
59+
+
60+
[source,yaml]
61+
----
62+
model:
63+
name: my-model
64+
endpoint: my-maas-instance
65+
port: 443
66+
----
67+
68+
When using an external model endpoint, the vLLM InferenceService is not deployed and the GPU node is not required for LLM inference. The guardrails pipeline continues to function normally with the external model.
69+
70+
[id="enabling-gpu-detectors-lemonade-stand"]
71+
=== Enabling GPU for detector models
72+
73+
By default, the HAP and prompt injection detector models run on CPU. You can enable GPU acceleration for these models to reduce inference latency, but this requires additional GPU resources.
74+
75+
To enable GPU for the detector models, set the `useGpu` flag in the Helm chart values:
76+
77+
[source,yaml]
78+
----
79+
detectors:
80+
hap:
81+
useGpu: true
82+
promptInjection:
83+
useGpu: true
84+
----
85+
86+
[NOTE]
87+
====
88+
Enabling GPU for both detectors requires 2 additional GPUs beyond the 1 GPU used for the LLM, for a total of 3 GPUs. You must provision additional GPU nodes before enabling this option.
89+
====
90+
91+
[id="configuring-detector-thresholds-lemonade-stand"]
92+
=== Configuring detector thresholds
93+
94+
The guardrails pipeline uses three detector models, each with a configurable detection threshold. Lower thresholds increase sensitivity (block more content) while higher thresholds reduce false positives.
95+
96+
The default thresholds are:
97+
98+
[cols="1,1,2",options="header"]
99+
|===
100+
| Detector | Default threshold | Description
101+
102+
| IBM Granite Guardian HAP
103+
| 0.5
104+
| Hate speech, abuse, and profanity detection
105+
106+
| DeBERTa v3 Prompt Injection
107+
| 0.5
108+
| Prompt injection and jailbreak detection
109+
110+
| Lingua Language
111+
| 0.88
112+
| English language confidence threshold
113+
|===
114+
115+
To adjust detector thresholds, modify the Guardrails Orchestrator configuration in the `fms-orchestr8-config-nlp` ConfigMap within the lemonade-stand-assistant Helm chart.
116+
117+
[id="configuring-regex-detector-lemonade-stand"]
118+
=== Configuring the regex detector
119+
120+
The FastAPI application includes a regex-based detector that blocks mentions of competitor fruit names (oranges, apples, bananas, and others) across 13+ languages. This detector runs locally in the application before the request reaches the Guardrails Orchestrator.
121+
122+
To modify the blocked terms or supported languages, edit the regex patterns in the `app_fastapi.py` file in the lemonade-stand-assistant repository.
123+
124+
[id="configuring-shiny-dashboard-lemonade-stand"]
125+
=== Adjusting the monitoring dashboard
126+
127+
The R Shiny dashboard polls the FastAPI application's `/metrics` endpoint to display guardrail activation statistics in real time. The default polling interval is 1 second.
128+
129+
To adjust the refresh interval, modify the `shinyDashboard.metrics.refreshInterval` value in the Helm chart values:
130+
131+
[source,yaml]
132+
----
133+
shinyDashboard:
134+
metrics:
135+
refreshInterval: 5
136+
----
137+
138+
Push your changes to your forked repository so the GitOps framework applies the updated configuration.

‎content/patterns/lemonade-stand-quickstart/getting-started.adoc‎

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
---
2+
title: Getting started
3+
weight: 10
4+
aliases: /lemonade-stand-quickstart/getting-started/
5+
---
6+
7+
:toc:
8+
:imagesdir: /images
9+
:_content-type: ASSEMBLY
10+
include::modules/comm-attributes.adoc[]
11+
12+
[id="deploying-lemonade-stand-quickstart-pattern"]
13+
== Deploying the Lemonade Stand AI Quickstart pattern
14+
15+
.Prerequisites
16+
17+
* An OpenShift cluster (version 4.18 or later). This pattern requires at least 1 NVIDIA GPU node for LLM inference.
18+
** *AWS*: The pattern automatically provisions 1 `g5.2xlarge` GPU worker node (NVIDIA A10G) during installation. No GPU nodes need to be present before you deploy.
19+
** *Other providers and bare metal*: A GPU node must already be part of the OpenShift cluster before you deploy this pattern. The pattern installs all required operators automatically.
20+
** To create an OpenShift cluster, go to the https://console.redhat.com/[Red Hat Hybrid Cloud console].
21+
** Select *OpenShift \-> Red Hat OpenShift Container Platform \-> Create cluster*.
22+
* The Helm binary. For instructions, see link:https://helm.sh/docs/intro/install/[Installing Helm].
23+
* The `oc` CLI tool. For instructions, see link:https://docs.openshift.com/container-platform/latest/cli_reference/openshift_cli/getting-started-cli.html[Getting started with the OpenShift CLI].
24+
* Additional installation tool dependencies. For details, see link:https://validatedpatterns.io/learn/quickstart/[Patterns quick start].
25+
26+
[id="preparing-for-deployment-lemonade-stand"]
27+
== Preparing for deployment
28+
.Procedure
29+
30+
. Fork the link:https://github.com/validatedpatterns-sandbox/ai-quickstart-lemonade-stand[ai-quickstart-lemonade-stand] repository on GitHub. You must fork the repository to customize this pattern.
31+
32+
. Clone the forked copy of this repository.
33+
+
34+
[source,terminal]
35+
----
36+
$ git clone git@github.com:your-username/ai-quickstart-lemonade-stand.git
37+
----
38+
39+
. Go to the root directory of your Git repository:
40+
+
41+
[source,terminal]
42+
----
43+
$ cd ai-quickstart-lemonade-stand
44+
----
45+
46+
. Run the following command to set the upstream repository:
47+
+
48+
[source,terminal]
49+
----
50+
$ git remote add -f upstream git@github.com:validatedpatterns-sandbox/ai-quickstart-lemonade-stand.git
51+
----
52+
53+
. Verify the setup of your remote repositories by running the following command:
54+
+
55+
[source,terminal]
56+
----
57+
$ git remote -v
58+
----
59+
+
60+
.Example output
61+
+
62+
[source,terminal]
63+
----
64+
origin git@github.com:your-username/ai-quickstart-lemonade-stand.git (fetch)
65+
origin git@github.com:your-username/ai-quickstart-lemonade-stand.git (push)
66+
upstream git@github.com:validatedpatterns-sandbox/ai-quickstart-lemonade-stand.git (fetch)
67+
upstream git@github.com:validatedpatterns-sandbox/ai-quickstart-lemonade-stand.git (push)
68+
----
69+
70+
. Optional: To customize the deployment, create and switch to a new branch by running the following command:
71+
+
72+
[source,terminal]
73+
----
74+
$ git checkout -b my-branch
75+
----
76+
+
77+
Make your changes, then stage and commit them:
78+
+
79+
[source,terminal]
80+
----
81+
$ git add <changed-files>
82+
$ git commit -m "Customize deployment"
83+
----
84+
+
85+
Push the changes to your forked repository:
86+
+
87+
[source,terminal]
88+
----
89+
$ git push origin my-branch
90+
----
91+
92+
[id="deploying-cluster-using-patternsh-file-lemonade-stand"]
93+
== Deploying the pattern by using the pattern.sh file
94+
95+
To deploy the pattern by using the `pattern.sh` file, complete the following steps:
96+
97+
. Log in to your cluster by following this procedure:
98+
99+
.. Obtain an API token by visiting link:https://oauth-openshift.apps.<your_cluster>.<domain>/oauth/token/request[https://oauth-openshift.apps.<your_cluster>.<domain>/oauth/token/request].
100+
101+
.. Log in to the cluster by running the following command:
102+
+
103+
[source,terminal]
104+
----
105+
$ oc login --token=<retrieved-token> --server=https://api.<your_cluster>.<domain>:6443
106+
----
107+
+
108+
Or log in by running the following command:
109+
+
110+
[source,terminal]
111+
----
112+
$ export KUBECONFIG=~/<path_to_kubeconfig>
113+
----
114+
115+
. Deploy the pattern to your cluster. Run the following command:
116+
+
117+
[source,terminal]
118+
----
119+
$ ./pattern.sh make install
120+
----
121+
122+
.Verification
123+
124+
To verify a successful installation, check the health of the ArgoCD applications:
125+
126+
. Run the following command:
127+
+
128+
[source,terminal]
129+
----
130+
$ ./pattern.sh make argo-healthcheck
131+
----
132+
+
133+
It might take several minutes for all applications to synchronize and reach a healthy state. This includes downloading detector models, initializing the GPU operator, and starting the vLLM inference service.
134+
135+
. Verify that the Operators are installed by navigating to *Operators -> Installed Operators* in the {ocp} web console. Confirm the following Operators are present:
136+
+
137+
* NVIDIA GPU Operator
138+
* {rhoai}
139+
* Node Feature Discovery Operator
140+
* External Secrets Operator
141+
142+
. After all applications are healthy, verify the inference service is serving by running:
143+
+
144+
[source,terminal]
145+
----
146+
$ oc get inferenceservice -A
147+
----
148+
149+
. Access the Lemonade Stand chatbot UI. Navigate to *Networking -> Routes* in the `lemonade-stand` namespace and open the route URL for the `lemonade-stand` service.
150+
151+
. Access the R Shiny monitoring dashboard. Navigate to *Networking -> Routes* in the `lemonade-stand` namespace and open the route URL for the `shiny-dashboard` service.
152+
153+
[id="next-steps-getting-started-lemonade-stand"]
154+
== Next steps
155+
156+
* link:customizing-this-pattern[Customizing this pattern]
157+
* link:cluster-sizing[Cluster sizing]
158+
* link:troubleshooting[Troubleshooting]

0 commit comments

Comments
 (0)