From 70741da64b4b8a42f40f2ebe9301b2d2b92454af Mon Sep 17 00:00:00 2001 From: ZePan110 Date: Wed, 4 Jun 2025 15:25:30 +0800 Subject: [PATCH 1/2] Refine README.md Signed-off-by: ZePan110 --- DBQnA/README.md | 22 +- DBQnA/docker_compose/amd/gpu/rocm/README.md | 145 ++++++------- DBQnA/docker_compose/intel/cpu/xeon/README.md | 205 +++++++----------- 3 files changed, 159 insertions(+), 213 deletions(-) diff --git a/DBQnA/README.md b/DBQnA/README.md index c13f2793b4..fa16f338f5 100644 --- a/DBQnA/README.md +++ b/DBQnA/README.md @@ -2,6 +2,15 @@ Experience a revolutionary way to interact with your database using our DBQnA app! Harnessing the power of OPEA microservices, our application seamlessly translates natural language queries into SQL and delivers real-time database results, all designed to optimize workflows and enhance productivity for modern enterprises. +## Table of contents + +1. [Architecture](#architecture) +2. [Deployment Options](#deployment-options) + +## Architecture + +The flow chart below shows the information flow between different microservices for this example. + --- ```mermaid @@ -46,14 +55,15 @@ flowchart LR --- -## 🛠️ Key Features - -### 💬 SQL Query Generation - The key feature of DBQnA app is that it converts a user's natural language query into an SQL query and automatically executes the generated SQL query on the database to return the relevant results. Basically ask questions to database, receive corresponding SQL query and real-time query execution output, all without needing any SQL knowledge. --- -## 📚 Setup Guide +## Deployment Options + +The table below lists the available deployment options and their implementation details for different hardware platforms. -- **[Xeon Guide](./docker_compose/intel/cpu/xeon/README.md)**: Instructions to build Docker images from source and run the application via Docker Compose. +| Platform | Deployment Method | Link | +| ------------ | ----------------- | ----------------------------------------------------------------- | +| Intel Xeon | Docker compose | [Deployment on Xeon](./docker_compose/intel/cpu/xeon/README.md) | +| AMD ROCm | Docker compose | [Deployment on AMD Rocm](./docker_compose/amd/gpu/rocm/README.md) | diff --git a/DBQnA/docker_compose/amd/gpu/rocm/README.md b/DBQnA/docker_compose/amd/gpu/rocm/README.md index 014d5722c4..f712c0ad24 100644 --- a/DBQnA/docker_compose/amd/gpu/rocm/README.md +++ b/DBQnA/docker_compose/amd/gpu/rocm/README.md @@ -1,98 +1,75 @@ -# Deploy on AMD GPU +# Example DBQnA Deployment on AMD GPU (ROCm) -This document outlines the deployment process for DBQnA application which helps generating a SQL query and its output given a NLP question, utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on an AMD GPU. The steps include Docker image creation, container deployment via Docker Compose, and service execution to integrate microservices. We will publish the Docker images to Docker Hub soon, which will simplify the deployment process for this service. +This document outlines the deployment process for DBQnA application which helps generating a SQL query and its output given a NLP question, utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on an AMD GPU. This example includes the following sections: +- [DBQnA Quick Start Deployment](#dbqna-quick-start-deployment): Demonstrates how to quickly deploy a DBQnA service/pipeline on AMD GPU (ROCm). +- [DBQnA Docker Compose Files](#dbqna-docker-compose-files): Describes some example deployments and their docker compose files. -## 🚀 Build Docker Images +## DBQnA Quick Start Deployment -First of all, you need to build Docker Images locally. This step can be ignored once the Docker images are published to Docker hub. +This section describes how to quickly deploy and test the DBQnA service manually on AMD GPU (ROCm). The basic steps are: -### 1.1 Build Text to SQL service Image +1. [Access the Code](#access-the-code) +2. [Generate a HuggingFace Access Token](#generate-a-huggingface-access-token) +3. [Configure the Deployment Environment](#configure-the-deployment-environment) +4. [Deploy the Service Using Docker Compose](#deploy-the-service-using-docker-compose) +5. [Check the Deployment Status](#check-the-deployment-status) +6. [Test the Pipeline](#test-the-pipeline) +7. [Cleanup the Deployment](#cleanup-the-deployment) -```bash -git clone https://github.com/opea-project/GenAIComps.git -cd GenAIComps -docker build --no-cache -t opea/texttosql:latest -f comps/text2sql/src/Dockerfile . -``` +### Access the Code -### 1.2 Build react UI Docker Image +Clone the GenAIExample repository and access the DBQnA AMD GPU (ROCm) Docker Compose files and supporting scripts: -Build the frontend Docker image based on react framework via below command: - -```bash -cd GenAIExamples/DBQnA/ui -docker build --no-cache -t opea/dbqna-react-ui:latest --build-arg texttosql_url=$textToSql_host:$textToSql_port/v1 -f docker/Dockerfile.react . +``` +git clone https://github.com/opea-project/GenAIExamples.git +cd GenAIExamples/DBQnA/docker_compose/ ``` -Attention! Replace $textToSql_host and $textToSql_port with your own value. - -Then run the command `docker images`, you will have the following Docker Images: - -1. `opea/texttosql:latest` -2. `opea/dbqna-react-ui:latest` - -## 🚀 Start Microservices +Checkout a released version, such as v1.3: -### Required Models +``` +git checkout v1.3 +``` -We set default model as "mistralai/Mistral-7B-Instruct-v0.3", change "LLM_MODEL_ID" in following Environment Variables setting if you want to use other models. +### Generate a HuggingFace Access Token -If use gated models, you also need to provide [huggingface token](https://huggingface.co/docs/hub/security-tokens) to "HUGGINGFACEHUB_API_TOKEN" environment variable. +Some HuggingFace resources, such as some models, are only accessible if you have an access token. If you do not already have a HuggingFace access token, you can create one by first creating an account by following the steps provided at [HuggingFace](https://huggingface.co/) and then generating a [user access token](https://huggingface.co/docs/transformers.js/en/guides/private#step-1-generating-a-user-access-token). + Docker image creation, container deployment via Docker Compose, and service execution to integrate microservices. We will publish the Docker images to Docker Hub soon, which will simplify the deployment process for this service. -### 2.1 Setup Environment Variables +### Configure the Deployment Environment -Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. +To set up environment variables for deploying DBQnA service, source the _set_env.sh_ script in this directory: -```bash -export host_ip="host_ip_address_or_dns_name" -export DBQNA_HUGGINGFACEHUB_API_TOKEN="" -export DBQNA_TGI_SERVICE_PORT=8008 -export DBQNA_TGI_LLM_ENDPOINT="http://${host_ip}:${DBQNA_TGI_SERVICE_PORT}" -export DBQNA_LLM_MODEL_ID="mistralai/Mistral-7B-Instruct-v0.3" -export MODEL_ID="mistralai/Mistral-7B-Instruct-v0.3" -export POSTGRES_USER="postgres" -export POSTGRES_PASSWORD="testpwd" -export POSTGRES_DB="chinook" -export DBQNA_TEXT_TO_SQL_PORT=18142 -export DBQNA_UI_PORT=18143 +``` +source amd/gpu/rocm/set_env.sh ``` -Note: Please replace with `host_ip_address_or_dns_name` with your external IP address or DNS name, do not use localhost. - -### 2.2 Start Microservice Docker Containers +The _set_env.sh_ script will prompt for required and optional environment variables used to configure the DBQnA service based on TGI. If a value is not entered, the script will use a default value for the same. It will also generate a _.env_ file defining the desired configuration. -There are 2 options to start the microservice +### Deploy the Service Using Docker Compose -#### 2.2.1 Start the microservice using docker compose +To deploy the DBQnA service, execute the `docker compose up` command with the appropriate arguments. For a default deployment, execute: ```bash -cd GenAIExamples/DBQnA/docker_compose/amd/gpu/rocm -docker compose up -d +cd amd/gpu/rocm/ +docker compose -f compose.yaml up -d ``` -## 🚀 Validate Microservices +The DBQnA docker images should automatically be downloaded from the `OPEA registry` and deployed on the AMD GPU (ROCm) -### 3.1 TGI Service +### Check the Deployment Status + +After running docker compose, check if all the containers launched via docker compose have started: -```bash -curl http://${host_ip}:$DBQNA_TGI_SERVICE_PORT/generate \ - -X POST \ - -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":17, "do_sample": true}}' \ - -H 'Content-Type: application/json' +``` +docker ps -a ``` -### 3.2 Postgres Microservice +For the default deployment, the following 4 containers should be running. -Once Text-to-SQL microservice is started, user can use below command +### Test the Pipeline -#### 3.2.1 Test the Database connection - -```bash -curl --location http://${host_ip}:${DBQNA_TEXT_TO_SQL_PORT}/v1/postgres/health \ - --header 'Content-Type: application/json' \ - --data '{"user": "'${POSTGRES_USER}'","password": "'${POSTGRES_PASSWORD}'","host": "'${host_ip}'", "port": "5442", "database": "'${POSTGRES_DB}'"}' -``` - -#### 3.2.2 Invoke the microservice. +Once the DBQnA service are running, test the pipeline using the following command: ```bash curl http://${host_ip}:${DBQNA_TEXT_TO_SQL_PORT}/v1/texttosql \ @@ -101,26 +78,34 @@ curl http://${host_ip}:${DBQNA_TEXT_TO_SQL_PORT}/v1/texttosql \ -H 'Content-Type: application/json' ``` -### 3.3 Frontend validation +### Cleanup the Deployment -We test the API in frontend validation to check if API returns HTTP_STATUS: 200 and validates if API response returns SQL query and output +To stop the containers associated with the deployment, execute the following command: -The test is present in App.test.tsx under react root folder ui/react/ +``` +docker compose -f compose.yaml down +``` -Command to run the test +All the DBQnA containers will be stopped and then removed on completion of the "down" command. -```bash -npm run test -``` +## DBQnA Docker Compose Files -## 🚀 Launch the React UI +The compose.yaml is default compose file using tgi as serving framework -Open this URL `http://${host_ip}:${DBQNA_UI_PORT}` in your browser to access the frontend. +| Service Name | Image Name | +| -------------------------- | -------------------------------------------------------- | +| dbqna-tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.1-rocm | +| postgres | postgres:latest | +| text2sql | opea/text2sql:latest | +| text2sql-react-ui | opea/text2sql-react-ui:latest | -![project-screenshot](../../../../assets/img/dbQnA_ui_init.png) +## DBQnA Service Configuration for AMD GPUs -Test DB Connection -![project-screenshot](../../../../assets/img/dbQnA_ui_successful_db_connection.png) +The table provides a comprehensive overview of the DBQnA service utilized across various deployments as illustrated in the example Docker Compose files. Each row in the table represents a distinct service, detailing its possible images used to enable it and a concise description of its function within the deployment architecture. -Create SQL query and output for given NLP question -![project-screenshot](../../../../assets/img/dbQnA_ui_succesful_sql_output_generation.png) +| Service Name | Possible Image Names | Optional | Description | +| -------------------------- | -------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------- | +| dbqna-tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.1-rocm | No | Specific to the TGI deployment, focuses on text generation inference using AMD GPU (ROCm) hardware. | +| postgres | postgres:latest | No | Provides the relational database backend for storing and querying data used by the DBQnA pipeline. | +| text2sql | opea/text2sql:latest | No | Handles text-to-SQL conversion tasks. | +| text2sql-react-ui | opea/text2sql-react-ui:latest | No | Provides the user interface for the DBQnA service. | diff --git a/DBQnA/docker_compose/intel/cpu/xeon/README.md b/DBQnA/docker_compose/intel/cpu/xeon/README.md index c227e1fe46..5f80a5a96f 100644 --- a/DBQnA/docker_compose/intel/cpu/xeon/README.md +++ b/DBQnA/docker_compose/intel/cpu/xeon/README.md @@ -1,156 +1,90 @@ -# Deploy on Intel Xeon Processor +# Example DBQnA Deployment on Intel® Xeon® Platform -This document outlines the deployment process for DBQnA application which helps generating a SQL query and its output given a NLP question, utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on an Intel Xeon server. The steps include Docker image creation, container deployment via Docker Compose, and service execution to integrate microservices. We will publish the Docker images to Docker Hub soon, which will simplify the deployment process for this service. +This document outlines the deployment process for DBQnA application which helps generating a SQL query and its output given a NLP question, utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on an Intel Xeon server. +- [DBQnA Quick Start Deployment](#dbqna-quick-start-deployment): Demonstrates how to quickly deploy a DBQnA service/pipeline on Intel® Xeon® platform. +- [DBQnA Docker Compose Files](#dbqna-docker-compose-files): Describes some example deployments and their docker compose files. -## 🚀 Build Docker Images +## DBQnA Quick Start Deployment -First of all, you need to build Docker Images locally. This step can be ignored once the Docker images are published to Docker hub. +This section describes how to quickly deploy and test the DBQnA service manually on Intel® Xeon® platform. The basic steps are: -### 1.1 Build Text to SQL service Image +1. [Access the Code](#access-the-code) +2. [Generate a HuggingFace Access Token](#generate-a-huggingface-access-token) +3. [Configure the Deployment Environment](#configure-the-deployment-environment) +4. [Deploy the Service Using Docker Compose](#deploy-the-service-using-docker-compose) +5. [Check the Deployment Status](#check-the-deployment-status) +6. [Test the Pipeline](#test-the-pipeline) +7. [Cleanup the Deployment](#cleanup-the-deployment) -```bash -git clone https://github.com/opea-project/GenAIComps.git -cd GenAIComps -docker build --no-cache -t opea/text2sql:comps -f comps/text2sql/src/Dockerfile . - -``` - -### 1.2 Build react UI Docker Image +### Access the Code -Build the frontend Docker image based on react framework via below command: - -```bash -cd GenAIExamples/DBQnA/ui -docker build --no-cache -t opea/text2sql-react-ui:latest -f docker/Dockerfile.react . +Clone the GenAIExample repository and access the DBQnA Intel® Xeon® platform Docker Compose files and supporting scripts: ``` - -Then run the command `docker images`, you will have the following Docker Images: - -1. `opea/text2sql:latest` -2. `opea/dbqna-react-ui:latest` - -## 🚀 Start Microservices - -### Required Models - -We set default model as "mistralai/Mistral-7B-Instruct-v0.3", change "LLM_MODEL_ID" in following Environment Variables setting if you want to use other models. - -If use gated models, you also need to provide [huggingface token](https://huggingface.co/docs/hub/security-tokens) to "HUGGINGFACEHUB_API_TOKEN" environment variable. - -```bash -export HUGGINGFACEHUB_API_TOKEN="xxx" +git clone https://github.com/opea-project/GenAIExamples.git +cd GenAIExamples/DBQnA/docker_compose/intel/cpu/xeon/ ``` -### 2.1 Setup Environment Variables +Checkout a released version, such as v1.3: -Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. +``` +git checkout v1.3 +``` -```bash -# host_ip should be your external IP address, do not use localhost. -export host_ip=$(hostname -I | awk '{print $1}') +### Generate a HuggingFace Access Token -# Example: no_proxy="localhost,127.0.0.1,192.168.1.1" -export no_proxy=${no_proxy},${host_ip} +Some HuggingFace resources, such as some models, are only accessible if you have an access token. If you do not already have a HuggingFace access token, you can create one by first creating an account by following the steps provided at [HuggingFace](https://huggingface.co/) and then generating a [user access token](https://huggingface.co/docs/transformers.js/en/guides/private#step-1-generating-a-user-access-token). -# If you are in a proxy environment, also set the proxy-related environment variables: -export http_proxy=${http_proxy} -export https_proxy=${https_proxy} +### Configure the Deployment Environment -# Set other required variables +To set up environment variables for deploying DBQnA service, source the set_env.sh script in this directory: -export TGI_PORT=8008 -export TGI_LLM_ENDPOINT=http://${host_ip}:${TGI_PORT} -export HF_TOKEN=${HUGGINGFACEHUB_API_TOKEN} -export LLM_MODEL_ID="mistralai/Mistral-7B-Instruct-v0.3" -export POSTGRES_USER=postgres -export POSTGRES_PASSWORD=testpwd -export POSTGRES_DB=chinook -export text2sql_port=9090 ``` - -or -edit the file set_env.sh to set those environment variables, - -```bash -cd GenAIExamples/DBQnA/docker_compose/intel/cpu/xeon/ source set_env.sh ``` -Note: Please replace with `host_ip` with your external IP address, do not use localhost. - -### 2.2 Start Microservice Docker Containers +The set_env.sh script will prompt for required and optional environment variables used to configure the DBQnA service. If a value is not entered, the script will use a default value for the same. It will also generate a _.env_ file defining the desired configuration. -There are 2 options to start the microservice +### Deploy the Service Using Docker Compose -#### 2.2.1 Start the microservice using docker compose +To deploy the DBQnA service, execute the `docker compose up` command with the appropriate arguments. For a default deployment, execute: ```bash -cd GenAIExamples/DBQnA/docker_compose/intel/cpu/xeon docker compose up -d ``` -#### 2.2.2 Alternatively we can start the microservices by running individual docker services - -**NOTE:** Make sure all the individual docker services are down before starting them. - -Below are the commands to start each of the docker service individually +The DBQnA docker images should automatically be downloaded from the `OPEA registry` and deployed on the Intel® Xeon® Platform: -- Start PostgresDB Service - -We will use [Chinook](https://github.com/lerocha/chinook-database) sample database as a default to test the Text-to-SQL microservice. Chinook database is a sample database ideal for demos and testing ORM tools targeting single and multiple database servers. - -```bash - -docker run --name test-text2sql-postgres --ipc=host -e POSTGRES_USER=${POSTGRES_USER} -e POSTGRES_HOST_AUTH_METHOD=trust -e POSTGRES_DB=${POSTGRES_DB} -e POSTGRES_PASSWORD=${POSTGRES_PASSWORD} -p 5442:5432 -d -v $WORKPATH/comps/text2sql/langchain/chinook.sql:/docker-entrypoint-initdb.d/chinook.sql postgres:latest ``` - -- Start TGI Service - -```bash - -docker run -d --name="test-text2sql-tgi-endpoint" --ipc=host -p $TGI_PORT:80 -v ./data:/data --shm-size 1g -e HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN} -e HF_TOKEN=${HF_TOKEN} -e model=${model} ghcr.io/huggingface/text-generation-inference:2.4.1 --model-id $model +[+] Running 5/5 + ✔ Network xeon_default Created 0.1s + ✔ Container tgi-service Started 5.9s + ✔ Container postgres-container Started 5.8s + ✔ Container text2sql-service Started 6.0s + ✔ Container dbqna-xeon-react-ui-server Started 0.9s ``` -- Start Text-to-SQL Service +### Check the Deployment Status -```bash -unset http_proxy +After running docker compose, check if all the containers launched via docker compose have started: -docker run -d --name="test-text2sql-server" --ipc=host -p ${text2sql_port}:8090 --ipc=host -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e TGI_LLM_ENDPOINT=$TGI_LLM_ENDPOINT opea/text2sql:latest ``` - -- Start React UI service - -```bash -docker run -d --name="test-dbqna-react-ui-server" --ipc=host -p 5174:80 -e no_proxy=$no_proxy -e https_proxy=$https_proxy -e http_proxy=$http_proxy opea/dbqna-react-ui:latest +docker ps -a ``` -## 🚀 Validate Microservices +For the default deployment, the following 5 containers should be running: -### 3.1 TGI Service - -```bash - -curl http://${host_ip}:$TGI_PORT/generate \ - -X POST \ - -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":17, "do_sample": true}}' \ - -H 'Content-Type: application/json' ``` - -### 3.2 Postgres Microservice - -Once Text-to-SQL microservice is started, user can use below command - -#### 3.2.1 Test the Database connection - -```bash -curl --location http://${host_ip}:9090/v1/postgres/health \ - --header 'Content-Type: application/json' \ - --data '{"user": "'${POSTGRES_USER}'","password": "'${POSTGRES_PASSWORD}'","host": "'${host_ip}'", "port": "5442", "database": "'${POSTGRES_DB}'"}' +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +2728db31368b opea/text2sql-react-ui:latest "nginx -g 'daemon of…" 9 minutes ago Up 9 minutes 0.0.0.0:5174->80/tcp, :::5174->80/tcp dbqna-xeon-react-ui-server +0ab75b92c300 postgres:latest "docker-entrypoint.s…" 9 minutes ago Up 9 minutes 0.0.0.0:5442->5432/tcp, :::5442->5432/tcp postgres-container +2662a69b515b ghcr.io/huggingface/text-generation-inference:2.4.0-intel-cpu "text-generation-lau…" 9 minutes ago Up 9 minutes 0.0.0.0:8008->80/tcp, :::8008->80/tcp tgi-service +bb44512be80e opea/text2sql:latest "python opea_text2sq…" 9 minutes ago Up 9 minutes 0.0.0.0:9090->8080/tcp, :::9090->8080/tcp text2sql-service ``` -#### 3.2.2 Invoke the microservice. +### Test the Pipeline + +Once the DBQnA service are running, test the pipeline using the following command: ```bash curl http://${host_ip}:9090/v1/text2sql\ @@ -159,26 +93,43 @@ curl http://${host_ip}:9090/v1/text2sql\ -H 'Content-Type: application/json' ``` -### 3.3 Frontend validation - -We test the API in frontend validation to check if API returns HTTP_STATUS: 200 and validates if API response returns SQL query and output +### Cleanup the Deployment -The test is present in App.test.tsx under react root folder ui/react/ +To stop the containers associated with the deployment, execute the following command: -Command to run the test +``` +docker compose -f compose.yaml down +``` -```bash -npm run test ``` +[+] Running 5/5 + ✔ Container postgres-container Removed 0.5s + ✔ Container tgi-service Removed 2.9s + ✔ Container dbqna-xeon-react-ui-server Removed 0.6s + ✔ Container text2sql-service Removed 10.3s + ✔ Network xeon_default Removed 0.4s +``` + +All the DBQnA containers will be stopped and then removed on completion of the "down" command. + +## DBQnA Docker Compose Files -## 🚀 Launch the React UI +The compose.yaml is default compose file using tgi as serving framework -Open this URL `http://{host_ip}:5174` in your browser to access the frontend. +| Service Name | Image Name | +| -------------------------- | ------------------------------------------------------------- | +| tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.0-intel-cpu | +| postgres | postgres:latest | +| text2sql | opea/text2sql:latest | +| dbqna-xeon-react-ui-server | opea/text2sql-react-ui:latest | -![project-screenshot](../../../../assets/img/dbQnA_ui_init.png) +## DBQnA Service Configuration -Test DB Connection -![project-screenshot](../../../../assets/img/dbQnA_ui_successful_db_connection.png) +The table provides a comprehensive overview of the DBQnA service utilized across various deployments as illustrated in the example Docker Compose files. Each row in the table represents a distinct service, detailing its possible images used to enable it and a concise description of its function within the deployment architecture. -Create SQL query and output for given NLP question -![project-screenshot](../../../../assets/img/dbQnA_ui_succesful_sql_output_generation.png) +| Service Name | Possible Image Names | Optional | Description | +| -------------------------- | ------------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------- | +| tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.0-intel-cpu | No | Specific to the TGI deployment, focuses on text generation inference using AMD GPU (ROCm) hardware. | +| postgres | postgres:latest | No | Provides the relational database backend for storing and querying data used by the DBQnA pipeline. | +| text2sql | opea/text2sql:latest | No | Handles text-to-SQL conversion tasks. | +| dbqna-xeon-react-ui-server | opea/text2sql-react-ui:latest | No | Provides the user interface for the DBQnA service. | \ No newline at end of file From ecba8945a0877fd189bd39addb311b0ee9b2d7d9 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 5 Jun 2025 05:23:16 +0000 Subject: [PATCH 2/2] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- DBQnA/README.md | 8 ++--- DBQnA/docker_compose/amd/gpu/rocm/README.md | 29 ++++++++++--------- DBQnA/docker_compose/intel/cpu/xeon/README.md | 9 +++--- 3 files changed, 24 insertions(+), 22 deletions(-) diff --git a/DBQnA/README.md b/DBQnA/README.md index fa16f338f5..86f3256a16 100644 --- a/DBQnA/README.md +++ b/DBQnA/README.md @@ -63,7 +63,7 @@ The key feature of DBQnA app is that it converts a user's natural language query The table below lists the available deployment options and their implementation details for different hardware platforms. -| Platform | Deployment Method | Link | -| ------------ | ----------------- | ----------------------------------------------------------------- | -| Intel Xeon | Docker compose | [Deployment on Xeon](./docker_compose/intel/cpu/xeon/README.md) | -| AMD ROCm | Docker compose | [Deployment on AMD Rocm](./docker_compose/amd/gpu/rocm/README.md) | +| Platform | Deployment Method | Link | +| ---------- | ----------------- | ----------------------------------------------------------------- | +| Intel Xeon | Docker compose | [Deployment on Xeon](./docker_compose/intel/cpu/xeon/README.md) | +| AMD ROCm | Docker compose | [Deployment on AMD Rocm](./docker_compose/amd/gpu/rocm/README.md) | diff --git a/DBQnA/docker_compose/amd/gpu/rocm/README.md b/DBQnA/docker_compose/amd/gpu/rocm/README.md index f712c0ad24..f15b5db2b4 100644 --- a/DBQnA/docker_compose/amd/gpu/rocm/README.md +++ b/DBQnA/docker_compose/amd/gpu/rocm/README.md @@ -1,6 +1,7 @@ # Example DBQnA Deployment on AMD GPU (ROCm) This document outlines the deployment process for DBQnA application which helps generating a SQL query and its output given a NLP question, utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on an AMD GPU. This example includes the following sections: + - [DBQnA Quick Start Deployment](#dbqna-quick-start-deployment): Demonstrates how to quickly deploy a DBQnA service/pipeline on AMD GPU (ROCm). - [DBQnA Docker Compose Files](#dbqna-docker-compose-files): Describes some example deployments and their docker compose files. @@ -34,7 +35,7 @@ git checkout v1.3 ### Generate a HuggingFace Access Token Some HuggingFace resources, such as some models, are only accessible if you have an access token. If you do not already have a HuggingFace access token, you can create one by first creating an account by following the steps provided at [HuggingFace](https://huggingface.co/) and then generating a [user access token](https://huggingface.co/docs/transformers.js/en/guides/private#step-1-generating-a-user-access-token). - Docker image creation, container deployment via Docker Compose, and service execution to integrate microservices. We will publish the Docker images to Docker Hub soon, which will simplify the deployment process for this service. +Docker image creation, container deployment via Docker Compose, and service execution to integrate microservices. We will publish the Docker images to Docker Hub soon, which will simplify the deployment process for this service. ### Configure the Deployment Environment @@ -44,7 +45,7 @@ To set up environment variables for deploying DBQnA service, source the _set_env source amd/gpu/rocm/set_env.sh ``` -The _set_env.sh_ script will prompt for required and optional environment variables used to configure the DBQnA service based on TGI. If a value is not entered, the script will use a default value for the same. It will also generate a _.env_ file defining the desired configuration. +The _set_env.sh_ script will prompt for required and optional environment variables used to configure the DBQnA service based on TGI. If a value is not entered, the script will use a default value for the same. It will also generate a _.env_ file defining the desired configuration. ### Deploy the Service Using Docker Compose @@ -92,20 +93,20 @@ All the DBQnA containers will be stopped and then removed on completion of the " The compose.yaml is default compose file using tgi as serving framework -| Service Name | Image Name | -| -------------------------- | -------------------------------------------------------- | -| dbqna-tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.1-rocm | -| postgres | postgres:latest | -| text2sql | opea/text2sql:latest | -| text2sql-react-ui | opea/text2sql-react-ui:latest | +| Service Name | Image Name | +| ----------------- | -------------------------------------------------------- | +| dbqna-tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.1-rocm | +| postgres | postgres:latest | +| text2sql | opea/text2sql:latest | +| text2sql-react-ui | opea/text2sql-react-ui:latest | ## DBQnA Service Configuration for AMD GPUs The table provides a comprehensive overview of the DBQnA service utilized across various deployments as illustrated in the example Docker Compose files. Each row in the table represents a distinct service, detailing its possible images used to enable it and a concise description of its function within the deployment architecture. -| Service Name | Possible Image Names | Optional | Description | -| -------------------------- | -------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------- | -| dbqna-tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.1-rocm | No | Specific to the TGI deployment, focuses on text generation inference using AMD GPU (ROCm) hardware. | -| postgres | postgres:latest | No | Provides the relational database backend for storing and querying data used by the DBQnA pipeline. | -| text2sql | opea/text2sql:latest | No | Handles text-to-SQL conversion tasks. | -| text2sql-react-ui | opea/text2sql-react-ui:latest | No | Provides the user interface for the DBQnA service. | +| Service Name | Possible Image Names | Optional | Description | +| ----------------- | -------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------- | +| dbqna-tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.1-rocm | No | Specific to the TGI deployment, focuses on text generation inference using AMD GPU (ROCm) hardware. | +| postgres | postgres:latest | No | Provides the relational database backend for storing and querying data used by the DBQnA pipeline. | +| text2sql | opea/text2sql:latest | No | Handles text-to-SQL conversion tasks. | +| text2sql-react-ui | opea/text2sql-react-ui:latest | No | Provides the user interface for the DBQnA service. | diff --git a/DBQnA/docker_compose/intel/cpu/xeon/README.md b/DBQnA/docker_compose/intel/cpu/xeon/README.md index 5f80a5a96f..1e816fed2a 100644 --- a/DBQnA/docker_compose/intel/cpu/xeon/README.md +++ b/DBQnA/docker_compose/intel/cpu/xeon/README.md @@ -1,6 +1,7 @@ # Example DBQnA Deployment on Intel® Xeon® Platform -This document outlines the deployment process for DBQnA application which helps generating a SQL query and its output given a NLP question, utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on an Intel Xeon server. +This document outlines the deployment process for DBQnA application which helps generating a SQL query and its output given a NLP question, utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on an Intel Xeon server. + - [DBQnA Quick Start Deployment](#dbqna-quick-start-deployment): Demonstrates how to quickly deploy a DBQnA service/pipeline on Intel® Xeon® platform. - [DBQnA Docker Compose Files](#dbqna-docker-compose-files): Describes some example deployments and their docker compose files. @@ -43,7 +44,7 @@ To set up environment variables for deploying DBQnA service, source the set_env. source set_env.sh ``` -The set_env.sh script will prompt for required and optional environment variables used to configure the DBQnA service. If a value is not entered, the script will use a default value for the same. It will also generate a _.env_ file defining the desired configuration. +The set*env.sh script will prompt for required and optional environment variables used to configure the DBQnA service. If a value is not entered, the script will use a default value for the same. It will also generate a *.env\_ file defining the desired configuration. ### Deploy the Service Using Docker Compose @@ -131,5 +132,5 @@ The table provides a comprehensive overview of the DBQnA service utilized across | -------------------------- | ------------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------- | | tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.0-intel-cpu | No | Specific to the TGI deployment, focuses on text generation inference using AMD GPU (ROCm) hardware. | | postgres | postgres:latest | No | Provides the relational database backend for storing and querying data used by the DBQnA pipeline. | -| text2sql | opea/text2sql:latest | No | Handles text-to-SQL conversion tasks. | -| dbqna-xeon-react-ui-server | opea/text2sql-react-ui:latest | No | Provides the user interface for the DBQnA service. | \ No newline at end of file +| text2sql | opea/text2sql:latest | No | Handles text-to-SQL conversion tasks. | +| dbqna-xeon-react-ui-server | opea/text2sql-react-ui:latest | No | Provides the user interface for the DBQnA service. |