Update docs

Author: pamelafox

Diff for: README.md

@@ -9,9 +9,9 @@ This project is designed for deployment to Azure using [the Azure Developer CLI]
 
 * [Features](#features)
 * [Getting started](#getting-started)
-    * [GitHub Codespaces](#github-codespaces)
-    * [VS Code Dev Containers](#vs-code-dev-containers)
-    * [Local environment](#local-environment)
+  * [GitHub Codespaces](#github-codespaces)
+  * [VS Code Dev Containers](#vs-code-dev-containers)
+  * [Local environment](#local-environment)
 * [Deployment](#deployment)
 * [Local development](#local-development)
 * [Costs](#costs)
@@ -27,7 +27,13 @@ This project provides the following features:
 * OpenAI function calling to optionally convert user queries into query filter conditions, such as turning "Climbing gear cheaper than $30?" into "WHERE price < 30".
 * Conversion of user queries into vectors using the OpenAI embedding API.
 
-![Screenshot of chat app with question about climbing gear](docs/screenshot_chat.png)
+![Screenshot of chat app with question about climbing gear](docs/images/screenshot_chat.png)
+
+## Architecture diagram
+
+The deployed app uses a user-assigned managed identity to authenticate to Azure services, and stores logs in Log Analytics.
+
+![Architecture diagram: Azure Container Apps, Azure Container Registry, Managed Identity, Azure OpenAI, Azure Database for PostgreSQL](docs/images/architecture.png)
 
 ## Getting started
 
@@ -98,7 +104,7 @@ Once you've opened the project in [Codespaces](#github-codespaces), [Dev Contain
     For GitHub Codespaces users, if the previous command fails, try:
 
    ```shell
-    azd auth login --use-device-code 
+    azd auth login --use-device-code
     ```
 
 2. Create a new azd environment:
@@ -111,7 +117,7 @@ Once you've opened the project in [Codespaces](#github-codespaces), [Dev Contain
 
 3. (Optional) If you would like to customize the deployment to [use existing Azure resources](docs/deploy_existing.md), you can set the values now.
 
-3. Provision the resources and deploy the code:
+4. Provision the resources and deploy the code:
 
     ```shell
     azd up
@@ -195,6 +201,7 @@ Additionally, we have added a [GitHub Action](https://github.com/microsoft/secur
 
 Further documentation is available in the `docs/` folder:
 
+* [Understanding the RAG flow](docs/rag_flow.md)
 * [Customizing the data](docs/customize_data.md)
 * [Deploying with existing resources](docs/deploy_existing.md)
 * [Monitoring with Azure Monitor](docs/monitoring.md)
@@ -204,5 +211,6 @@ Please post in the issue tracker with any questions or issues.
 
 ## Resources
 
+* [RAGHack livestream: Building RAG with PostgreSQL](https://www.youtube.com/watch?v=Dk65oQjYAfo)
 * [RAG chat with Azure AI Search + Python](https://github.com/Azure-Samples/azure-search-openai-demo/)
 * [Develop Python apps that use Azure AI services](https://learn.microsoft.com/azure/developer/python/azure-ai-for-python-developers)

Diff for: docs/customize_data.md

@@ -1,4 +1,4 @@
-# Customizing the data
+# RAG on PostgreSQL: Customizing the data
 
 This guide shows you how to bring in a table with a different schema than the sample table.
 
@@ -30,6 +30,7 @@ If you don't yet have any embeddings in `seed_data.json`:
 
 ## Add the seed data to the database
 
+Now that you have the new table schema and `seed_data.json` populated with embeddings, you can add the seed data to the database:
 
     ```shell
     python src/backend/fastapi_app/setup_postgres_seeddata.py

Diff for: docs/deploy_existing.md

@@ -1,9 +1,9 @@
-# Deploying with existing Azure resources
+# RAG on PostgreSQL: Deploying with existing Azure resources
 
 If you already have existing Azure resources, or if you want to specify the exact name of new Azure Resource, you can do so by setting `azd` environment values.
 You should set these values before running `azd up`. Once you've set them, return to the [deployment steps](../README.md#deployment).
 
-### Openai.com OpenAI account
+## Openai.com OpenAI account
 
 1. Run `azd env set DEPLOY_AZURE_OPENAI false`
 1. Run `azd env set OPENAI_CHAT_HOST openaicom`

Diff for: docs/images/advanced_hybrid_flow.png

@@ -1,4 +1,4 @@
-## Load testing
+# RAG on PostgreSQL: Load testing
 
 We recommend running a loadtest for your expected number of users.
 You can use the [locust tool](https://docs.locust.io/) with the `locustfile.py` in this sample

Diff for: docs/images/azure_architecture.png

@@ -1,5 +1,4 @@
-
-# Monitoring with Azure Monitor
+# RAG on PostgreSQL: Monitoring with Azure Monitor
 
 By default, deployed apps use Application Insights for the tracing of each request, along with the logging of errors.
 

Diff for: docs/images/function_calling_diagram.png

@@ -0,0 +1,61 @@
+# RAG on PostgreSQL: Understanding the RAG flow
+
+This RAG application uses an LLM to answer questions based off matching rows in a PostgreSQL database. It supports two different RAG flows:
+
+* [Simple RAG flow](#simple-rag-flow): First searches the database for a matching row, then uses the LLM to generate an answer.
+* [Advanced RAG flow](#advanced-rag-flow): Adds a query rewriting step before searching the database.
+
+## Simple RAG flow
+
+The simple flow is only used if you de-select the "Advanced flow" option in the Developer Settings dialog. We do not recommend this flow for production use, as it is less sophisticated, but it may be helpful for development and comparison.
+
+This diagram shows the simple RAG flow:
+
+![Simple RAG flow diagram](images/simple_hybrid_flow.png)
+
+1. The user asks a question.
+2. The app uses an embedding model to generate a vector embedding for the question.
+3. The app searches the target table in the PostgreSQL database using a hybrid search (full-text search plus vector search) to find the most relevant rows.
+4. The app sends the original user question and the retrieved rows to the LLM to generate an answer.
+
+When using the app, click the lightbulb on an answer to see the steps:
+
+![Simple RAG flow example](images/simple_rag_thoughts.png)
+
+For the full code, see [/src/backend/fastapi_app/rag_simple.py](/src/backend/fastapi_app/rag_simple.py).
+
+## Advanced RAG flow
+
+The advanced flow is used if you select the "Advanced flow" option in the Developer Settings dialog, and is the default flow for the RAG application.
+
+This diagram shows the advanced RAG flow:
+
+![Advanced RAG flow diagram](images/advanced_hybrid_flow.png)
+
+The difference between the simple and advanced flows is an additional query rewriting step before the database search, which can improve the relevance of the search results. All the steps:
+
+1. The user asks a question.
+2. The app uses an LLM to rewrite the question into a text query that can be used to search the database.
+3. The app uses an embedding model to generate a vector embedding for the rewritten text query.
+4. The app searches the target table in the PostgreSQL database using a hybrid search (full-text search plus vector search) to find the most relevant rows.
+5. The app sends the original user question and the retrieved rows to the LLM to generate an answer.
+
+When using the app, click the lightbulb on an answer to see the query rewriting step call and its results.
+
+![Query rewriting example](images/query_rewriting_thoughts.png)
+
+For the full code, see [/src/backend/fastapi_app/rag_advanced.py](/src/backend/fastapi_app/rag_advanced.py).
+
+### Query rewriting with function calling
+
+Let's dive even deeper into the query rewriting step.
+
+The query rewriting step uses [OpenAI function calling](https://platform.openai.com/docs/guides/function-calling) to generate the rewritten query. That function may also optionally specify column filters, which are then applied to the SQL query for the search step. For the sample data, the filters are for the "brand" and "price" columns, but you could change those to match your own [custom data schema](customize_data.md).
+
+This diagram illustrates the function calling process for a sample question:
+
+![Function calling diagram](images/function_calling_diagram.png)
+
+The `search_database` function definition is in [query_rewriter.py](/src/backend/fastapi_app/query_rewriter.py), and the few shot examples are in [query_fewshots.json](/src/backend/fastapi_app/prompts/query_fewshots.json). The function calling response is parsed in [query_rewriter.py](/src/backend/fastapi_app/query_rewriter.py) to extract the suggested SQL query and column filters, and those are passed to [postgres_searcher.py](/src/backend/fastapi_app/postgres_searcher.py) to search the database.
+
+To be able to use function calling, the app must use a model that has support for it. The OpenAI GPT models do support function calling, but other models may not. If you're developing locally with Ollama, we recommend llama3.1 as it has been tested to work with function calling.