python-recipes/agents/02_full_featured_agent.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qYvD2zzKobTC"
      },
      "source": [
        "![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120)\n",
        "\n",
        "# Full-Featured Agent Architecture\n",
        "The following example demonstrates how to build a tool-enabled agentic workflow with a semantic cache and an allow/block list router. This approach helps reduce latency and costs in the final solution.\n",
        "\n",
        "Note: This notebook summarizes this [this workshop](https://github.com/redis-developer/oregon-trail-agent-workshop). For a more detailed step-by-step walkthrough of each element, please refer to the repository.\n",
        "\n",
        "## Let's Begin!\n",
        "<a href=\"https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/agents/02_full_featured_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NTFxCojYECnx"
      },
      "source": [
        "# Setup\n",
        "\n",
        "## Packages"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "Zz62U5COgF21"
      },
      "outputs": [],
      "source": [
        "%pip install -q langchain langchain-openai langchain-redis langgraph sentence-transformers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### OPEN_AI_API key\n",
        "\n",
        "A open_ai_api key with billing information enabled is required for this lesson."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VO0i-1c9m2Kb",
        "outputId": "ec942dbf-226a-426d-8964-e03831e0dd99"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "OPENAI_API_KEY:··········\n"
          ]
        }
      ],
      "source": [
        "# NBVAL_SKIP\n",
        "import os\n",
        "import getpass\n",
        "\n",
        "\n",
        "\n",
        "def _set_env(key: str):\n",
        "    if key not in os.environ:\n",
        "        os.environ[key] = getpass.getpass(f\"{key}:\")\n",
        "\n",
        "\n",
        "_set_env(\"OPENAI_API_KEY\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Po4K08Uoa5HJ"
      },
      "source": [
        "## Redis instance\n",
        "\n",
        "### For colab"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vlF2874ZoBWu",
        "outputId": "e5e7ebc0-b70c-4682-d70c-b33c584e72d4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb jammy main\n",
            "Starting redis-stack-server, database path /var/lib/redis-stack\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "gpg: cannot open '/dev/tty': No such device or address\n",
            "curl: (23) Failed writing body\n"
          ]
        }
      ],
      "source": [
        "# NBVAL_SKIP\n",
        "%%sh\n",
        "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n",
        "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n",
        "sudo apt-get update  > /dev/null 2>&1\n",
        "sudo apt-get install redis-stack-server  > /dev/null 2>&1\n",
        "redis-stack-server --daemonize yes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### For Alternative Environments\n",
        "There are many ways to get the necessary redis-stack instance running\n",
        "1. On cloud, deploy a [FREE instance of Redis in the cloud](https://redis.com/try-free/). Or, if you have your\n",
        "own version of Redis Enterprise running, that works too!\n",
        "2. Per OS, [see the docs](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)\n",
        "3. With docker: `docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest`\n",
        "\n",
        "## Test connection"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "My-zol_loQaw",
        "outputId": "b58c2466-ee10-480c-ad4c-608cbf747e8b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os\n",
        "from redis import Redis\n",
        "\n",
        "# Use the environment variable if set, otherwise default to localhost\n",
        "REDIS_URL = os.getenv(\"REDIS_URL\", \"redis://localhost:6379\")\n",
        "\n",
        "client = Redis.from_url(REDIS_URL)\n",
        "client.ping()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p8lqllwDoV_K"
      },
      "source": [
        "# Motivation\n",
        "\n",
        "The goal of the workshop is to create an agent workflow that can handle five Oregon Trail-themed scenarios, mimicking situations that often arise when implementing agent workflows in practice.\n",
        "\n",
        "## Scenario 1 - name of the wagon leader\n",
        "\n",
        "**Learning goal:** Test basic LangGraph setup and execution. <br>\n",
        "\n",
        "**Question:** `What is the first name of the wagon leader?` <br>\n",
        "**Answer:** `Art` <br>\n",
        "**Type:** `free-form` <br>\n",
        "\n",
        "## Scenario 2 - restocking tool\n",
        "\n",
        "**Learning goal:** Agent interaction with custom defined tool and **structured output** for multiple choice questions. <br>\n",
        "\n",
        "**Question:** `In order to survive the trail ahead, you'll need to have a restocking strategy for when you need to get more supplies or risk starving. If it takes you an estimated 3 days to restock your food and you plan to start with 200lbs of food, budget 10lbs/day to eat, and keep a safety stock of at least 50lbs of back up... at what point should you restock?` <br>\n",
        "**Answer:** `D` <br>\n",
        "**Options:** `[\"A: 100lbs\", \"B: 20lbs\", \"C: 5lbs\", \"D: 80lbs\"]` <br>\n",
        "**Type:** `multi-choice` <br>\n",
        "\n",
        "## Scenario 3 - retrieval tool\n",
        "\n",
        "**Learning goal:** Agent implements Retrieval Augmented Generation.\n",
        "\n",
        "**Question:** `You’ve encountered a dense forest near the Blue Mountains, and your party is unsure how to proceed. There is a fork in the road, and you must choose a path. Which way will you go?` <br>\n",
        "**Answer:** `B` <br>\n",
        "**Options:** `[\"A: take the northern trail\", \"B: take the southern trail\", \"C: turn around\", \"D: go fishing\"]` <br>\n",
        "**Type:** `multi-choice` <br>\n",
        "\n",
        "## Scenario 4 - semantic cache\n",
        "\n",
        "**Learning goal:** Implement semantic cache that bypasses expensive agent workflow for known answer. <br>\n",
        "\n",
        "**Question:** `There's a deer. You're hungry. You know what you have to do...` <br>\n",
        "**Answer:** `bang` <br>\n",
        "**Type:** `free-form` <br>\n",
        "\n",
        "## Scenario 5 - allow/block list with router\n",
        "\n",
        "**Learning goal:** Implement semantic router that blocks requests for non-related topics.\n",
        "\n",
        "**Question:** `Tell me about the S&P 500?` <br>\n",
        "**Answer:** `you shall not pass` <br>\n",
        "**Type:** `free-form` <br>\n",
        "\n",
        "\n",
        "\n",
        "# Final Architecture\n",
        "\n",
        "In the end, we are building a workflow like the following:\n",
        "\n",
        "![diagram](../../assets/full_featured_agent.png)\n",
        "\n",
        "As a reminder for more detail see: [Redis Developer Oregon Trail Agent Workshop](https://github.com/redis-developer/oregon-trail-agent-workshop).\n",
        "\n",
        "# Defining the agent with LangGraph\n",
        "\n",
        "## Tools\n",
        "\n",
        "Tools are functions that the central LLM powered \"agent\" can determine to invoke depending on the situation.\n",
        "\n",
        "### Restock tool\n",
        "\n",
        "The first tool we will define implements the restocking formula. LLMs are designed to predict text responses, not to perform deterministic math. In this case, the agent will act as a parser, extracting the necessary information from the human query and calling the tool with the appropriate schema.\n",
        "\n",
        "One of the advantages of `LangGraph` is that the schema for the tool can be defined as a `pydantic` model. Note: It is also essential to include a well-written `doc_string` with the tool function so the agent can determine the appropriate situation to use the tool."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {},
      "outputs": [],
      "source": [
        "from langchain_core.tools import tool\n",
        "from pydantic import BaseModel, Field\n",
        "\n",
        "class RestockInput(BaseModel):\n",
        "    daily_usage: int = Field(\n",
        "        description=\"Pounds (lbs) of food expected to be consumed daily\"\n",
        "    )\n",
        "    lead_time: int = Field(description=\"Lead time to replace food in days\")\n",
        "    safety_stock: int = Field(\n",
        "        description=\"Number of pounds (lbs) of safety stock to keep on hand\"\n",
        "    )\n",
        "\n",
        "\n",
        "@tool(\"restock-tool\", args_schema=RestockInput)\n",
        "def restock_tool(daily_usage: int, lead_time: int, safety_stock: int) -> int:\n",
        "    \"\"\"restock formula tool used specifically for calculating the amount of food at which you should start restocking.\"\"\"\n",
        "    print(f\"\\n Called restock tool: {daily_usage=}, {lead_time=}, {safety_stock=} \\n\")\n",
        "    return (daily_usage * lead_time) + safety_stock"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Retriever tool\n",
        "\n",
        "Sometimes an LLM might need access to data that it was not trained on, whether because the data is proprietary, time-sensitive, or otherwise unavailable.\n",
        "\n",
        "In such cases, Retrieval-Augmented Generation (RAG) is often necessary. Here, a vector search is used to augment the final LLM prompt with helpful and necessary context.\n",
        "\n",
        "RAG and agents are not mutually exclusive. Below, we define a retriever tool that performs RAG whenever the agent determines it is necessary."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "09:04:55 redisvl.index.index INFO   Index already exists, not overwriting.\n"
          ]
        }
      ],
      "source": [
        "\n",
        "from langchain.tools.retriever import create_retriever_tool\n",
        "\n",
        "from langchain_redis import RedisConfig, RedisVectorStore\n",
        "from langchain_core.documents import Document\n",
        "from langchain_openai import OpenAIEmbeddings\n",
        "\n",
        "## Helper methods\n",
        "\n",
        "INDEX_NAME = os.environ.get(\"VECTOR_INDEX_NAME\", \"oregon_trail\")\n",
        "REDIS_URL = os.environ.get(\"REDIS_URL\", \"redis://localhost:6379/0\")\n",
        "CONFIG = RedisConfig(index_name=INDEX_NAME, redis_url=REDIS_URL)\n",
        "\n",
        "def get_vector_store():\n",
        "    try:\n",
        "        CONFIG.from_existing = True\n",
        "        vector_store = RedisVectorStore(OpenAIEmbeddings(), config=CONFIG)\n",
        "    except:\n",
        "        print(\"Init vector store with document\")\n",
        "        CONFIG.from_existing = False\n",
        "        vector_store = RedisVectorStore.from_documents(\n",
        "            [doc], OpenAIEmbeddings(), config=CONFIG\n",
        "        )\n",
        "    return vector_store\n",
        "\n",
        "## Relevant data\n",
        "\n",
        "doc = Document(\n",
        "    page_content=\"the northern trail, of the blue mountains, was destroyed by a flood and is no longer safe to traverse. It is recommended to take the southern trail although it is longer.\"\n",
        ")\n",
        "\n",
        "## Retriever tool\n",
        "vector_store = get_vector_store()\n",
        "\n",
        "retriever_tool = create_retriever_tool(\n",
        "    vector_store.as_retriever(),\n",
        "    \"get_directions\",\n",
        "    \"Search and return information related to which routes/paths/trails to take along your journey.\",\n",
        ")\n",
        "\n",
        "## Store both tools in a list\n",
        "tools = [retriever_tool, restock_tool]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# State\n",
        "\n",
        "State is the set of messages that is passed between nodes in our graph so that the proceeding node knows what happened at the last node and so on. In this case, our state will extend the normal `MessageState` but also add a custom field for `multi_choice_responses`. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {},
      "outputs": [],
      "source": [
        "from typing import Literal\n",
        "\n",
        "from langgraph.graph import MessagesState\n",
        "from pydantic import BaseModel, Field\n",
        "\n",
        "\n",
        "class MultipleChoiceResponse(BaseModel):\n",
        "    multiple_choice_response: Literal[\"A\", \"B\", \"C\", \"D\"] = Field(\n",
        "        description=\"Single character response to the question for multiple choice questions. Must be either A, B, C, or D.\"\n",
        "    )\n",
        "\n",
        "\n",
        "class AgentState(MessagesState):\n",
        "    multi_choice_response: MultipleChoiceResponse\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Nodes\n",
        "\n",
        "Nodes are steps in the process flow of our agent where functions can be invoked."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {},
      "outputs": [],
      "source": [
        "from functools import lru_cache\n",
        "\n",
        "from langchain_core.messages import HumanMessage\n",
        "from langchain_openai import ChatOpenAI\n",
        "from langgraph.prebuilt import ToolNode\n",
        "\n",
        "\n",
        "## Function definitions that invoke an LLM model\n",
        "\n",
        "### with tools\n",
        "@lru_cache(maxsize=4)\n",
        "def _get_tool_model(model_name: str):\n",
        "    if model_name == \"openai\":\n",
        "        model = ChatOpenAI(temperature=0, model_name=\"gpt-4o\")\n",
        "    else:\n",
        "        raise ValueError(f\"Unsupported model type: {model_name}\")\n",
        "\n",
        "    model = model.bind_tools(tools)\n",
        "    return model\n",
        "\n",
        "### with structured output\n",
        "@lru_cache(maxsize=4)\n",
        "def _get_response_model(model_name: str):\n",
        "    if model_name == \"openai\":\n",
        "        model = ChatOpenAI(temperature=0, model_name=\"gpt-4o\")\n",
        "    else:\n",
        "        raise ValueError(f\"Unsupported model type: {model_name}\")\n",
        "\n",
        "    model = model.with_structured_output(MultipleChoiceResponse)\n",
        "    return model\n",
        "\n",
        "### Functions for responding to a multiple choice question\n",
        "def multi_choice_structured(state: AgentState, config):\n",
        "    # We call the model with structured output in order to return the same format to the user every time\n",
        "    # state['messages'][-2] is the last ToolMessage in the convo, which we convert to a HumanMessage for the model to use\n",
        "    # We could also pass the entire chat history, but this saves tokens since all we care to structure is the output of the tool\n",
        "    model_name = config.get(\"configurable\", {}).get(\"model_name\", \"openai\")\n",
        "\n",
        "    print(\"Called multi choice structured\")\n",
        "\n",
        "    response = _get_response_model(model_name).invoke(\n",
        "        [\n",
        "            HumanMessage(content=state[\"messages\"][0].content),\n",
        "            HumanMessage(content=f\"Answer from tool: {state['messages'][-2].content}\"),\n",
        "        ]\n",
        "    )\n",
        "    # We return the final answer\n",
        "    return {\n",
        "        \"multi_choice_response\": response.multiple_choice_response,\n",
        "    }\n",
        "\n",
        "\n",
        "# Function for conditional edge\n",
        "def is_multi_choice(state: AgentState):\n",
        "    return \"options:\" in state[\"messages\"][0].content.lower()\n",
        "\n",
        "\n",
        "def structure_response(state: AgentState, config):\n",
        "    if is_multi_choice(state):\n",
        "        return multi_choice_structured(state, config)\n",
        "    else:\n",
        "        # if not multi-choice don't need to do anything\n",
        "        return {\"messages\": []}\n",
        "\n",
        "\n",
        "system_prompt = \"\"\"\n",
        "    You are an oregon trail playing tool calling AI agent. Use the tools available to you to answer the question you are presented. When in doubt use the tools to help you find the answer.\n",
        "    If anyone asks your first name is Art return just that string.\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "# Define the function that calls the model\n",
        "def call_tool_model(state: AgentState, config):\n",
        "    # Combine system prompt with incoming messages\n",
        "    messages = [{\"role\": \"system\", \"content\": system_prompt}] + state[\"messages\"]\n",
        "\n",
        "    # Get from LangGraph config\n",
        "    model_name = config.get(\"configurable\", {}).get(\"model_name\", \"openai\")\n",
        "\n",
        "    # Get our model that binds our tools\n",
        "    model = _get_tool_model(model_name)\n",
        "\n",
        "    # invoke the central agent/reasoner with the context of the graph\n",
        "    response = model.invoke(messages)\n",
        "\n",
        "    # We return a list, because this will get added to the existing list\n",
        "    return {\"messages\": [response]}\n",
        "\n",
        "\n",
        "# Define the function to execute tools\n",
        "tool_node = ToolNode(tools)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Graph\n",
        "\n",
        "The graph composes the tools and nodes into a compilable workflow that can be invoked."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {},
      "outputs": [],
      "source": [
        "from typing import Literal, TypedDict\n",
        "from langgraph.graph import END, StateGraph\n",
        "\n",
        "\n",
        "# Define the config\n",
        "class GraphConfig(TypedDict):\n",
        "    model_name: Literal[\"anthropic\", \"openai\"]\n",
        "\n",
        "# Define the function that determines whether to continue or not\n",
        "def should_continue(state: AgentState):\n",
        "    messages = state[\"messages\"]\n",
        "    last_message = messages[-1]\n",
        "    # If there is no function call, then we respond to the user\n",
        "    if not last_message.tool_calls:\n",
        "        return \"structure_response\"\n",
        "    # Otherwise if there is, we continue\n",
        "    else:\n",
        "        return \"continue\"\n",
        "\n",
        "\n",
        "# Define a new graph\n",
        "workflow = StateGraph(AgentState, config_schema=GraphConfig)\n",
        "\n",
        "# Add nodes\n",
        "workflow.add_node(\"agent\", call_tool_model)\n",
        "workflow.add_node(\"tools\", tool_node)\n",
        "workflow.add_node(\"structure_response\", structure_response)\n",
        "\n",
        "# Set the entrypoint\n",
        "workflow.set_entry_point(\"agent\")\n",
        "\n",
        "# add conditional edge between agent and tools\n",
        "workflow.add_conditional_edges(\n",
        "    \"agent\",\n",
        "    should_continue,\n",
        "    {\"continue\": \"tools\", \"structure_response\": \"structure_response\"},\n",
        ")\n",
        "\n",
        "\n",
        "# We now add a normal edge from `tools` to `agent`.\n",
        "workflow.add_edge(\"tools\", \"agent\")\n",
        "workflow.add_edge(\"structure_response\", END)\n",
        "\n",
        "\n",
        "# This compiles it into a LangChain Runnable,\n",
        "# meaning you can use it as you would any other runnable\n",
        "graph = workflow.compile()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Evaluate graph structure\n",
        "\n",
        "When we invoke the graph, it follows four primary steps:  \n",
        "\n",
        "1. **Evaluate Conditional Edge**: The graph evaluates the conditional edge between tools and the agent via the `should_continue` function. This determines whether it should `continue` and call a tool or move to `structure_response` to format the output for the user.  \n",
        "2. **Invoke Tools**: If it decides to invoke the tools, the response from the tool is appended as a message to the state and passed back to the agent.  \n",
        "3. **Determine Next Step**: If tools have already been called or are deemed unnecessary, the graph moves to the `structure_response` node.  \n",
        "4. **Handle Multiple-Choice Questions**: If the question is identified as a **multiple-choice question** within the `structure_response` node, a model is invoked to ensure the response is returned as a literal `A, B, C, or D`, as expected by the game. Otherwise, it simply proceeds forward.  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUkAAAFlCAIAAADpho2yAAAAAXNSR0IArs4c6QAAIABJREFUeJzt3XdcE/f/B/BPBiQhIYQpS0DEgSigorWKW6riwlkV3LbYOmqddbXWr9U6aofWUb+u1lUH1r03LhQHqIiLIbITIHvn98d9f3z5sgyQ8Lk73s+Hf+CRfPLOJS/u7nN3nw/DZDIhAADtMHEXAACwCsg2APQE2QaAniDbANATZBsAeoJsA0BPbNwF0FB+hkYu1Sulep3OpFUZcZdjFg6PacNh8oUsvoONq7ct7nKABUC2LebVI/nbZPnbp4omrfgGg4kvZDs1smWycJdlHhNC+ZlqhVRvy2Vlpir8Wwv82wj8guxw1wVqjwHXrtTd83vS2ycLfQP5foH8Jm34bBsG7orqRK0wvE1WZKepc9NUnQe5+Lfh464I1AZku06KC3Tn/8x18eJ0GeTM5VNkG222onzd7ZOFTAYjYlwjqv/BaoAg27X3+rH87hnxoM89HVxscNdiRfnvNEc3Zg2b7tXIl4u7FlADkO1aynqpenq7pN9Ed9yF1JPDP7+LiHEXudL5rxjNQLZrI+lmSdYrZeRkD9yF1KvDv2R17OvkGwgdbNQA57drLPuN6vVjWUMLNkJo5GzvKwfzFCUG3IUAs0C2a0atND64VDRspjfuQvAYu8jv0oE83FUAs0C2ayb+n4JmbQW4q8CGw2U08uE8uFiEuxDwYZDtGijK0+VlqAM7CnEXglOnSOd758RGalxu16BBtmsgOb6k61C3+nktuVz+4sULXE+vXs+Rbg8vw6ab7CDb5jKZUNKtYp+WvPp5udGjRx8/fhzX06vn3Yz3/F6JlRoHlgLZNldassK/df1dfanVamv3ROKkZq2fbg6hsw3bhinJteJLgLqDbJvr/VtV83b21mh59+7dkZGR4eHhU6ZMSUhIQAgNHDhQIpEcPnw4LCxs4MCBRFZ///33wYMHf/TRRwMGDNi8ebPB8J9zUWvWrPnkk09u3LgxdOjQsLCw+/fvV3y6xbUIE75LVVqjZWApcB+YufIy1M1CLd9DnpCQsGnTpn79+nXu3Pn27dtKpRIhtHbt2hkzZrRv3z46OtrW1hYhxGKx7t27161bN29v79TU1J07dwqFwpiYGKIRuVy+efPmb775RqVSdejQoeLTLc5OwMx+q7ZGy8BSINvmUkr1dvaWX13Z2dkIoVGjRgUHB0dGRhILW7VqxWazXVxcQkNDiSUsFmvPnj0Mxn9u2MjKyrpy5UpptrVa7dKlS1u3bl3V0y2O78BWlOit1DiwCMi2uRQyg53Q8nd6hYeHC4XCZcuWzZ8/Pzw8vJpHSiSS7du33717VyqVIoTs7f97gMDlckuDXT/shGyFFLJNanC8bR4T4nCZTKbl73N0cXHZuXOnr6/v7Nmzp0yZkp+fX+nDxGJxdHR0QkLCF198sXHjxsDAwNLjbYSQnV19X+PNZjPYNvDlITX4eMzDQEwWw0pbKj8/v99++23Lli2vX79evnx56fKyt/EcPXpUIpFs3ry5b9++QUFB7u4fvv/MqncByYv1Nhy4o5vUINvmsrNnqWRWuU2COF/VoUOHrl27ll5wwuPxCgsLSx9TXFzs6OhYGuni4uLqo1vu6RankOr5QjigIzX4eMzl7sdTyS2f7WfPni1cuHDUqFF2dna3b99u1aoVsbxt27bnzp3bvXu3UCgMDg4OCws7dOjQli1bQkJCrly5cuvWLaPRWFxcLBKJKm223NMDAgIsW7ZWbXT25Fi2TWBZrLI7gaAaKpkh/bnCv42FT4OVlJS8fPnywoULCQkJ7dq1W7x4sUAgQAgFBwenpqaeOXPmxYsXQUFBvXr1MhqNhw8fvnz5cuPGjZctW/bo0SOlUhkWFnbr1q20tLRx48aVbbbc05s0aWLZsm/EFbb+WCgQwbaBvGBsBnNpVMY9K9I/X+2PuxD81ArD3tUZU1fCqiA1+LtrLg6P6d9GkJehrmbYsPXr1586dari8sDAwJSUlEqfsmvXLotvVMuJj49funRppb/y9vbOysqqaVXvXqlbdXKwaI3A8mC7XQPvX6sSzkmGzvCq6gHFxcXEhWXlMBhVrmc3Nzc227p/YdVqtUQiqfRXVRVWfVW7lqePnO0NO+QkBx9PDXgF8Fg2jIwUZVVjholEoqo6tzDicrmenp6Wai3pZol/Gz4Em/zgHFjNdBnskvpAhrsKnNKeKboMcsFdBfgwyHbNOHvYejfnXT5Y+dVjtBe3MatDhCPbFq5aoQDIdo21+khoy2HeOSXGXUh9u/BXXkCovWfTehqdAtQR9KXV0pPrxSqFsVOkE+5C6snFvXnN2tn7tYLBySkDttu1FNJdxGCgM7tycBdidXqt6dCGd14BPAg2tcB2u07eJCmuHclv38sxtAfpusct4u4ZceYLZY8Rbm4+cIUpxUC268pgQHdOFqYmykK7i/yC+M4edJiYPi9DnfVKdfes+KN+zmF9HBH0nVEQZNsylDJDcnzJmyS5XmcMCLZnsBBfyLZ3ZBsM1Fi9TAZDKtEpZQYGAz2/JxU6sQNC7UO6i5hw0EZZkG0Lk4p12WkaeZFOKdMzmAx5sYVv+U5PT+dyuebcv10jfAcWk8GwE7LsHW28Anh29nSbS7wBgquLLEzobCN0tuJEtmvX/unk69v/U2sNhAZoA3a5AKAnyDYA9ATZphihUMjlVnmTKQClINsUI5VK1WoY9B98GGSbYjgcjrXv9wb0ANmmGI1Go9fDoP/gwyDbFMPj8WxsrHiODdAGZJtiVCqVTqfDXQWgAMg2xTg6OvJ4cAc1+DDINsUUFRWpVCrcVQAKgGwDQE+QbYrhcrksFtzIAT4Msk0xarW67Oy8AFQFsk0xXC4XzoEBc0C2KUatVsM5MGAOyDYA9ATZphihUMjhwLCE4MMg2xQjlUo1Gg3uKgAFQLYBoCfINsWIRCIYmwGYA7JNMcXFxTA2AzAHZBsAeoJsUwzcBwbMBNmmGLgPDJgJsg0APUG2KQbGMAZmgmxTDIxhDMwE2QaAniDbFAPjkwMzQbYpBsYnB2aCbFMM3AcGzATZphi4DwyYCbINAD1BtimGx+NBXxowB2SbYlQqFfSlAXNAtilGJBLBvSLAHJBtiikuLoZ7RYA5INsUA9ttYCbINsXAdhuYCbJNMXw+39bWFncVgAIYJpMJdw3gwwYPHkx8UjKZjM1mE7vlDAbjxIkTuEsDJAVnSqnBzc0tMTGxdAbP4uJio9HYp08f3HUB8oJ9cmqIjo52dnYuu8TFxWXChAn4KgJkB9mmhp49e/r5+ZX+12QyBQcHBwUFYS0KkBpkmzLGjBkjFAqJn52dnadMmYK7IkBqkG3K6N27d7NmzUwmE7HRDgwMxF0RIDXINpWMHj1aJBI5OztPnToVdy2A7KCf3PLUSmNhlkatMli85cZOHVr59nJ0dGRrvF8/kVu8fb4928WLY8NhWLxlUP/g/LYlmUzo/F95714ovJrzDXrqrVid2iDO0QSECHqOcsNdC6gryLbF6DSmI79lte3l7BVgh7uWOkl9UPL+tWJIrCfuQkCdQLYtZv/azPAod8dGdLggNC1Z/u6lbMBkD9yFgNqDvjTLSL0v82rKp0ewEUJN2giYTGb2G5jkgMIg25aRn6Xh8lm4q7AkGw5TnAODLlIYZNsyNCqj0IUmG22Cg4utQgqDN1EYZNsytCqD0WDEXYUlGfRGA0SbyiDbANATZBsAeoJsA0BPkG0A6AmyDQA9QbYBoCfINgD0BNkGgJ4g2wDQE2QbAHqCbANAT5BtmjMYDMnJj3FXATCAbNPcup/+teGXVbirABhAtsnufXZWXcbG0WrgHuwGCsY5xUOr1f751/YrV87nF+Q5O7t8EjFg4oRYYrovnU63c9eWS5fPqlTK4OB2L1+mjIuZOmTwCITQo8cPtv9705s3Lx0dndqGdpg6ZbqzswtCaNCQHrO/WhQff/XuvXg+XzBo4PAJ4z9DCP24dvnVaxcRQj17hyGEDv991sXFFfdbB/UEso0Hi8VKTLz3cedunh7er1+n7t23095eOGpkDEJo6x+/njhxZOqU6S4ublu2/qzRqPv3G4wQSnyY8M2iWRF9IodGfSqTlhyNOzBn3rRtW/ZyuVyE0I9rvps4IXb06AnXrl3cvWdbi+aBnTqFx4ydXJCfl5PzftE3KxBCDg4i3O8b1B/INh4sFmvz73sYjP+MBJ6dk3Xj5pVRI2MMBsOpU3EDIqM+HTWOmPfrh1VLk58+bt+u48ZN6wYNHDZr5gLiKWFhnSZMGnH/wZ2u4T0RQpH9h0SPnYQQCmja/PSZfxIe3OnUKdzb28fBQSQpErdpE4r17QIMINvYFBVJ/vxr+/0Hd2UyKULIXmCPECopKdZqtV5ejYnHED/IZNLc3JyMjLT379+dOn2sbCP5+XnED1wuj/iBxWK5urqJCwvq/Q0BcoFs4yGRiD+fFs3j2U2e9IWnp/fOnZvfZWUQu80CviA5+fHIEdEIoZSUpwihpv7NiorECKEJ4z/v1rVX2XacnFwqNs5msQ1Gy89qAqgFso3HiZNHi4okv2/c3aiRO0LIzc2dyDaLxRozZuL2f29a+cMSFxe34ycODx82pnFj33fvMhBCGo3ax8fPjOb/BwxB3zDBOTA8pNJikciRCDZCqERaXJrAqCGjOoR1KiqSyOWyJYtXzpg+FyHk7e3TqJH72XMnVCoV8TC9Xq/T6T74QlwuTyIRG420GqcRmAOyjUdoaJhEIt65a8u9hNvrf1p5796twsKCkpJihNC/flgsFDpERka1bduBgRh5ebkIIQaDMf3LuWJx4fSZE/85fjgu7uD0GROPnzj8wRcKCW4nk0k3/Lzq/PlTT548rJc3B0iBtXz5ctw10MGrR3KRG8fB7CHKfX2bmEzGf44fvnnjsqdX43lzlyUnP1KplKGhYUVF4lOn4y5fOX/j5pUrVy8c++dv90aeTZs29/Vp0rJFq6SkRxcunk558bSpf7OIiAHE+e0DB3c3a9ayQ1gnovFTp+L4fEGvnn0RQv7+ATJZyeUr554kPfT29g0MbG1mhQXv1CajyacFtec2a8hgPjDLOLMjx7e1vU9LQd2bMhgMxEUsCCGpTPrNollsNvu3X/5d95Zr5NntIoPO2GWwcz2/LrAU6EsjnZ82/PDmzcuPP+4mEjlmvkt/+/bVgAFDcRcFqAeyTTodO3bOz889Grdfp9N5eHiNH/cZcT4MgBqBbJNOj+59enTvg7sKQHnQTw4APUG2AaAnyDYA9ATZBoCeINsA0BNkGwB6gmwDQE+QbQDoCbINAD1BtgGgJ7jm1DL4IrqtSRabaWvuHauAjGC7bRksW11hFq1G+c/LUAmdbXBXAWoPsm0Be/fu/ef8v2VFHx7hiEJUcr1PCz7uKkDtQbbrJC8vjxhscP2mb72acuOP5eGuyDIu78tu18sxt+Ad7kJA7cG4K7Wk1+uXLFkSFRX18ccfly5MviV9m6xo3JLv4sll2zCwFlgbGqVRkqt5eruo50g3n5a8mJiYmJiYfv364a4L1AZku5bi4+PVanWfPuVvtM5+q065J1XI9MV5VtlFl8tlLBaLx7PKMGYCEdvZ0za0u8jB5T9H2qdPnx4wYEBhYaGLSyUDoQMyg2zXzOvXr5ctW3bgwAEsr56TkxMbG8tisY4dO2bGwy1m69atDAYjNja2Pl8U1BEcb5uL+CN4/PjxH3/8EVcNBw4ceP/+fXZ29sGDB+vzdadNm8ZgMNRqtVKprM/XBXUB222znDp16uXLl3PmzMFYQ0FBweTJk3NychBCvr6+R48erecCTCbTkydP7ty588UXX9TzS4NagO32B6hUKqVS+erVK7zBRgj9+eef2dnZxM+5ublxcXH1XACDwQgNDbWxsbl06VI9vzSoBch2ddavX5+WlsbhcL7++mu8leTl5V2/fr10Tl+NRrN//34slUydOrVTp04IoS1btmApAJgJsl2lP/74w8vLq1WrVqUzAWB0+PDh0o02ITs7u/433QSBQIAQ4vF4v/76K5YCgDngeLs8qVS6devWBQsWaLVaW9JcUR0VFZWVlVVuIZaj7rKKioocHR0vXrwYERGBsQxQKbrd4VB3X3755bx58xBC5Ak2Quiff/4hfli7dq2vr++nn36KuyKEEHJ0dEQIcbncESNGHDlyBHc54H9Atv/j7du3mZmZPXr02Lt3L+5aqsPlcm1syHULR9euXZs0aYIQSk1NbdGiBe5ywH/A8TZCCL1//37hwoWhoaG4C/kwpVJpMBhwV1Get7c30ZH+6aefwjlwkmjo2X769Gl+fj6LxTp8+LBIJMJdjllKe8vJpnnz5j/88MPDhw9VKhXuWkDDzvatW7fWrVvn5OTk7u6OuxZzcTgcLpeLu4oqBQQEhIeHm0ym2NhYvV6Pu5wGrYFm+82bN8S5nD179rDZVOp0kEgkTCbZPzU7O7vPPvuM5D0XtEf2b4k17N27d9euXQihkJAQ3LXUmMlk4vF4uKv4sLCwsIkTJyKENmzYgLuWBqphZVsmkxEXXaxcuRJ3LbUkFov5fCoNhxIcHEycUwT1rAFle//+/ZcvX0YIDR8+HHcttUdcLoK7ihro06fPd999hxBKSEjAXUvD0lCy/fr165ycnKioKNyF1FVxcTG1so0Qsre3J3aaZs+ejbuWBoRK3Ui1c//+fV9fX3d397lz5+Kupa6MRiODwaDoECi9e/e2tbWVy+WlV6QDq6L5djshIWHHjh1ubm70+DJlZGRQ62C7nK5duwoEgtevX//999+4a6E/mmebyWRu3boVdxUWk5GR4efnh7uKugoNDc3IyHj16hXuQmiOntlOT08nDq3DwsJw12JJhYWFrVu3xl2FBSxYsMDR0TE/P5/YRQfWQM9sHz58uPTGKTq5e/cuDbbbBBcXFycnpwEDBuTn5+OuhZ7olu1Dhw4hhObPn4+7EKt4/PgxFa+3qQqbzb5+/XpSUhIMImANtMr2qlWrPDw8cFdhLe/evRMIBJQ7AfZBffr0MZlMq1evxl0I3VR5DoxaB0JGo5HJZI4dO9bFxaWayk0mE3GulYpSUlIqTnVAD0wms1mzZsQ8B7hroY8qs02hu3CNRqNcLhcKhXZ2dtWXzWAwqJvtM2fOUPqKuuqNGDGi4qBRoC7osE+uUCiEQiHuKqxLr9ffvXu3a9euuAuxImKAh06dOmm1Wty10AEdsk3dTbH5rly50rNnT9xV1If4+PhDhw5B71rdUTjber2+pKQEdxX15NmzZw1kPk02mx0TE6PX61NTU3HXQm3Ysv3ixQuNRlN2yYYNG7766iszn240Go1Go4ODg3WqI5fCwsJz5851794ddyH1x8bG5vvvvy8qKsJdCIXhyfbFixfnzJmjVqvLLrSzs6vRqAOkGmPYqg4ePDh69GjcVdS3/fv3Jycn466CwvDcB1ZpZ8m0adPMea7JZJJIJM7Ozlaoi6QOHjx48eJF3FVg0K1bt2vXroWGhlJlmEpSqXJekYpXAqrV6oMHD16/fl0sFru5ufXu3XvUqFEsFksikWzfvv3BgwcGg6FVq1ZTpkwhRqtesWKFt7c3i8U6d+6cXq/v0KHD9OnT+Xz+xYsXf/7559Jmv/7664iIiIkTJ+bn57dq1Wr9+vUIoZEjR06fPv3OnTsJCQl8Pj8yMnLs2LEIoUePHi1ZsmTDhg0tW7Yknj506NDBgwdPmjSJmAFv+/btjx494nA4TZs2HT9+fPPmzf/n3TIYrq6uVliNVnTu3LlXr17NnDkTdyHYDB48eMuWLV5eXrgLoRhz98kNBsPy5cvj4uK6dOkye/bs8PDwrKwsFoulVqsXLVr0+PHjyZMnz5gxQywWL168uPTqkbi4uLy8vOXLl8fGxsbHxxOzRoeFhQ0bNgwhtHz58nXr1hG3c8yaNatp06ZlX3HDhg3+/v5r167t1avX3r17Pzhqh0QimTdvnkwmi42NnTRpkl6vX7BgQXp6ei1XDGmsX79+3LhxuKvA6cSJEzweD3rOa8rcffL4+PikpKSvvvqqb9++ZZdfvXr13bt3q1atIgbuDwoKmjx58okTJ4jNrJeX1/z58xkMRosWLW7dupWYmDhlyhRHR0fiytAWLVqUdoa1a9cuLi6u7BH4J598QsyM4+/vf/78+YcPH7Zp06bcIXpZBw4cEIlEq1atIsYt7dWr19SpU8+fPx8bG1vblYPfrl27oqKiYI+Uz+cfP36cBsPm1Cdzs52YmMjhcCpe85iUlMTn80tn5GjUqFHjxo1fvnxJ/JfD4ZQOlN+oUaOUlBTzKysdhZvFYjk7OxcWFjIYjGqG5n7w4EFBQUHZK7d0Ol1BQYH5r0g2Op1u27Ztd+/exV0IfhwOx8/Pb+XKlUuXLsVdC2WYm+2ioiInJ6eKs9UqlcpyJ6Ls7e0lEkklr8Rm13qyGzabbTQaqx9zv6ioqGPHjsSBdylKj1KyYcOGOXPm4K6CLEJDQwMCAmQyWUO4VMkizM22QCCo9GSjs7Pzixcvyi4pKioys7/K/CMok8lkNBqrny5HIBBIpdLGjRub2SbJpaSkvHjxYuHChbgLIRFiPCaDwQAHKeYwty8tJCRErVZfu3atdAkxI0xgYKBMJiuNd1paWnZ2dlBQUPWtEVvgSjfvlTIYDMRkGsSHKhaLieUSiaR0YprQ0NDnz5+XHamH0rNSzZkzZ82aNbirIB0/P79yPT6gKuZut3v27Hny5MkNGza8fPnS398/PT390aNHGzdu7Nmz56FDh1avXj1mzBgGg3Hw4EEHB4cP3qnXqlUrFou1bdu2iIgIrVYbGRn5gSr/f1ofb29vNze3gwcPikQilUq1Z88eYnuOEIqOjr5///7SpUuHDh0qEokSExMNBsO3335r5hsklR9//HHKlClubm64CyEdNpt97NixxMTE9u3b466F7MzdbnM4nNWrV/fu3fvq1aubN29OTEwMDw/X6/VsNnvlypXNmjXbvn37tm3bvL29165d+8HxAzw8PGbOnJmVlbVt27YbN25U/+DS9BIf7eLFi9ls9tKlS3fu3Dl27NjSq9M8PDzWr18fGBh46NChP/74o6SkhKI3V9y+ffv9+/cjRozAXQhJeXp6QrDNUYNrV7DQ6XQVu+tqjRLXrsTGxm7evLlityUolZubu2DBgj///BN3IaRG9vvA9Ho9Jaa2s5SYmJjZs2dDsKvn7u7eu3fvM2fO4C6E1Mi+3bYskm+3V6xYERISMmTIENyFADog+3Zbp9PhLqGeHDx4kMfjQbDNl5KSQsyjDipF6mxrtVpKn8cyX1JS0uPHj+k69LKVuLq6Tp8+HXcV5EXqfXKtVmsymTgcjqUaJOc+eXZ2dmxs7MmTJ3EXQj0nTpxo3rx56U2BoCxSZ9viSJhttVrdu3fvW7du4S4E0E2V2S57VhmXZ8+eubm5WTCNRqOx9DIYkujatev58+ft7OxwF0JVe/bsGTlyJKzAiqo83maSwL59+5KTky3YINmCHRUVdeDAAfhe1kVubu6pU6dwV0FG5PqulxMcHEy2XWgL+uyzz9asWUMMyg1q7fPPP09LS8NdBRlVuU8OrCo6OnrZsmXQCQSsh9TnwF69ekXLP8kxMTEQbAtavHgxjIhaEdmzvXPnTtxVWFh0dPSSJUsg2Bbk4eGRmJiIuwrSIfvx9pMnT3BXYUnDhg3bv39/9QPIgJoaP348zFJQERxv158BAwb8+uuvAQEBuAsBDQKp98kRQo8fPy4uLsZdhQV07959x44dEGxr0Ol0M2bMwF0F6ZA928nJybt378ZdRZ0UFxeHhYWdPn3a3d0ddy30ZGNj8+LFC9gtL4fs2R40aBClhxZKTU2NiYl58OCBQCDAXQudHTx4ENZwOXC8bUXx8fGbN2/ev38/7kJAQ0T27TZC6Pnz51Qcf//YsWNHjhyBYNePn3/++dGjR7irIBdSnwMjNG/evEuXLvfu3cNdSA38/vvvMpnsl19+wV1IQ5GTk2P+kNgNBDX2yR88eODk5OTv74+7ELMsXrw4ICBg8uTJuAtpQNLS0hwcHJycnHAXQiLUyDaFTJgwYezYsTA+PsCOAsfbhO+//57kR90lJSVfffXV/PnzIdj1b8WKFefOncNdBblQZrv97NmzAwcOrFy5MiIiQiwWR0dHz507F3dR/5WUlDR79uyjR49+cN4FYEF9+vQhxntWKBQ2NjbERBQCgeDo0aO4S8OPAn1phKCgoPj4+Hbt2jGZTAaDYcFB1Oru4sWL+/fvv3LlCu5CGhyBQJCVlUX8TMzNbjQa27Zti7suUqBAtqOiosRisUKhIMZOIRZaaqaRutu4cSODwdi1axfuQhqigQMHbtmypezsrl5eXtHR0ViLIgsKHG8HBwdzOJzSVBNIMgnzvHnz7O3t4WJmXEaPHu3j41N2SevWrdu0aYOvIhKhQLZXrFgRHR3t4eFRuoTFYpWLOhYjRowYOHDgxIkTcRfScAkEgv79+5dut93d3WGjXQp/QswxadKk+fPnBwQEED1/bDbbxsYGYz0ZGRlhYWHr16/v0aMHxjIAQmjs2LGlm+6QkJDWrVvjrogsqJFthFC3bt3WrVsXFBTEZDJZLBbGvrQbN258/fXX9+/f9/Pzw1UDKCUQCAYNGsRisdzd3UePHo27HBIhRV+avMhgMHx4OHQhz33jhh3r1q17+vSpScsrKazXqcJYLKbAkRUXF3fz5s24uLj6fOlaKynQIYYZj6O4vr2Gnj1xPSAgwMejZT1/K7AgvooffBjm89vXjxa+TJS6+fCK87XmP8ug17PqfaRxkZttfqbKzq0kejbZT7FIxbrbpyRvkmQ+LQWSHA3ucoCFidxs8jPVLcKE3Ya5VPMwbNk26E37Vme2j3Bp5Mfj8KhxaKBRGfMyVIkXC6O/8WGxSbpBLM7XH9uc1XuMp4OrLROm8aYpjdKYm656eLm6ryK2bP/1Q0bXYR7OnrYcp+CcAAAWvUlEQVRYXr0uJDma60dyxy/1xV1IJaRiXdzG98O/ho6ABkGcrbkZlztuSeVfRTzZfnytWKtlBH5ElutPaupFQgmbbWrbU4S7kPLO/5kX+JGjozv1/mKC2km5V2LLMYV2r+SriGdnOOuVSiAiRTde7fAd2FmvyTgx+OsnMgdXCHYDwhey31fxVcR1oMtwdCPRBeE15ejGYZCvA7qkQOfTUgDH2A2KYyMOMlX+VcST7aJ8jZEi959VymgyFeWRr/+ZgYpyyVcVsCaj0VRUUPmHTo0OagBATUG2AaAnyDYA9ATZBoCeINsA0BNkGwB6gmwDQE+QbQDoCbINAD1BtgGgJ8g2APREmWwbDIbk5Md1bOTX39YMG/GJhSqiLYus6ko9T3mq0cAV7/WEMtle99O/NvyyCncVDYKVVvW58yenz5ioVpPx3lhaoky2tfD3vr58cFXXbjyPumyxLTKCCFWmvrMUamT7x7XLr167mJ7+tmfvsJ69w3JysxFCer1++783jRjVL6Jvp6mfj4m/da308c9Tns6aPbVv/85DhvZes/Z7qUxaabP7D+weNTqy/4DwmV9NSXyYUI9viCzu3o2fPPXTfpFdJk4eGXfs76pWNXEsc/v2jZjxQ3v2Dnv46P6OnZs/6fdxaTsvUp/37B12L+E28d/k5Mfz5n8ZObBr5MCui5bMfvnqxbnzJ3/59UeEUNSwPj17h507fxIhVE0jFV8RIfTo8YMvZ0zs27/z6LED16z9XiwurP7dXbt+qWfvsPj4azO/mhLRt9Ou3VuJacM2/f7T0OERAwZ1m/bFuCtXLxAPfvcuY87caf0HhI8aHbnh51VGoxEhNGhIj/kLps+YNblfZJdPxwzYuWuLXq8nHi8WF678YcmgIT36DwhfsHDG27evieVHju7/csbEq9cuxoyL6j8gfNbsqZmZ6VWt7WrqqTtqDH4SM3ZyQX5eTs77Rd+sQAg5O7kghNb/tPLS5bMx0ZP9/Jpeunx22bfzfv15e3Bw2/T0t3PnTfPza7pg/nclxUW7dm/Nz8/9af2Wcm0mPkzY/u9NvXv3+6hD54T7t1VKJaY3h41Go1m+YqGfr//cOUvT0l6LxQVVrWqEkEIh37Fr8+yvvlGrVe3adnj8+EFVzd5/cHfR4q+a+jebFjvbaDTeuXPDoNd/1LHLqJExhw7vXf3DL3y+wNvbp6qnlyr3iokPE75ZNCuiT+TQqE9l0pKjcQfmzJu2bcteLpdbfTu/blwzdfL0yZO+8PbyMRqNS5Z+nZubHT12kkjk9Pjxg3+tXKxWqyL7D1n3078yM9OnfzlXqVQ8evygdOKazHfpX0z72sXZ9c7dm/v275LLZbNmLlCr1XPmTZNKSz7/bBaXwz3w954586b99ecxe4E9Qigl5emhQ3/NnbtUr9dv2PDD6jXfbfl9j1KprLi2q6mnhh9mJaiRbW9vHwcHkaRI3KZNKLEkMzP9/IVT48dNnTghFiHUvVvvmPFDd+/ZtuGnrXv37WAymWvXbCJWtL29cNWP3z558jAkpF3ZNnNzsxFCQ4eMCgoKjoiIxPTOcFIqFRqNpmvXXhF9+pcurLiqCVqtdt6cpYGBH561Y9Pv693dPTf+tpOYMTdqyEhiuaenN0IoMLC1g4NZ48yVe8WNm9YNGjhs1swFxH/DwjpNmDTi/oM7XcN7Vt/O0KhP+/YdSPx87fqlpORHB/addHFxRQj16d1PpVIejTsQ2X9Ibm5282YtBw4YihAaNTKm9Ok9ukf06N4HIdS6dYhUWnLyVNyECbE3blzOzEz/af2Wdm07IITatGk7NmZwXNzBCeM/I571w8qfnZycEULDho3evOXnEmmJXC6ruLZv3LxSVT3mrKLqUSPbFT1JeogQCv//z5XBYHQI63Tx0hmE0OMniW3bdiCCjRDq0OFjhFDqy+flst3po3B7e+Gq1ctmzpjfqVM4jjeBmUjkGBQUvHffDi6XN2jgMCKKVeFyueYEOyc3OzMzfeqU6dW3Zo6yr5ibm5ORkfb+/btTp4+VfUx+ft4H22nXrmPpz3fvxuv1+rExg0uXGAwGPl+AEIroE7n/wO7fNq4dFzPV0dGp0qY6dux86vSxV69ePHmSKOALiGAjhNzdPXx8/FJfPi9TPI/4oVEjD4SQuLCgSZOmFdd2NfXUHVWzrVDIEUKOov9+BkKhg1KpVCgUCoVc5PDfCe7t7YUIocLCgnItODu7bPpt5+9bNixaMrt165Bvl652dXWrx3eAH4PB+HHVb//esWnrtl8OH9m7aOGKcn/+yuLx7Mxps7hIghByc21U9/LKvmJRkRghNGH859269ir7GCen6gbfJ9j9bzvOzi4b1m8t+wBiHoupU6Y7Ojrt3bfz7LkTn382a2jUqIpNCQT2CCGVSilXyB1EjmV/JRQ6iCt8xxBCNmwbhJDBaKh0bVdTT91Roy+NULaf08XFDSEklZaULpFIxGw2m8vluri4lV1eVCQp/VTK8fHxW7P6t5/Wb0lLe71m7XLrvwPSEQgEs7/6Zs/uo3y+YOmyOcr/73T4YJdy2SmvyyK2OZIicVVPLNtyVY1UVqc9QkijUfv4+JX9JxDUbBNnby8sLi5q1MijbCNent5EMSOGj9331/Eunbv/tnFtpWf4CwvyEUKuro1c//c7Rnz9Kv2O/e+7KL+2q6mn7iiTbS6XJ5GIid5L4rCNwWDcvRdP/Fer1d69Fx8UFMxisYKCgh8/SVSr1cSvbty4jBAijh5tbGxVKmVpV6dWq0UItWvboVOnri9fvcD0znAizkt5engNGzparpATfRDlVnWlHBwcdTpdyf9/v4knIoQaN/Z1dXU7f+FU6Uo2mUxEUzwur9wOVFWNVOTt7dOokfvZcydUqv+cHtfr9Tpdjaf+ateuo8FgOHHySOmS0gaJVcHn8ydOnIYQqvh9MJlMZ8+dsBfY+/o0CQoKlsmkKSlPiV+9efPq/ft35XooKqq4tqupp+4os08eEtzu7LkTG35e1aZ1qL29sHPnbn0/Gbh7zzaDweDp6X369DGJRLx40b+Int4rV84vXDRz0MDh+fm5e/78o21oWGhIe4RQs4AWarV6+YqFX0z7Wiot+X7Fwqgho3g8u4SE2y1btML9FuubXq+fMGl4j+4RTfyaHj9+WMAXEN1dFVd1xeeGtf+IwWBs+n39iOFj09PebNv+G7GcwWB8/tmsH1YtnT5jYt++g5hM5oWLp4cOGRURERnUOoTFYm3avL5/38EarWbwoOFVNVIRg8GY/uXcb7+bP33mxMGDRhgNhvMXTkVERI4YPrZGbzmiT+TJU3Fbt/2ak5vdvFnL169fxt+6unvnES6Xu3zFQgFfENa+E7HBaNE8kHjK1WsXnJ1dOBzu9euXHj1+EPv5LB6P16d3/337dy1fsXBczFQmk/nXX/8WiRyHDB5ZzUvrdLqKa7txY9+q6qnR+6oUa/lyDPuiSTdLmrSx5/BqMJS2v3+ATFZy+cq5J0kPHRxE7dt17BD2sUIhP3vu+JUr5/l2/HlzlxLdZkKhQ5vWbe8/uHPy1NHUlyk9e3wyf963xJy+TZo0VatV9+/fCWwR5OAgevPm5dWrFx4+TAgJaff17MXm92FoVMa0ZFlIN3LNK6JRGlMfyAI/MrcqjUaTmZkef+vqzfgrzs6u3yxY7uXlXemqvnfvVkZG2qejxpU+VyRy9HD3unz5bNyxg0qlYuSI6Phb1/r06e/t1djfPyAgoPmTJ4kXL515+TLFy6txeHhPV1c3ob3Q1bXRtWsX79y5KZNJ+/YdWE0jFV/R16dJyxatkpIeXbh4OuXF06b+zSIiBjg7V3e8nZ7x9vr1S0OjRpX2zLNYrB7dI+Ry6bVrF2/cvKJQyvv3G9KmTSiTyczOzrp7L/7ylXMqterzz2aGh/dACB04uNvDwyv15fNLl88ihKLHThr96XiEEJPJ7Pxxt7S01ydOHrl371bz5oHfLlvt7u6BEHqeknz//p3osZOIKeKzsjIvXzk/aNBwDpeblZVZbm1XU4/5H3r6M1lw10o+dDxzBv31Q0avsZ5CJ5v6f2mLkEp0V/ZljyPZlGAlhbrjW7KHziJXVZQ2aEiPyP5RX0ybjbuQKpUU6q4dyo5ZVMmHTpl9cgAqksvlY6IHVvqr2M+/Ik5WN1iQbUBhdnZ2f2zbX+mvhPZUnUnSUiDbgMKYTKaHu6f12j95/JoZjyIpypwDAwDUCGQbAHqCbANAT5BtAOgJsg0APUG2AaAnyDYA9ATZBoCeINsA0BNkGwB6wpNtJ3dbptljbpAQk8FwcufgrqICE8PJg3xVAWtiMpFjo8o/dDzZZjCROEeN5aUtQpKrQQzSDWTv4Mp+l6ow6EhXGLAecY6mqnu98WTbp7mdvEiP5aUtQl6sa9zcrLEB61mztoKiPC3uKkD9URTrvJvxKv0Vnmy37uKQmSrLeC7H8up1lJmiSH8mC+5KxlsIw6NcL+59j7sKUE8ynsszU+VtulT+VcQz7gpCyGRCR39736SNvVtjrsitrmNZ14/ifG1+pjrtqXTELG9E1u4Clcywe0V6r9GeDq42fAe4h5eeivK0+Zmq9Gey4bO8q+q5wpZtwoOLRS8fyjg8ZuF7sk/l5+LF0aiMzdvah33iaMbDcTLoTPEnCtOeKoTONgXvKNyvYT6j0chgMMwfFJnSnD25WrWheTv7sIjqvoqYs00wGJBRj7+M6jHZDFYNxm4kBZ2G7GvVUlauXNmxY8dPPmkQk6ub+VUkxT4bi4VYrAbxF7ee2XAaylo1MXQMlqHhvF9zwLUrANATZBvQgUgkIsYDB6Ug24AOiouLazGFEL1BtgEdODs7131WYJqBbAM6EIvFxMyNoBRkG9ABbLcrgmwDOoDtdkWQbUAHtra25s992UDA6gB0oNVqjUYj7irIBbINAD1BtgEduLi4QF9aOZBtQAeFhYXQl1YOZBsAeoJsAzoQCoVwPXk5kG1AB1KpFK4nLweyDQA9QbYBHcC1KxXB6gB0ANeuVATZBnTQQEZBrBHINqADMgzpSTaQbQDoCbIN6IDL5UJfWjmwOgAdqNVq6EsrB7INAD1BtgEdODg4wH1g5UC2AR2UlJTAfWDlQLYBoCfINqADGOe0Isg2oAMY57QiyDYA9ATZBnRgY2MDl5SXA9kGdKDT6eCS8nIg24AOYJzTiiDbgA5gnNOKINuADmAsxIog24AOYCzEiiDbgA4EAgGbzcZdBblAtgEdyOVyvV6PuwpygWwDOoBrTiuCbAM6gGtOK2LAGX9AXYMGDcrJySkdC5HBYJhMptDQ0B07duAuDT/YbgMK6969e2mqiWtORSLRpEmTcNdFCpBtQGHR0dFeXl6l/zWZTM2aNQsPD8daFFlAtgGFeXh4dOvWrfS/Dg4OMTExWCsiEcg2oLYxY8b4+fkRG+0WLVrARrsUZBtQm5eXF7Hpho12OZBtQHkjR4709vYOCAjo0qUL7lpIBM6BgXqVm65+m6zKzVSpZAaVQm/LYyuKLHBe2mgwMBgMhiWmFnH04KmkWq6ALXKxdfe1bRosEDpT8mpWyDaoD3qd6e7Zoud3im3tbOxdBbZ2bDaHxbZls2yZiGzTgTCRXm3Qaw0GnUEuVsnFSlsuM7SbQ0g3B9yV1QxkG1hd/HFJ0s0iz5YuAlc7ti31DgPVcl3xe6lcrAwf4tIyTIC7HHNBtoEViXMNZ/fk2vK5bk1FuGupK51an/dKwrNDQ6Z5UOKWM8g2sJb3r9SnduQEdPZm2VBvW12VkjyFJKNowjJf8s8aCtkGVlGQpTu7J8+nnQfuQixPo9AVvikcPdeLxSb1yKqk/+MDKEiSqz25PZuWwUYIcfg2rs1c9/wrA3chHwDZBpa3f02m/0feuKuwIlse27Wp87HN2bgLqQ5kG1jY6X/n+bX3QKTeXbUAe1c7vdHm6W0p7kKqBNkGlpT9Vl2YpxM4c3EXUh+cfUXx/xTgrqJKkG1gSdePFrr6O+Guop4w2UwnH4d754pwF1I5yDawmLwMjdHItBNxcBdSiXsPjs9b9pFUWmjZZp19HJ7fI+luOWQbWMybZLmtgIzBth6WDZPBYuakqXEXUgnINrCYN08U9q52uKuob3wn/pskBe4qKkGFa+cAFShlBpYti2tvlYGEtVr12UtbHiWd1+k0ri6+PcKjQ9tEIIRu3D7wOPlSt85jzl7aIpMVenm2HDlkkZurH/Gs99mp/5zZ8O79c6G9i6uzjzUKIzrMxblkPOSGbAPLUMoMGpXBGi0bjcad++YWFeX06jZBIHB68zZx76GlGq3qo/aDEUKZWU+v39o3cshig0F/5MTqg3ErZsXuRAjlFaRv2fkF304UGfEli8m+eM1aI5+ybZhZ78i4Tw7ZBpahlOptOCxrtJz8/Gpa+uPFc/9xELoihNoF99VolfF3/iayjRCaFL1eaO+MEArvNOrkuV8VyhK+ncPp8xsZDObM2B0CviNCiMFkxp1ca43y2ByWWmGVP2p1BNkGlqFWGrnW6UhLSb1lMOpXbRhausRoNPC4/73XkmPLI35wFHkghKTSAhs2J/X13Y87DCeCjRBiMa34VXdpbKcsMdg5WOVPW61BtoFl2Ngy1AqrzOwhk4uF9i7TJv1ediGzsqyyWTZE8qWyQoNB7+RYTxe0i98ruQLSdUtDtoFl8IVsvcYqu6Z2PKFcUeQo8rCxMXe/gNhcy+X10cVl0BnZNkwmi3QX2ZLujw2gKL6QrddaJdsBTTsYjYbbCUdLl2i0quqfwuXyXZwbP3l2Wa+3+qTceo1B4GBj7VepBdhuA8vgi1jIZNJrDGxL96i1D+l/78E/p85vLCrO8fJokZ37Kvn5tQWz/ra1re6q9U96Tt1/5LuNf0zt2G4gg8m8eedvy1ZVSlmidvEm4xU7kG1gMX6t+NICpZO3vWWbZbNtPpvw25kLvz9KunDn/jFXZ5/OHYexWB/46rYL6adSya7d2nfqwsZGrv6+jVsXFFrljmuFRBnSn4zDJMK4K8Bi0p8pbp4qaRzcCHch9erphbQZPwfgrqISsN0GFuMXxL/xj8SgM1YzQNrSH3pXulxgJ5IriysuD2rZbczw7yxVoUot/+GnIZX+yrdxm4x3yRWX83kOi+bEVdVgSY6iZUcybrRhuw0s7NldadJtlUegS1UPkBRVPlaJXq9jsyvpkbK15ZWeo647o9FYXJJb+e9MDMSoJAsMBtNR5F5Vg6k3MsYv8eUJyHVmmwDbbWBJQZ2E988XaRQ6Dr/yrmMnR896L+q/mEymBQsQZ5S0CBOSM9hwDgxYXv+J7oVpYtxVWJ1BZ1SI5T2GV7mHgh1kG1hYI19OcBf7vFQLj4JANm/uZI2Y5YW7iupAtoHlteksbBbCzU6hbbzfJeVGTfe0syfp3jgBsg2sol1PB/9A25zn5B0qsHYMOuOr+Mz+41zdSHm9SlnQTw6sKCVBlnRLbu/uQM5B1GqqKEuW91oSs8hHIKJAJzRkG1iXOFt7YX++wcB0C3C2taNAJColK1DmvZI0bsbrO94Ndy3mgmyD+pD2TPHwqrSkUMd3tnNoJODw2Qwm6W6cKsdoMCkkKlmBUlao9GzC6zrUWeRKxntCqgLZBvWnMFv7+on8Xao6/52SxWbaclk8BxutdUZiqjU7e05JgVKrMvAdbOwd2S3aC5q05pO826xSkG2Ah0ZpVEj1WpXRSLJvIIvF5PKZfAc224bsexbVg2wDQE9wDgwAeoJsA0BPkG0A6AmyDQA9QbYBoCfINgD09H8glswvq62G0wAAAABJRU5ErkJggg==",
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from IPython.display import Image, display\n",
        "\n",
        "display(Image(graph.get_graph(xray=True).draw_mermaid_png()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Run scenarios\n",
        "\n",
        "Note: LLMs are fundamentally probabilistic so wrong answers are possible even if implemented correctly.\n",
        "\n",
        "## Scenario 1 - name of wagon leader\n",
        "\n",
        "This test confirms that our graph has been setup correctly and can handle a case where tools don't need to be invoked."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            " Question: What is the first name of the wagon leader? \n",
            "\n",
            "\n",
            " Agent response: Art\n",
            "\n"
          ]
        }
      ],
      "source": [
        "scenario = {\n",
        "    \"question\": \"What is the first name of the wagon leader?\",\n",
        "    \"answer\": \"Art\",\n",
        "    \"type\": \"free-form\",\n",
        "}\n",
        "\n",
        "print(f\"\\n Question: {scenario['question']} \\n\")\n",
        "\n",
        "res = graph.invoke({\"messages\": scenario[\"question\"]})\n",
        "\n",
        "print(f\"\\n Agent response: {res['messages'][-1].content}\\n\")\n",
        "\n",
        "assert res[\"messages\"][-1].content == scenario[\"answer\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Scenario 2 - restocking tool\n",
        "\n",
        "In this test we want to see the agent choose the restocking tool and choose to use the multiple choice output."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            " Question: In order to survive the trail ahead, you'll need to have a restocking strategy for when you need to get more supplies or risk starving. If it takes you an estimated 3 days to restock your food and you plan to start with 200lbs of food, budget 10lbs/day to eat, and keep a safety stock of at least 50lbs of back up... at what point should you restock? \n",
            "\n",
            "\n",
            " Using restock tool!: daily_usage=10, lead_time=3, safety_stock=50 \n",
            "\n",
            "Called multi choice structured\n",
            "\n",
            " Agent response: D\n"
          ]
        }
      ],
      "source": [
        "# helper function for multi-choice questions\n",
        "def format_multi_choice_question(q):\n",
        "    question = q[\"question\"]\n",
        "    options = q.get(\"options\", \"\")\n",
        "    formatted = f\"{question}, options: {' '.join(options)}\"\n",
        "    return [HumanMessage(content=formatted)]\n",
        "\n",
        "scenario = {\n",
        "        \"question\": \"In order to survive the trail ahead, you'll need to have a restocking strategy for when you need to get more supplies or risk starving. If it takes you an estimated 3 days to restock your food and you plan to start with 200lbs of food, budget 10lbs/day to eat, and keep a safety stock of at least 50lbs of back up... at what point should you restock?\",\n",
        "        \"answer\": \"D\",\n",
        "        \"options\": [\"A: 100lbs\", \"B: 20lbs\", \"C: 5lbs\", \"D: 80lbs\"],\n",
        "        \"type\": \"multi-choice\",\n",
        "    }\n",
        "\n",
        "print(f\"\\n Question: {scenario['question']} \\n\")\n",
        "\n",
        "res = graph.invoke({\"messages\": format_multi_choice_question(scenario)})\n",
        "\n",
        "print(f\"\\n Agent response: {res['multi_choice_response']}\")\n",
        "\n",
        "assert res[\"multi_choice_response\"] == scenario[\"answer\"]\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Scenario 3 - retriever tool\n",
        "\n",
        "In this test, we want to see the retrieval tool invoked and multiple choice structured response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            " Question: You’ve encountered a dense forest near the Blue Mountains, and your party is unsure how to proceed. There is a fork in the road, and you must choose a path. Which way will you go? \n",
            "\n",
            "Called multi choice structured\n",
            "\n",
            " Agent response: B\n"
          ]
        }
      ],
      "source": [
        "scenario = {\n",
        "        \"question\": \"You’ve encountered a dense forest near the Blue Mountains, and your party is unsure how to proceed. There is a fork in the road, and you must choose a path. Which way will you go?\",\n",
        "        \"answer\": \"B\",\n",
        "        \"options\": [\n",
        "            \"A: take the northern trail\",\n",
        "            \"B: take the southern trail\",\n",
        "            \"C: turn around\",\n",
        "            \"D: go fishing\",\n",
        "        ],\n",
        "        \"type\": \"multi-choice\",\n",
        "    }\n",
        "\n",
        "print(f\"\\n Question: {scenario['question']} \\n\")\n",
        "\n",
        "res = graph.invoke({\"messages\": format_multi_choice_question(scenario)})\n",
        "\n",
        "print(f\"\\n Agent response: {res['multi_choice_response']}\")\n",
        "\n",
        "assert res[\"multi_choice_response\"] == scenario[\"answer\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Scenario 4 - Semantic caching\n",
        "\n",
        "Agent workflows are highly flexible and capable of handling a wide range of scenarios, but this flexibility comes at a cost. Even in our simple example, there can be multiple large-context LLM calls in the same execution, leading to high latency and increased service costs by the end of the month.<br>\n",
        "\n",
        "A good practice is to cache answers to known questions. Chatbot interactions are often fairly predictable, particularly in support or FAQ-type use cases, making them excellent candidates for caching.\n",
        "\n",
        "\n",
        "![diagram](../../assets/cache_diagram.png)\n",
        "\n",
        "## Creating a cache"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "09:20:47 redisvl.index.index INFO   Index already exists, not overwriting.\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "'oregon_trail_cache:602ac35f09671fc9e2a4f4902c6f82f06b9560ea6b5a5dd3e9218fcc1ff47e52'"
            ]
          },
          "execution_count": 43,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import warnings\n",
        "from redisvl.extensions.llmcache import SemanticCache\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "\n",
        "hunting_example = \"There's a deer. You're starving. You know what you have to do...\"\n",
        "\n",
        "semantic_cache = SemanticCache(\n",
        "    name=\"oregon_trail_cache\",\n",
        "    redis_url=REDIS_URL,\n",
        "    distance_threshold=0.1,\n",
        ")\n",
        "\n",
        "semantic_cache.store(prompt=hunting_example, response=\"bang\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Testing the cache"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            " Question: There's a deer. You're hungry. You know what you have to do... \n",
            "\n",
            "Cache hit\n",
            "Response time 0.18901395797729492s\n",
            "\n",
            " Question: You’ve encountered a dense forest near the Blue Mountains, and your party is unsure how to proceed. There is a fork in the road, and you must choose a path. Which way will you go? \n",
            "\n",
            "Invoking agent\n",
            "Called multi choice structured\n",
            "Response time 3.500865936279297s\n"
          ]
        }
      ],
      "source": [
        "import time\n",
        "\n",
        "scenarios = [\n",
        "    {\n",
        "        \"question\": \"There's a deer. You're hungry. You know what you have to do...\",\n",
        "        \"answer\": \"bang\",\n",
        "        \"type\": \"cache_hit\",\n",
        "    },\n",
        "    {\n",
        "        \"question\": \"You’ve encountered a dense forest near the Blue Mountains, and your party is unsure how to proceed. There is a fork in the road, and you must choose a path. Which way will you go?\",\n",
        "        \"answer\": \"B\",\n",
        "        \"options\": [\n",
        "            \"A: take the northern trail\",\n",
        "            \"B: take the southern trail\",\n",
        "            \"C: turn around\",\n",
        "            \"D: go fishing\",\n",
        "        ],\n",
        "        \"type\": \"multi-choice\",\n",
        "    }\n",
        "]\n",
        "\n",
        "for scenario in scenarios:\n",
        "    print(f\"\\n Question: {scenario['question']} \\n\")\n",
        "\n",
        "    start = time.time()\n",
        "\n",
        "    cache_hit = semantic_cache.check(prompt=scenario[\"question\"], return_fields=[\"response\"])\n",
        "\n",
        "    if not cache_hit:\n",
        "        print(\"Invoking agent\")\n",
        "        res = graph.invoke({\"messages\": format_multi_choice_question(scenario)})\n",
        "    else:\n",
        "        print(\"Cache hit\")\n",
        "\n",
        "    response_time = time.time() - start\n",
        "\n",
        "    print(f\"Response time {response_time}s\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Scenario 5 - Allow/block list router\n",
        "\n",
        "When ChatGPT first launched, there was a famous example where a car dealership accidentally made one of the latest language models available for free to everyone. They assumed users would only ask questions about cars through their chatbot. However, a group of developers quickly realized that the model was powerful enough to answer coding questions, so they started using the dealership's chatbot for free. <br>\n",
        "\n",
        "To prevent this kind of misuse in your system, adding an allow/block router to the front of your application is essential. Fortunately, this is very easy to implement using `redisvl`.\n",
        "\n",
        "![diagram](../../assets/router_diagram.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Creating the router"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "10:35:18 redisvl.index.index INFO   Index already exists, not overwriting.\n"
          ]
        }
      ],
      "source": [
        "from redisvl.extensions.router import Route, SemanticRouter\n",
        "\n",
        "# Semantic router\n",
        "blocked_references = [\n",
        "    \"thinks about aliens\",\n",
        "    \"corporate questions about agile\",\n",
        "    \"anything about the S&P 500\",\n",
        "]\n",
        "\n",
        "blocked_route = Route(name=\"block_list\", references=blocked_references)\n",
        "\n",
        "router = SemanticRouter(\n",
        "    name=\"bouncer\",\n",
        "    routes=[blocked_route],\n",
        "    redis_url=REDIS_URL,\n",
        "    overwrite=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Testing the router"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 53,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            " Question: Tell me about the S&P 500? \n",
            "\n",
            "Blocked!\n"
          ]
        }
      ],
      "source": [
        "scenario = {\n",
        "        \"question\": \"Tell me about the S&P 500?\",\n",
        "        \"answer\": \"you shall not pass\",\n",
        "        \"type\": \"action\",\n",
        "    }\n",
        "\n",
        "print(f\"\\n Question: {scenario['question']} \\n\")\n",
        "\n",
        "blocked_topic_match = router(scenario[\"question\"], distance_threshold=0.2)\n",
        "\n",
        "assert blocked_topic_match.name == \"block_list\"\n",
        "\n",
        "print(\"Blocked!\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Putting it all together\n",
        "\n",
        "Once you have defined all the pieces, connecting the various aspects of the full architecture becomes easy and you can tie them together with whatever logic you wish. \n",
        "\n",
        "This could be as simple as:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 54,
      "metadata": {},
      "outputs": [],
      "source": [
        "def respond_to_question(question):\n",
        "    blocked_topic_match = router(question, distance_threshold=0.2)\n",
        "\n",
        "    if blocked_topic_match.name == \"block_list\":\n",
        "        print(\"App block logic - short circuit\")\n",
        "        return\n",
        "\n",
        "    cache_hit = semantic_cache.check(prompt=question, return_fields=[\"response\"])\n",
        "\n",
        "    if cache_hit:\n",
        "        print(\"Cache hit - short circuit\")\n",
        "        return cache_hit\n",
        "    \n",
        "    return graph.invoke({\"messages\": question})\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.9"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}