mRAG.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "ijGzTHJJUCPY",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Copyright 2024 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NDsTUvKjwHBW",
    "tags": []
   },
   "source": [
    "<!--\n",
    "<span style=\"font-size:2em;\">A Richer Knowledge Base for LLMs using mRAG</span> \n",
    "\n",
    "\n",
    "<table align=\"left\">\n",
    "  <td style=\"text-align: center\">\n",
    "    <a href=\"https://colab.research.google.com/github/Fady-Ibra/RAG/blob/master/mRAG.ipynb\">\n",
    "      <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
    "    </a>\n",
    "  </td>\n",
    "  <td style=\"text-align: center\">\n",
    "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FFady-Ibra%2FRAG%2Fmasterm%2FRAG.ipynb\">\n",
    "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
    "    </a>\n",
    "  </td>\n",
    "  <td style=\"text-align: center\">\n",
    "    <a href=\"https://github.com/Fady-Ibra/RAG/mRAG.ipynb.ipynb\">\n",
    "      <img width=\"32px\" src=\"https://upload.wikimedia.org/wikipedia/commons/9/91/Octicons-mark-github.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
    "    </a>\n",
    "  </td>\n",
    "  <td style=\"text-align: center\">\n",
    "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/Fady-Ibra/RAG/master/mRAG.ipynb\">\n",
    "      <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
    "    </a>\n",
    "  </td>    \n",
    "</table>\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YR65ni7TRNYG"
   },
   "source": [
    "- Authors: [Lavi Nigam](https://github.com/lavinigam-gcp) & [Fady Ibrahim](https://github.com/Fady-Ibra)\n",
    "- Forked from: [Google Cloud Gen AI Repo](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/qa-ops/building_DIY_multimodal_qa_system_with_mRAG.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VK1Q5ZYdVL4Y",
    "tags": [],
    "toc-hr-collapsed": true
   },
   "source": [
    "# What We will Do\n",
    "\n",
    "## Overview\n",
    "- We'll create a QA sys that understands both text and images\n",
    "- We'll build this system using Vertex AI\n",
    "\n",
    "* **Focus on Fundamentals**: We will start with the essential design pattern of \"Retrieval Augmented Generation\" (RAG) – a way to find and use relevant info to answer questions.\n",
    "* **Work with Text and Images**: We will expand RAG to handle both text and images found in PDF documents.\n",
    "* **Use Vertex AI**: We will use Vertex AI Embeddings API and Vertex AI Gemini API.\n",
    "\n",
    "By the end of this guide, we will have a solid foundation in building multimodal QA systems.\n",
    "\n",
    "## Gemini\n",
    "- Gemini is a family of GenAI models that is designed for multimodal use cases. \n",
    "- The Vertex AI Gemini API gives us access to:\n",
    "    - Gemini 1.0 Pro Model\n",
    "    - Gemini 1.0 Pro Vision Model\n",
    "    - Gemini 1.5 Pro Model\n",
    "    - Gemini 1.5 Flash Model\n",
    "\n",
    "## Comparing text-based and mRAG\n",
    "Multimodal RAG (mRAG) offers several advantages over text-based RAG:\n",
    "1. **Enhanced knowledge access:** mRAG can access and process both textual and visual info, providing a richer and more comprehensive knowledge base for the LLM.\n",
    "2. **Improved reasoning capabilities:** By incorporating visual cues, mRAG can make better informed inferences across different types of data modalities.\n",
    "\n",
    "## How to implement RAG \n",
    "We will implement RAG using:\n",
    "- Vertex AI Gemini API\n",
    "- Vertex AI Embeddings API\n",
    "  - [text embeddings](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text-embeddings)\n",
    "  - [multimodal embeddings](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/multimodal-embeddings), to build a doc search engine.\n",
    "\n",
    "## Objectives\n",
    "This notebook provides a guide to building a doc search engine using mRAG, step by step:\n",
    "1. Extract and store metadata of docs containing both text and images, and generate embeddings the docs\n",
    "2. Search the metadata with text queries to find similar text or images\n",
    "3. Search the metadata with image queries to find similar images\n",
    "4. Using a text query as input, search for contextual answers using both text and images\n",
    "\n",
    "## Costs\n",
    "- [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing)\n",
    "- [Pricing Calculator](https://cloud.google.com/products/calculator) (generates a cost estimate)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DXJpXzKrh2rJ",
    "tags": [],
    "toc-hr-collapsed": true
   },
   "source": [
    "# Getting Started\n",
    "## Install Vertex AI SDK and other dependencies\n",
    "**Next Command Explanation**\n",
    "- **!** to execute a command\n",
    "- **pip3** a Python Package Manager (with a recursive acronym PIP Installs Packages)\n",
    "- **--upgrade (-U)** INSTALL OPTIONS upgrade all packages to the newest available version\n",
    "- **--user** install packages in the user home directory (pip defaults to installing Python packages to a sys directory, as /usr/local/lib/python3.4, which requires root access)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "kc4WxYmLSBW5",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-aiplatform\n",
      "  Downloading google_cloud_aiplatform-1.61.0-py2.py3-none-any.whl.metadata (31 kB)\n",
      "Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0dev,>=3.20.2 (from google-cloud-aiplatform)\n",
      "  Downloading protobuf-5.27.3-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)\n",
      "  Downloading protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (679 bytes)\n",
      "Downloading google_cloud_aiplatform-1.61.0-py2.py3-none-any.whl (5.1 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.1/5.1 MB\u001b[0m \u001b[31m19.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hDownloading protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m50.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: protobuf, google-cloud-aiplatform\n",
      "  Attempting uninstall: google-cloud-aiplatform\n",
      "    Found existing installation: google-cloud-aiplatform 1.60.0\n",
      "    Uninstalling google-cloud-aiplatform-1.60.0:\n",
      "      Successfully uninstalled google-cloud-aiplatform-1.60.0\n",
      "\u001b[33m  WARNING: The script tb-gcp-uploader is installed in '/home/jupyter/.local/bin' which is not on PATH.\n",
      "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "apache-beam 2.46.0 requires grpcio!=1.48.0,<2,>=1.33.1, but you have grpcio 1.48.0 which is incompatible.\n",
      "google-cloud-pubsub 2.21.4 requires grpcio<2.0dev,>=1.51.3, but you have grpcio 1.48.0 which is incompatible.\n",
      "tensorflow 2.11.0 requires protobuf<3.20,>=3.9.2, but you have protobuf 3.20.3 which is incompatible.\n",
      "tensorflow-serving-api 2.11.0 requires protobuf<3.20,>=3.9.2, but you have protobuf 3.20.3 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0mSuccessfully installed google-cloud-aiplatform-1.61.0 protobuf-3.20.3\n"
     ]
    }
   ],
   "source": [
    "!pip3 install --upgrade --user google-cloud-aiplatform | grep -v \"already satisfied\"\n",
    "!pip3 install --upgrade --user pymupdf                 | grep -v \"already satisfied\"\n",
    "!pip3 install --upgrade --user rich                    | grep -v \"already satisfied\""
   ]
  },
  {
   "attachments": {
    "e882374e-dcbc-46b4-b351-d3c3b2791d00.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnkAAAFlCAYAAACX9f7cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAMTgAADE4Bf3eMIwAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13eJPl18Dx70naMlr23nsjbkEQRFEQRcWtqLjQn7gV0FfFLeKqW5x1L1RQQQXFgYCiKCqKIsjeexc6kpz3j/tJ2tIUWjrS1vO5fK6kz7wTH5KTe5xbVBVjjDHGGFO++GJdAGOMMcYYU/QsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGGGOMKYcsyDPGlHoiEi8iraOs7ykiV4tI11iUyxhjSjML8owxZcEJwAIRaRZeISI3ANOAZ4Afvb+NMcZ4LMgzxpQFPYB/VHUZgIg0BB4EJgCtcIHe3SJSMXZFNMaY0sWCPGNMWVAPWJzt7+uBIDBEVRcDjwHVgJYxKJsxxpRKFuQZY8qCVcABIuITkXrAFcArqrrB277Le0yKSemMMaYUsiDPGFMWfAA0Af4GZnvrHsq2vb33uKYkC2WMMaWZBXnGmFJPVecAZwJrgblAX1VdmW2XM4CfVXVFLMpnjDGlkahqrMtgjDHGGGOKmNXkGWNKPRE5SURu2sv2JiLykYhUKclyGWNMaWZBnjGmLDgAOH4v24PAQKBjyRTHGGNKv7hYF8AYY/LizXJRGzfoorqIdIuyWzVgsPd8U0mVzRhjSjsL8owxpdmlwC1ktTrM3Mu+76jqwuIvkjHGlA028MIYU6qJSH3gaaAdLunxnnYBC1X11xItmDHGlHIW5BljSj0RaY6b3WJkjItijDFlhgV5xpgyRUQEN81ZhqpujnV5jDGmtLLRtcaYMkFEmovIWGAnbmaLUdm2jRCRe2JWOGOMKYUsyDPGlHoiUgP4FjgJmAws22OXdcD/iYjNXWuMMR4L8owxZcFFQFWgk6qeAfy+x/Y/gQSgdUkXzBhjSisL8owxZUFH4BdV3bMGL8zSQRljzB4syDPGlAX/Aj1FpFYe2/sBGcDfJVckY4wp3SzIM8aUBe8Du4EvRORaoDpuBoyeIvIgcAfwlKpmxLKQxhhTmlgKFWNMmSAiRwEvkHt+2hDwKnC1qqaXeMGMMaaUsiDPGFNmiIgPOAroANQFlgCzVHVBTAtmjDGlkAV5xphST0QSVTV1H/u0BZZZbZ4xxjjWJ88YUxZcJSIjom0Q5xpcWpWKJVssY4wpvSzIM8aUBauAh0XkZRGJD68UkcbAl8DTwBfArhiVzxhjSh3LLWWMKfVU9R0RUSAFaCEiZ+Jmv3gaCAKDVPXdWJbRGGNKG+uTZ4wpM0TkYOBj3OwX1YHxwFWqui6mBTPGmFLIgjxjTJkiInWAD4E2QFtV3RnjIhljTKlkzbXGmFJHRM4Hog20UFy/uzSgHjBXRLZm236UBX3GGONYkGeMKY3Sga172e4HpkdZb00TxhjjseZaY4wxxphyyFKoGGNKPRHxe7NdGGOMySf70DTGlAVnActEpGq0jSJyp4gstEDQGGOy2AeiMaYs6AbsVNXteWyfAbQCWpdckYwxpnSzIM8YU6JEZH8GfCWy99kstnmPUWv6jDHmv8iCPGNMsRCR5iJyk4jUzbauDvDqfpxuHtDRm8Ysml64kbXz9+PcxhhTLtnoWmNMsRCRQ4DZwG5czruPga9wza6HF/BcdXCB3ibgYWAOsB2XK28AcAPwjqpeUmQvwBhjyjgL8owxxUZE2gG3AJcAO4CdwLGq+s9+nKsn8DLQNsrmscCVqrq33HrGGPOfYkGeMabQRKQFbnDEElX9cY9tB+IGRiQBKao6pBDXSQB64AK9usAiYI6q/rW/5zTGmPLKgjxjTKGIyFnAW0CCt+pN4GJVDYnIkcDnwGbgHWAkMEBVP4tJYY0x5j/EgjxjTKGIyDJgGXAzcAjwGJAMfIvrh7ccOE5VV4vIDCBdVfvs57X8wNFAe6CWd93ZVpNnjDG5WZBnjNlvIlIZSCVb7ZyIDAGeBwLAX0A/Vd3obbsZGKmqBU51IiLdgZeAjntsUuBdYIiq7t7f12KMMeWNpVAxxuw3Vd2Fy193SLbVAviBTOD4cIDnCXrbCsRLw/IRbvDGOUBnoDmuH+DdwEDgyQK/AGOMKcesJs8YUygi8howGDe4oipwIDAN6A48rKq3e/slePvEqeoh0c+W5zWuBf4PaKeqO6Nsvxg38raK1eYZY4yzP5nnjTEmu2twM04cgxtgcRnwGi433oMi0g/4Gxf0tQIu349rHAD8Fi3A83yDqyFsD/y2H+c3xphyx4I8Y0yheIHX9VE2PSQiO3E1cIfiBkkMA1L24zJLcalT8tLBe1y2H+c2xphyyZprjTHFTkTiVTWzEMcfCPyCG9Bxv6quy7btWOAVYL6q9it0YY0xppywIM8YUyaIyPXAI0A8sBU34KMWUAGYixvksTZ2JTTGmNLFgjxjTJnhzaxxCa7vXU1gMTATeENVg7EsmzHGlDYW5BljjDHGlEOWJ88YU+qJSLyItI6yvqeIXC0iXWNRLmOMKc0syDPGlAUnAAtEpFl4hYjcgMvH9wzwo/e3McYYjwV5xpgiISL1RaSviBwlIlWK+PQ9gH9UdZl3rYbAg8AEXO69Z4C7RaRiEV/XGGPKLAvyjDGFJiIPAauAL4DpwDIRuagIL1EPN8gi7HrcFGlDVHUx8BhQDWhZhNc0/0HinCUiY0Rkkoj8LiJfi8grIjJQRCpEOaZVLMpqzL5YMmRjTKGIyInAzbhExK8Dm4DzgZdF5DdV/aMILrMKOFZEfEAd4ArgFVXd4G3f5T0mFcG1zH+Ul4/xNeAgb1UG7n7uAByLG9m9SETOVNXfvWOOBcbgRnwbU6rY6FpjTKGIyLvAKUANVc3w1iUAS4BPVfV/RXCNA3HTlS3ABXKJwAGqutLbfjQwFWiqqisKez3z3yMitYHZuMqPR4H3gLXqfUmKSAPcj5dhQCWgI3AU8CYwT1UPinZeY2LJmmuNMYXVApgeDvAAvOefkTXdWKGo6hzgTGAtLvFx33CA5zkD+NkCPFMI9wHVcffW46q6RrPVgnh/Pwr08VZ9DbwL/AQcX+KlNSYfrLnWGFNYtYFvoqxfDRTZNGOqOh4Yn8e264rqOuY/60jgI1X9a287qerfIjIOuBQ3D/NV2X/gGFOaWJBnjCkK7UXk3D3WdQCSoqzfqqqT9/dCIiK4gRjVgJWqmrq/5zIGIvdUR1zTa36E+5lagGdKNQvyjDFF4TRviebdPf6eCxQ4yPOmNLsbOAc3X214/RJcH6rnVTVU0PMao6oqIquAZvvc2WkKrLIAz5R2NvDCGFMoItITNxAiv3ao6vcFvEY94GfcyNrPgHlAADgUOByoDzyhqjcW5LzGhInIeKALcLCq7tjLfpWBX4BFqnpySZXPmP1hQZ4xptQTkZuBkUA3Vf07yvZHgZtwI3y3lXT5TNknIsfh8jx+A1yjqvOj7NMGeAroC/RX1S9LtpTGFIwFecaYQhGRCcC3qvp4MV4jBWikqifksb0lsAg4QlV/Lq5ymPJNRG4DRgGKqzleCKwH6uJmVjnC2/U2VX0wJoU0pgCsT54xprA6ArlqPYrYArIS1EbTEMgk56wYxhSIqj4gIpOAO4GeZAV14JIifwTcF06EbExpZ3nyjDFlwUtAVRG5XUSqZd8gIl2Ap4GHVHVTTEpnyg1V/U1VT1PV2ri+pi2ARFWtrapnWIBnyhKryTPGFIW6IpLfjP+7o/V3yk5EBgO3ZVsVHjV7P3CPiGwAdgO1gKpAOvCdiCTYiEdTVFR1F7A01uUwZn9ZkGeMKQqDvSU/5gIH7GOfXbjZLfa0ao+/l2d7fhAQj5tv1Bhj/vNs4IUxplBEZCEu2JqQz0M2qupbxVgkYwpMRObgUqjk1xJVbVlc5TGmKFhNnjGmKMxW1SdiXQhjCikTl0LlX1zC7XhA8th3fUkVypj9ZUGeMaZMEJE44HLgZKAl8KGqjvS2DQQCqvppDItoyrY3gKG4+Zab4GZqeVdVF8W0VMYUgo2uNcaUeiLiAyYBY3BpLRrjBl2EtQDeEJH4GBTPlAOqmqyqrYGuwJe4gG+hiPwkIteLSIPYltCYgrMgzxhTWDeRe37aonY+0As400tt8dUe22cCNYDWxVwOU86p6ixverwmwLHAHFzevJUi8rWIXCYi1WNaSGPyyZprTcx4+c1OBdrgktk2BCrHtFDl0y5gDbAal1R4gqrOKcLzP4ZLEvtrEZ5zT92A6ao6Lo/tu73HKkV1QRFpBQwEOpN1fxbZ+U1EOln351Lgc+AHVQ3t7aDi5l3/W+BbEbkaN5XZecATwBgReVtVL41V+byaxYHAIWTdn9XJuw+h2T+ZwDrc/bkCmAJ8U1ZSNVmQZ0qUl8h2GDAIN01QROXKiVq5UqJ9QBWxXbtTddeu1A7ZVt3r8/mWeiNcHy0jc71uAjrtJQ/eQbipqOYV5iIiUgHXTHcpe6R5qVixgiZVTbL7s4hlpGewfduOttlW3SoiG0TkA+ABVd0zbU4shIA0bwkASRRsJG6REBHB1WpfhfvhE7kf4+PjqVajakkXqdwLZAbYtnV762yZSG4Ske0i8gkwal85P2PNUqiYEuH1lRoqIneqaq3KlRK1d8/jpG+fARzc5XDq1W1AYuXEWBez3Erdlcq69Wv4Y+6vfPnNp3zz3ZeaumuniMhmVb0feHZ/f5l6KVQ+UtURRVvqHNfoCnwPpADPAPfhan/CNSyvAr+o6smFuMb5IjJKVZslJMRrt55HSJ8TjqZ7r640bNyAqtWsEq+4pO1OY/3aDcz5bS7fTP6Ob76Ypps3bRERSfNGbY9W1e0lXS4RORJXe3cWUB/3Y2McrnvCtJKsbRSRvsBDwEE+n4/Duh1MvwF96NH7SBo0qkeNmtVxMaApapkZmaxft5FFCxYz5fNv+fKzb3T1yjUiIkFVfRm4W1Wj5fWMOQvyTLETkTq4D8aeiZUT9crLbpArLrmOypUsqIuVtLTdvPzGszz74qO6M3WnAD8Cp+3PB5UX5CnwVz4PWaaq1+/Hda4FHgUSgCCumS/eW/4B+qjq6v04b2Vc8HhufHycXnTFILnulqHUqGndrmIlGAzywVsf88h9T+r6tRtERBao6smquqC4r+11IzkPOBdoDuwAPgbeA6aoamZxl2GP8vhxwd0wEWHg2QO4+c7radysUUkWw+xh8oSvGH1Xsi7+d6mIyEZVPV1Vp8e6XHuyIM8UKxE5QEQ+VdWmJxx3CqPvfpLaterEuljGs3nLJm6+4xq++HoiIrLa+yItUN86L8hrjGvKyo+/VbV7gQvrrtUUuBjoANQFluAC1NdUNbAf52sMTAQOOvDQAxjzWjJNWzTZn6KZYrB7Vxqj70zm1effQkS2qeo5qvpFcVxLRK7BNdV3xN3Ln+Fq7D5T1fze20VdpqrA+0C/Fq2b67OvJcsBB3WMRVFMFIFAkJRnX2f0nY9pKBQKqupQr2av1CiXQV5KcuKBuFF29YEG2R6r4BJYrsFNmbTGW369bFjqutiUtvwSkcYiMhuoe+PVt3LDVbdac0IppKo8/fzDPPr0KEA3qephqro0v8cXV3OtiJyAG1Dxo6qmF+W5vfMnAT8AB5x1/kBGP3k3FSpWKOrLmCLwwVsfcct1d2kgM5Chqker6k9FfQ1vxov2wDTga9z0eHtLhrxVVZ8v6nJkK48fNwilb+/je/Lsa8nWZaCU+v67H7nyght065ZtApytqh/Eukxh5SLIS0lOjAeOBk4VkdNUtRFAXFwlKlSqqVWq1g9VrdZAEiom+VK3rw9u376G1B0b/BnpWwmFAuCamn7EjRD85LJhqcXeJFDeiUglYDpw6N23Psxlg6+KdZHMPrz57svcdu8NAH8C3VV1Z36OK8Yg7zXgIlytyo+4kY5TgZ8KG/R5efc+Ak65cMi5PPDEXYUrrCl233wxjYvPuhKUdd4PkZVFef7SNq2ZiDwG3Hhsv1688v4Y/H5/cV3KFIF//lrAqcecq7t3p6WpandV/T3WZYIyHuSlJCe2Bm4XkdNVtWpS1UahNh36S5PmR0lS1WbExVXE5/fjYji8R7doKEQwGCAzfTub1v/J4n+/DS1f+oMvkLkLEZnvVbk+c9mw1JhU05d1IjIGGHremRfz8H3PxLo4Jp9uvft63hqbAvCiqv4vP8cUY5DXATf7QHfgSFyTMLjavZm4gG8qLugr0KAREbkReKx7r668PSGFuDj7Ai0LXnzqVe677WGA71S1d1GeW0QSKFju2FBxpdEQkf7A5y3bNNdPv3tfqlS1GryyYPKEr7ji/OsAFqlqh5LuvxlNmQzyUpIT6wB3glxZpWoj3wGHnOtr0KQHSVUb4/MJGtyCBjegoV2gu9HQLjSUimoaIhUQqQBSESQB8SXhi2sMVCKQmcbm9X+xZNFUnT/vM1ENrFbV24A3LxuWGtOcTWWJiLQRkXnNm7XyfTPxZ4mLs0kIyopAIJM+pxzB4iX/BoHOqvrPvo4RkduBP1V1QnGWTUSa4IK9cNB3EG4Qxm5cs+tU4KF9fbCKSDURWZKYlFh9xp9fSK3aNYuz2KaIDT79f3z75TSAk8vjNHZeLfPv4pMDJs0YT6cu7WNdJFMAt91wL2++/C7Ataoa8xqOMhXkpSQnJuJy1NySUKFa5a49r5PmrU/EHxeHBjcRylyNBtbg+siGX1fIPdcQEEQ1CARBg6ABlABoEJ/Uwudvii+hOeKrxq6d6/l99qu6YP4kAeaq6s2XDUudFKOXXqaIyFjg7Ocef5MBJ5wW6+KYApr81QQuv3YQwMeqWmr/B4pIReAMXE67owE/UH1fef9EZDTwf7fcfSPXDL+i+AtqitQ/fy2g75ED0ZD+BXQp7jQmIlKtJHNJisjFwKtnDjqVx198sKQua4rIxg2b6NH5eN29K22TqrZU1R2xLE+ZCfJSkhM7icinPn+F5od0HaLtOp8j8QmVCWUuJZSxADQT1z9W3INCpGmWkAvqIo9BVAOgASDgPc9EgkEIBPD56uFP7IbE1WP7lqX8POvF0PLlM324CayvuGxYapF3Ai8vRKQRsKJT+y4yafz3NtCijDrlnGP47Y+fFWi6r75PIvI2UJB8Dov3d6YA7/46DujjLQ1xKS6+w2WiH7O3UbYiEiciG2rXqVXth7+mSMVKFfenGCbGrrlkOJ988Bm4tDnfFMU5RaSuqq4XkfrASKAHbgBfEq62eCnwDS5Bc4FT9RSgHD/7/f5Dv587RRo1selyy6LkUU/zxOgxAENUNSWWZSkTc9emJCeeJCI/1qzTocmZgz+m8yGXit+3jeCuqYQy5gEKkgC+CuBL8J57j5KARJZ48JbIc+IRiQOJR/1+8PsJBdcS2PwhwS2fU7VKNfocf7/v+OPvwe+vMFjgu5TkxHqxfk9KsVMAOe3kcyzAK8NOP+VccD+X8lOT1wiXTyz7chBwVJT1zSlAQOg1rQ4UkadFZB6wEngZN1vKy0BPoJaX+uWpfKRR6amq1U8c2NcCvDJs4NkDwk9PLYrzichhwDLvB8u/uCTblYAZuDQqM3A1xVcDi0TkwqK4bpRyNAUOO7Ln4RbglWFnnBe5LU+PZTmgDAR5KcmJw4GJLdudULn/6S/5K1VKJLh7OqH031HN8AK5Cl5fu4RInzuRBMTngjskZ8CXM9CLA+K8QC8uEujh9xHKWERg3Rvotmk0adqdU095mkqV6xwuIrNTkhMPivV7U0qdCtC3z4B97WdKseOPPTH8dJ9Bnqr2VtXm2RfcrBRr9lzvLf3yUwYRGYWbYeAjXO3d17i5Omupag9VvUtVZxSwc/OpAP0G9CnAIWVPelo6X0z8mq8mTY11UfL0w7Sf+GLi12zfVvDWrF7HdiepSqKKFE2QhxtR/gFuusUk4FZVba+q/VV1kKr2VdV2wOHAz8BLInJoEV07u4EAfcv5/VneNW/ZlDbtWyEix4lITEfNlOogLyU5cYyIPHJ4j+s5qs+9vjjfLoK7Z6ChnUB8jlq67DV34cDPLdn2CQd3hGv04iK1eOBH8IP4UZ8P9fnB5wcfhHbOJrjuQ6pXa8jAU5/11anTob7ADynJiT1j/R6VNiJyaP16DWnRrNW+dzalVqMGTcL/Dw+LYTEa4mpPluJqUmYBsws5vdUhfr9fux11eNSN27ZuZ+b0WcycPovU1F17PdFvP//BzOmzyMyI+QC6XDZv2sKQ867hmouHxbooebprxAMMOe8alixcWuBjEyokcGTPI0SVZiJS6JEzXkqeGsAiYAIwWkTGecmys+/3C3Ayrkb5tsJeN4rDAI7qfWQxnNqUpKN6H4mqJrDHHNglrdQGeSnJiTeJyNBj+j9Ep4MvFIIrCO7+Edc062rhXE1cgvd3QtSFHDV32Z+7mjsX6PkjzxGvJs/nA5/PC/h8aPpygqvfpKI/RP9+D/ubNetRQUQ+SUlObBHr96q08CaPr9WwfuN972xKvUYNmgBU8ZIGx8JQ4HhgLK7591VghYgsEJHnReRsb8q8gmhYp24t4hOij/j+/Zc/OLv/RZzd/yIeuuvxvZ7oqotv4uz+F7F1S4n1yTfZNGwcac5sWESn7A68q6qn4mp8DwPmiciNXmJiALxBGJ8CxVGT1xCgoTXVlnnZmtuL6v7cL6UyyEtJTuwHPHJ4j+u0WatjCWX8TTD9DxCfC8rI2bdOctXeVchqnvXt2UQbl9VEi3t0i59IbR5ZwR0+n7uuzweBLQRXvoE/cyO9jhrhq1GjZVUR+SwlOdGSGDn1Aalfzz6gyoO6deuHn8bkQ0pV01T1K1X9P1U9HKgDnIlrtj0WF/ytE5E5IvK4iJzspZ/Ik4g0qtugbr46i77x0rvMmzu/sC8j355++HmO7HRciV2vrKtXPxLfF/r+9O6bqsAWAC8dUEdgDG7e2F9EpGu2Q/4FmolIUeeHalg5sZImJlYu4tOakla3CO/Pwih1QV5KcmI7Efmgdfv+dDhwkIQylhHKWOgFX15g5jWxZg2k2HOARbhvXs5aPIkct2ctnt979LmaPMmqyXMBnoB4QZ+mE1rzAXFk0u/Yu/0JFaq1F3gnJTmx1L2XMVADoGaN2rEuhykCNapHWsFqxLIcYaq6WVXHefNDtsUN4Lgc12/vWlwzW54/uESkqqpWrFGz+j6vVbNWDYLBICNvuq+ISr9vE8dPLrFrlQeJVRLDTwt9f3ppWP4hW9Oaqqaq6i3AwcBO4AcReU5EquMGEG0rhmS3NWrWqmEj1sqBbJ8zMf38jIvlxfeUkpyYKCKf1arTvnK33rf5JLSZQPrvkUCMbM2qkb504gK/v+atZOwH0+jcuSWnntyDCglxQAjBS5lCZlbqPFGEEEqIBQs38d4Hv9CqeTUGntyWShV8CD5UfCACPnHPfeJGi4pAYCe6ciyVm17ECcfcKRO/uHmAhgL34obd/5fZh1P5tNf/ryJyErmDq45AZRE5N8ohW1V1vyIabxaM3t7SC1d7DLAc+Ao3BVpe8v1D7MzzB/Lp+EnM+mE249+bwOnnnrI/xWXzpi38NWceoVCILod0Jq8A89svpzFv7nzqN6zHwgWLARARWrXJ3Rtk65ZtzJn9J/64OLoc3Gmv85nGxbuP+EAgyK+zfmf9ug2079iGVm1b7nP0+84dO/nrj3/Yvm0HXQ7uRL0Gdff5eoPBIAvnL2bRv0to0aoZbTu0LvB0XJkZmSxbugKAJk0b5TmfcCgUSf9VVJ87s4GBIjJaVSNTW6rqXyLSC7gYeBg3GCkOl06lqNlnaPkT0/+npSrIA27y+Su06nPSw8T5AgR2zfRWe02pkVq3bDVxxPPVt38y4LSRBAJBAE4feDTvv3M3LqoLABJJnSeEcD/a/Pw8ezHHnfQYaenux9ibY1sy4b2ziPP7cE22rgYvEtxlX9LXElr1EbUbnUn3w4YwY9bzt6QkJ7562bDURSX7lhkTc0/iUppE826UdXOBfAV5ItKerKCuNxBOX7QZ9yX7NfCVqi7Md2nzITMjkztG38LQC29k1MhH6TugD0lJifs+0LNuzXpuuPwWZkz9Mcf6E045jkfHjKJa9aqRdZeefRVTPv8WgLWr13HMISdFtq3YOS/yfFfqbm6+5o5wfjjABYKXDr2Q2+8fHrWfYaXKlfh55myuvng4a1atjaxv3bYlr374HM1bNs11zM6dqdx+w718NHYi2fOoHtr1IJ55NZnGTaO3Pn3w9sfcOXwUO3dkTXlctVoVHnrmXgacdkLUY6IZcfVIxr07gfCcrSXoHlxfvCkicj/wjqqmAqiqekneawKP4L64m4pIG1X9tyQLaUxBlJogLyU5sY6I3HLwEZdopcS6Ekj92iU49iV4zag+wsGe+0HuR4hj2450zrlgVCTAAxj/8Xe89MokrrjsZFBBI3F0yDsuRGZAOGfwS5EAD+DbaYt56PGZ3D78CFxSZV/u4E5cwmUVkO1/oZWa07p1f+b8/Ylvx841o4BoNRfGlGeXAPmPgFzy4n0SkeeB8Py5u4HpeEEd8HtxznSwe/duBpx2Am/3HsuMqT/y2KhnuHP0Lfk6dt2a9fTtNpDNm7Zw/InHcNQxRxIMhvjuqxlMnvAVf/85ny9//Jhwv6sTTj6O9p3b8fTDz1OjZnVG3Hk9QI6atlAoxNknXsSc2X9yZM8jOOm0fuzckcp7r39Iypg3WL9uA2NefyxXWVJ37uLiM4fSqUsHhlw9mLj4OCZP+IqZ02dxSu9zmPHnN+LUTwAAIABJREFUlzlqAjPSMzi519ksXLCYw7sdwvEnHUtiUmVm/TCbTz74jH5HDmTSjHE0bdEkx3Uee+AZHn/gWWrWqsHVwy6nWYumLF28jPdeH8fQC29k7UPrGXL14H2+dy898zrj3p1A2w6tefa15ALXAhaGqi4RkfNwA3xeBJ4XkbXAeqA2rtY4DlgGPI6bacWSLZpSrdQEecCdCRWqV27X+SzRzOUQ2uz62SG4oM4H4efZmm8/GP8j27btzHWyl1ImcsUQl4dQVFEUwY/iB4J8MWUuq1ZvznXcq2//wa03dc36gPUm0XBBn1tUxJVLBNZ9jb/aAXQ/7DLfF1PvPzslOfHRy4al/lIs75AxpZCqTi+mUy8ERuGCuplemosSkZ7m5p2/L3kkx3cbyKvPvcl5F51Jm/b7Tg30xINj2LxpC5cOvZB7HsnKsjHk6sGMuPoOxr4xjmeTX+JmL5g7+8LT+euPf3j64edJrJLIhUNy/078dPxk5sz+k2P69uK1D5/D53Mtz4MuOYsenfvy6fjJXHn9pXQ5pHOO43Zs30HfAX148e0nIwHTRVcM4uIzhzJ1ynReGfMmN9x6VWT/1196l4ULFtPnhKN5+b1niYtzxwy+/Dw6HtCe0Xcm88AdyTz/1hORY9auXseYx14mMbEyn3z7Xo7awXMvOpP+3U8n+b6nGHjWSdSuWyvXawt/1k7/diajbn+EWrVr8toHz5FUpeQHdavqZBFpCVyBS7TdFKiLS5kyHfgFN6tKmog8pWVlyijzn1UqBgukJCe2Bq7s2vMqiY+vSDB9DlnRlVuEbE2nCOIFfmM/mBr1nL/+No+Fi1ZnpUWJ1Aa6YPH98dHjsFWrtzJz1moiQV32qdLwuvVFAj8gkArrZ9CwUTfq1G4fEleVb4wpJFV9VFVHqurUkgzwAAIBN3FG63atuHTohQQCQe4Ydv8+j9u509Wu+f1+rrppSI5tIsL1N1+Jz+fjpadfI5C5r8k5sqSMeQOAa0dcEQnwwHXu7n/q8agqX34evYvY5ddclKNGzO/3R2rVxr45Lud1nn0dgKE3DokEeGGXDr2AatWr8tnHX7BsyfLI+rdeGUt6Wjonn9k/V/Nvi1bNGHjOAHbuTOWNl6O13EPlxEosX7KCqwbfiM/v46V3n6JJ89ilYVLV3ar6pKqeqapHeAm8j/SSIj+mbnJ0LMAzZUGpCPKA26tUa+xr1qovoYwFaCh7rlOJ8pi1LFy4Is+T/jN/OVk1gNmDNeGfBWvyPG7BotScPSVzdZvMXgZg/TR8wVS6HzrEr9A7JTmxe94v1RhTltx429XUrV+H77/7kYnjJ+1132WLVxAIBGnUtGHUgQpNmjemfsO6pO1OY/myvU4JnMOCv12Xw/T0TH6dNSfH4ve7j/FF85dEPbbLwZ1zrTvkiAMBWL1ybSTY3L0rjVUr1iAiHHpE7gl9KlaqSJeDO7nyzMvqehy+7mFdD456/SO6u3Ry8/+K3nUtLj6eIeddy9Yt23jo6Xs5/MjiSD9nzH9TzJtrU5IT40Xk9M4HnuHz+/0EUufkeyxKKCSsWbshz+3LV6zLc9uKFXkft2JV7ubfvRckA9ZNo1bD/iQm1gulpq47HfihYCcpHBER+2VpSquyfH8mJSUyctQIrrvsZu679WH69OtN5cRKUfcN13DVrZt3GqE69eqweuVaFi1YQsvWzfd5/U0bN7NzZyoA5w24JM/91kX5LIyL80cta5WqVahQsQLpaemsXrmGpi2asNwb0Vq9RrXIqNxoZQdY/O8S4BgAli5eBkDtPF5zHW/94jxmtrjvtoeZN3c+1apXpe9Jx+b5+opTWb4/jdmbmAd5wNGqWrVhk25oYA2qu13eu0i+k2iPbskMhHIMuNjTrl1puMEWWce40yi7dufd+rNrd4Ac/9r3/Kev2c4XtvUPpNGJtGvRS37768MzgOF5v+SiIa7tujdwEfAF0UcyGhMT3v3ZCxgMTANej22J9t9p55zMWyljmfXDbJ56+Dn+756biIsyKCAYdJ9He0tPEt4WCuZv3Eia91nl8/m4Yy+DP+o3jJLiZB9pUgCCIVeO/JXdPWb/3A2nMvH5oh8nXvNyXp/VUz77hg6d2zFv7nxG3nQfT79Scj1evPlnBwNrgdEldmFjSkhpCPJOTaraKJRUpZEvlPETLhBTkOyBlDdwQkPuURQlRIUKfmrVqs6mTVujnrhJ4/qgQW8JucUL+po0rsv8BcujH9eoRlY5cgSH2SoZNdsTVcjYBrtW0bRxN/l17gfNU5ITO102LPWvInh/chGRdrgPpgtwHYOX4JLCGhNzItIGuNBbmuNy2A2NZZmKwv2P3UH/Hmfw4tOvcfaFp1M1WxqUsOYtXJ+0Des35nmejd62pi3y1++sYeP6VKxUkbTdaXkOXshLIDPA7l1pVKqccxDoju07SE9LR0Ro5E0P1szrT7dl81YCgWCuPnkAG9Zv8vbNGl3brEUT/v7zHzasi/6aN6xzNYzNWzaJuj35+Qfof8px9Dn8ZD5+/1P6ndynQClXCkpEGuI+OwcDnYBdgE1PacqlmPfJE5HT2rbtI+ITQhlL0XBQpy54cn+Hci7hwC2UQeNG9fI8d9OmtUEz0UhCZPcIIZo2yTuxZ5PGVcgR3Hm1f3hlknDwp0qOWr5tf1O9ZlsqVKiqwMDCvC97EpGaInKViPyIy8x+Gy7AA7ivGDKvl5hgMEh6elqOJRDMf6d0E3siUkNErhSRH4AFwB24AA/gXlXNiFnhikiHzu0YfPl5ZGZkcteIB4g29VTzVk3x+XysXL6KTRtzj95fv3YDa1atw+/306xF1iCFcL+6aIMxRCTSrPvbL38UuNxz//g717rffnbnqVu/DgkVEgBITKxM3fp1UFX+/C3379NAIMjff/wDkKOZuUXrZgD8Pntu1OvP8da3aNUs6va27VuRVCWJR551A1tuu/6ePAPG/SUilUXkfBH5AliBm6qsk7f5GVVdX6QXNKXarB9m8+bL7zFz+qxYF6XYxTTIS0lOPFBVGzVu1l00cwMa3EHugG6PxQvSXNCWQd/ju0U9d+3aNTjs4JbeTBeZoIFIcKgapN/x0TsJV6gQT6/ujV0AF8oK7HIseIFeOPgL2/oXfp+P1k17ILB/KfKzEZF4ETlVRMYBa4Bnga577BYA3i7stWLpuZTHaX1Q7RxLi87V6dy1MccMOJQbbrmcGT9OxbrMlC7e/XmKiHyIuz+fA47cY7cA8GaJF66YDL/jOmrXqcXUKdOjBkJVqlbhjPNOIRAI8toLuf9ZvvbC2wSDQc4YdGqOvnINGrmJO9atWc+2rdtzHXfRFYMAePKh59i9K/ekHsFgkIz06HH06y+8k+NvVY2U7bRzTs6x7ZIrLwDg5Wdfz/Xv7eP3P2Xjhk0c1vVgOh7QPrL+gkvPIT4hngkffpYrONu0cTMfv/8pfr+f8y87J2r5wnr16cGgS85iy+at3HzNHXvdNz/E6S0ir+CaY98C+pL7e6/MdiMoKFVl6pTpTJ0yna1btuW538zps5g6ZXqJzt1ckj4aO5HbbriHD97+ONZFKXaxrslrDZBUtSmasdoLnLKaVMPBHNlr4jSIasAL2gJceH70Cb3PPfsE4vzpkQBPI4/uXOeddXjU5ogB/Q+hepUMIISoayKOGugpXnmzfRCmrYNgGnVrtRZE2u7vmyIih4nIU8Bq4GPgdCAhj92XlIdaEoA4fxy1a9Wldq261KpZm52pO1i4aD7jJrzLeZcM4J4H85eM1hQvETlURJ4EVgGfAGcA0eeeKkf3J7gZHG699yaAyGCIPY2483qSkhJ5YvQYhl81kkkTpjBpwhRGXD2Spx95gcTEygwfeW2OY6pVr0qjJg1QVW697m5+mPYTkyd8FQnozhl8Op26tGfO7D/p3+N0nk1+iYnjJ/He6+O4++bR9Oh8PPP/zj16tUbN6nz+yZcMvfBGJo6bxBcTv2bohTcy5fNvqZxYif9dl3Mgx5CrB9OkeWMmfPg5l51zNR+NncjXk7/jgTseZcRVt+P3+7ltVM7uxk2aN2bI1YPZtnU7A44+i1eee5Pp3/zA6y++w6nHnMemjZsZdMnZUado29Mdo2+hUZMGfDVpKmPfGLfP/aMRkbbejBVLgG9xybrzmvstiMvH+J8QyAxw4WlXcOFpV/D3n/9E3eetlLGc3f8ihl54Y4kmozbFI9Z98hrExVcmzl8RMndAKAR+N+2YeEGdS2AcjMxBG56mTL00yR3aVuOaq87hmTFjIydt3Kget444FzQDJZRVi0emO14D1K6p3HHrBdx1X9aPuJo1krhvZH+ULVk1hyEX6EkoW7AX8oLRSNNtNpnbqFS5JqpaPSU5scJlw1Lzld9LRDrj5kYcSN5TREWz0pv6qTRoXpiD27ftxKTx30f+DoVCLF7yL08+/xAff/o+KW+MofdRx9O75/GFLacpIBHphPuyLMv3Z96TvBbAWRecxluvjI00ee6pQaP6TPphPNdeOoKxb4zLEax0PrADz7yWHKm5y+7OB/+Pay4ZzsTxkyKpWn5bPINKlSvi9/v58Is3uXP4KD54+2MevCvn7BY1alanSpQ5bE8+oz89endj2JW38+lHWTPJ1WtQl1fGPpurf1/FShWZOHUsI64ayZTPv41MtwbQuGlDnnrlEQ7vdkiu69x67zDqNajLA3ckc9eIByLr4+L8jLjjOq4Z8b9cx0STlJTII2NGMejkS13w2vvIPKdRy05EGuP6JZ8KHJivizkrgZb7mse3BMW0ID//+Ct3DL8fEeHJlx+ibYfWsSyOKQKxDvLqV6xYU/0+nwQDO72gKgT+rNo7t/i8QC/ggjtvnjJFkdAOkh+8hMqVKvD2u5Po3Lk1o+8bSt3aIa/WLpRV+0fQ9dHTAGgmtw7rTSgU4NXXv6RVq/rcO/J0WjRJQ0NBF9SFgzkvwMsR6GlWoJdDxnYqVqwZ/qsBsHRvb4CI1AZewE16vT//wI8B5u1zrzLI5/PRulU7nnzoZf6a9wf/LvqHWbN/sCCv5D2AG8Vd7u/Pgw7rwvuTXqd2nbwHN4gIL73zdCQlSPUa1XLt07xlUz755l0Wzl/MnNl/4o/z0/nAjrRq2yLP2pETT+3L1NmfMXP6LLZv30mLVs2oUat6ZHtSlSQee2E0N91+LfP+/Ieli5eTUCGBdh3bcOgRB+WYu7ZW7Zq8P+l1WrRqRv2G9Tiq95F8PXkqW7dso13HNhx46AFR+xSGj33l/TEsW7KcObPnsit1N526tKd9p7ZR58cNvyeXXTWYs84/jTmz/2TxwqW0bN2czgd1pEbN6lGPSX5uFKmpu2jVrmWO9T2POZKJU8eye3fuZuk8XIrrErA/32fNKF3355ZYXXjt6nX87/zrCWQGGD7yOvoO6BOropgiFOsgr0FSlTohUL8Gdrim0VAI9QUjtXdKwJvdwtXgReaN1WyjblnPA/ecwwP3DXG1d6FtoGlue7gfn9dMGw7w0ACh4HpuG34Ut998EhpKRTOXEwrtBlw5XKDnBXshV7MnoZBXzux99LLJ3E5CpUgH44bsI8hT1Y0icgFwNW7E1wEUrBn9J+CJfe5VMpoBDxb1SX0+H506dOHfRf+wZFnOlpX5//7Na2+/wDE9j6dvnwHMmz+XZ19KZsHCeVRJqsq4t77MsX8gGOCDj97itzm/MG/BXOrVacCBnQ/mzIHn06B+oxz7vvDKkyxdvpijj+rDCcfl7GKZnp7GPQ/+H6rK8ceeyLG9+uXYvmPHdh5Idv2Khl93B7VqulxhCxcv4PV3XmDhovls3rqJunXq06JZK07qdxpHHNo9avqKjyaOZeas6cxbMJfGDZtwcJfDufC8IVSqmPtL+snnHmT9hnWMuvNxMjLSeen1Z/j6u8ns3LmDG666lRP7nprPdz2He4DP2b/7cxZuns/SoDKQsrcdqlWvypE9j9jnieo1qBs12XF2Pp+Pth1aF6g2pGmLJrnmhd1T46YN91m7lVAhIcfrqFqtSq7+d/vSrEXTHIND8qNqtSr0PLY7PY/ddz74Padgy+6gw7oU5LKv44K8q4BDgeiRaHQrgJsLcrFi9lQsLpqRnsHlg65jw7qNnDSwH9fdcmW+jvHH+QvcpKuqbN2yjSpVq0TtMhUWCoVI3ZlKlaoFq4BPT0snPiE+x8ww/2UxD/KqJdUTFCSw3TWt+oIQ8rnHSNOsD0FQwlUJ4VG3ihAC/ISC6yG4gZyVDdn6+KkLGN3gi4A3ICOABtYR0pVe4JcJGkSCQS+4czV6uYI9L9CTUJQ8V5nbSKheDRFBVRvl3iE3Vd0NPAo8KiINgPNxw/sPyMfhlVX1vfxcp7iJyEEUQ5AHsGmTl4ahac6WwlVrVvDW2BQyMjOoU6ceZ190ImlpuwFo0SznvitWLuXqYZfw2x8/51j/xdcTeen1Z3j8wRfpc3RW6oY161bx1tgUVq1ZkSvI++Gnabz53ssArF6zMleQ98tvP/LW2BSqV6vBqDtdjDNpygSuvOECQtnum7//+ZOp06fw6lvPM/7trzj8kKyBRLt2pzLstiv5dPJHkXW///ELn07+iLHj3+Slp9+lZfOcAcTESeOZ/+/f3Hj1rQy//Sq+/i6riS6vPGb5kK6q2e/PQbj7Mz/fxFVL0f1ZnX0EeaZMCqrq28CbIlINOBuXO7RHPo6tC7yvrjYg5kQkJj/Yb73hHn7/5Q86HtCex14YnWeuRFXlxade5ZMPP2fe3PnExcXRuUsHLr36Qk4+vX+u/V986lXeefUDHnz6Hg494iAeuONR3n3tQ1JTd3Fsv168Pu4FAB6+90k+/+gLnkp5mFp1anH/bQ8zZdK3pKelk1QlicO6HczdD9+aZ7/OUCjEUw89z+SJU5j/978kVEig84Edueqmy+lzwtFF90aVQbEO8pIqVKjsA0UDu8EXRII+kCAqgoo3Zy3iUucRHsuqXsgXQvEDPkTDc9NC1p7hNCyhSE1eVrAXyNZ0GwACWUFgKIhEmmyjBXvZlj0F0xHx4/MlEAymF7gPkKquISvgOxj3ZToI92EUTRsR8ZWWD6ni8MNP0/hh1jQqVqzE+WdHz/i/cNF8rhl+Kc2btuTyi66hbesOOX5hhkIhLh56FgsWzuOwg7ty2/BRdGjbibXr1zB+wrs8/cIjXHb1OUz55CfatHJdyLof0YuUN8bw6++zUNUcH3zh4Kl2rbrMnDWdjIx0EhKyxh78/OtMAI48oic+n49gMMiw26905Tj/fww66xKaNW3BipXLmDFzKr/98XOOAA/gzlEj+HTyR3Rq34XR9zxJuzYdWbJsEbffcyOzf/+JG265nAljvyWaex78P76b8RVXDbmJnj2OpWKFSrRp1W4/3v2cvPszGUgWkQNxX6aDgLxyGbUWkfiynOLHlB2qug14CXhJRFqRla+xZR6HVMC1QESfE+4/4LUX3ub9N8dTs1YNUt57Js/ZXNLT0rnm0uFMnvAV8QnxtG3fmu3btvPLT7/xy0+/MWf2XEaOGpHjmE0bN7Po3yUsWrCE998cHxnNWqlyRZo2z6qx3rBuA4v+XcLnn3zJ2DfGs2njZpq1aEKlypVYuGAxU6dMZ0DPX5n+5xe5ulLs3JnK/86/nmlff0+FihVo17ENmzdtZdYPs/l55q8MH3kt191S5tN07rdYB3nrd+zcGETV76MSoVA6GgpCSJCQkJoWpHJlL9CTcKAXbqYNIfjBC/I0xxy1nkgy46wgTyPBXiCr6Zbwc68WLxjyluy1ejmDvWAgGP3Ni69CMJBGMJgObtj+flPV34DfRGQE0A/3hXoKOUcyVsSlBZic+wz5JyLNgU2quqMw5ymMdRvW8tgzWZ22t+/YyuKlC5k6fQr16jbgyYdepnGj6Lm2fp0zi84dDmTc219SuVJiru3jPnmHBQvn0bhhU8a+9nkkIGudVIWbb7iLjMwMXnjlSe596FbefNHVnHXzArRt27eyaPECWmcLkr6Z9iUN6jdiwAmn89JrTzPr15kc1a13ZPus2W5Wux7eukVLFrBjx3YqVazMyBGjqFDBJadt16Yj7dp0zFXepcsXM+7jd6hcKZHXXxhHvbouYW2n9l14+Zn36NanA7/98TOTv5qQq5YR4ONP3+flZ96lX5+CNdEVhKrOAW7Kdn8OxnV8z555Nw44CTdKfL+JSDNgq/clbsw+qeoi4G4RuQc4Cnd/ngXs2YnyVArZ5UVE6rlLlq18ez/PnM09t4wmLs7P8289QeNmeTc+vf7iO0ye8BXtOrbhtQ+fj3QZmDzhK4YOvoEXnnyFE0/tG5kXObu3Ut7jn7//5c7Rt3DGoFOpWatGZIaV7J5NfokOndvx2fQPadjYDU5a9O8Szux7IRs3bGLMYy9z5x6zvjz/RArTvv6eAw89gFfGPkvd+m7qvXHvTuDGK/6Pxx54lhNOOf4/O4gk1o3Wa7bvWOf61/kqRYIqCQbBC6JmzV7tpT/JBDK95xleLVwmGsrw/s5wc8hGlnRvfTqqGd4SbpLN9jwS4AWQYCBnGbxgT0LZ/g6FSE8P8vvC3dFfUXwVMtIjfWdXF8WbpKoBVf1MVc8G6gP/I+fcuHcVxWWAFSLytoj0E5ESHzu/YeM6Hn/2gciS8sYYvp32Jcf26serz31A96699nr8iBvujBrgAbw37g0ATj/1vBw1bmGDzx0CwNTpU1i52s2EUrVKNTp1cB9Yv/z+Y2TfhYsXsGLlUg47uCuHH+LSwn0346vI9szMDObM/RWAHl1dU0HNGq5P3u60Xfz595x9vBMwdtwbBIIBjj/2xEiAF1a7Vh26HX4UALNmz4x6fNfDehRrgJedqgZV9XNVPRd3f14OTCcrieRdUvjhiwFgqYi8JyInxuL+NGWTOtNV9XLc/XkeMAk3qg/gFhGJXn2VfztwP8g/FZGzRaTiPo+Ise1bd3DtpSMIBIKcdu4pe+2Lmpq6izGPue4pj465P0ef0BNOOY6TvBlKnn7k+ajHz50zjzseuJnLr72YmrVqAOTZl2/M68mRAA+gVZsWXDDE5Vj849ecCbe3bN5KyjNv4PP5eOLF0ZEAD+CM806h9/E9CQaDPJv8Up6vrbyLdZC3NjV1k19DikhlL5AKIgEXVFXwK1UTE5k0ZWmuAE01K7jTHEu6t2REHtH0PfYNnysz8jwcWEow4NXehZtss4I7CQUJZAZ54qPNHNi6TvRXFF+V9LTINGtFEuRlp6pbVfVFVe0BtAHuBeqJSKHmAVLVZbi5bwfhagVXiMgjIpKffoFFomH9xjxw1xOR5ZYb72bQWZfw0+zv6X96Dy6/dhA7duROFBt2xCF5d/ZeunwxAAd0jJ5doWmTFlSv5j58Fi/JyjcWDixn/5aVGf0br6n2iEN70PWwHogIU6dPiWz/8+/fSUvbTZ3a9SK1f7Vr1YkMejj3kpN45Kn72Lgp7x/9i5e6MoRCIb74emKuJTMzI8fr2tMRh+6743txUNVtqvqyqvbC5cG8G5e6ZL9GfGQ77ypc0u9zgM+AVSISbjI2Jl9UNU1V31PVE4HGuDnG1+N+OBfmvLtwA4xOAsYCa0XkRRE5qrBlLi733/4Iq1asAWDCh5+zZNGyPPf9ZeavbNq4mXoN6kYdFNO9l8vRn1fy5Oo1qnHhZefus0wHHNSR1u1yZ2jq0Nl9jq5cvirH+hlTZ7JzZyot27SIelz3Xi5wzSsn4H9BrJtr12SkbyUYDOALB3nicuB5/9GxVTVm/LSBb75bzrFHN0aIA2+wRaQ/Xo6m2j0GXkRG2GabEi3cbOsNwogEl8EAEgiC95i13i2BjBDXjVnLdWe2J54N0V9RfBV2b14JruYhj52KhqouxNWS3I1L11JYo3DpCBK88w0HhovI78AbwDuquq4IrhNVzRq1uNCrUcvu5hvu4rRBfZj81QTi4+MZ81juBPVJiUkkJUXvArk7bRfrN7iW87p1cucnC6tbpz5bt21h8dKF9Orh0gd079qLF155MkdN3jfTvgDgqG69qVmjFh3bHcBf//zBuvVrqFe3AT97tWs9uuXs8Pv4gy9SsWIlxk94j6eee4jnU57grIHnc80Vw3I1Qy9b7roITZw0jomT8k4Ku3Fj9ECxQb18jfkpVqq6GDcy9x5vvtDCGo2rJUzA9QG8Cddc/AduhOU7qlqoLhLmv8O7V8L9S4viH8wY3EjdOrgm4cuBy0VkMe7z803v30SpsGb1Wh57YTQfvTeB6d/OZPjQ2/nwizejDrpYtmQF4AZe3H3z6Fzbly52AeKaVevISM+ITJUX1qJ1szzT72TXoHH0r7HwCNs9Z3tZttiVKy0tLWq5Fvyz0Ct/9Hnq/wtiHuSFQgEy07dR0V8d9drow7eYAmRsZcigQxl8zacsW7GLS85vDcR5yZG9/ngq3qCL8JFZQzRyjLD1pkkLz5xBuBnWe4wEeEH3GOmTFwySlhZg2PPradeyER3rp0NGlHEOEgfx1UndtQGfyIZLbtpZIvNwqZt/qNC1hqq6UkT+Avac8+0gb3nYm/vxDWCCquY7kVVh1KpZm5uuuZ1rR1zK519+THp6WqRPW1hcXN4fIMFg1v8rvy/vVr5w80G4lgxcbV2cP45Fixewfcc2fD4/P/3yA40aNInU0vXq0Ye//vmDad9/zVmnXcCsX6MHeZUrJfLkQy9z8fn/44VXnmTSlAm8/f4rfDRxLKPvfpLTT8n6pbtrt5tNoc/RJ9Clc+7ks2EN6kePneLiY/1POydVLYr7c1Ue92cX3Jf1wyLyJS7g+6Sk7k9T9nk1xYU9xy4RmYabASa7lrga7btFZAbu8/P9WPcvTX5uFAPPHkDX7odyXNdTIvO5Dr78vFz7Ll3sgqT1azeQMuaNPM8ZCoVITd2VK8irUy+Plq89RMs5uTfh4G3lslV7LdfuXWkEg8H/5Awesf4m+AXQTRvmSuNGhyLrXZDnRtVih8R+AAAgAElEQVSCC9DAp8t5+K6TOeOS95jz13buv7UzSVUSIFKTJ2S1POeuyYuMsM02F254QEX2IC8S1AUCXu2dC/Q2bcnkmmc2khZI4OlTG0Pqguivpmpr8MWxdNWvoZDqjOJ4w0rAfHJ/iYaFO9GfBGwTkfeBN7QEXms4VUgwGGT5yqWREbD5kZSYRJ3a9diwcV2kRi+a8LZmTbKG6SclJtGl8yH8OmcWc/6cTVp6GoFAZo6EzD17HMtzKY8zfea3nHXaBfz6+09AVn+8PR3c5XCef+ItFi35l7seGMF3M75i2O1Dad+2Ex3bH+C93jYsWbaITh0P5KZrbsv3a/0P2Nv96Qf6e8s2EfkAF/B9rzbxsSkZ+0qsfJS3PCUiE3D355fqsvWXqHD/taYtmjB85HXcd9vDjL4jmT4n9KZRk5w1ahW8oO3AQw/g9vuH5zpXdtHy2uW3R67fX7AeZPHxrlzde3Xlhluv2se5/3sBHsQ4yLtsWOqGlOTEHxYtmXFk4yY9ffx/e3ceZ2P5/3H8dZ0zm2XGkn1fBhFqsiaEUFqUkrQrlaS+Ukqpfq3aJFJoV+rbzhdFZQ2lUkSUfR1jH+vMmOXMuX5/nDNjVjM0nHuO9/PxuB9nnHs5n9HpeJ/rvpawaphU3xcq618yzFow1ku1MruYNP4m7n5wMjcPXMrI/2tMg+hIcLl8t3et8b+Tcs6TlxH0sqxa4T3Wxy5jYEXWVrvMgOdJZ/22FAa/dYijqSF89HhbQpKO02m+TFOSUxPYsfsvF741PQvFGDOQ/Nf+PN3ynp4+t6y3I+LxdWT+4vinnLzYON83NmMM1aoef7LYvNStXZ+9+3azZv0/dOtyea79e/ftZv+BeABq18o+20K7Nh1ZtmIJq9euIn6/bwH2Th2OrZnc+vwLiIgoweLfFrJn7y72xe+lRvXa1KxR57g11a/bgElvT+GKPhex8u8/mbfwhywhL5q5C2D12lXHvcZpco0xpm3Bh50WJ/L+vNO/xRtjvsPX2ifB5wJjTP79ME6vvIf/5xaBbz6/PsARY8wi4PFTVlUB+g+6lWlfz+SvZat49D9P8fH/3sm2v150HQASjyQWarLw06VegzoAHD161FF1OUmgW/IApm7dvuRCjyeFkJIN8SZtw1hyLB/mBnuYRpXdTHz9OgYOm85dQ1dzWZcKXHdlZerVLuHry2cg5xQqhoxrHVuH9ljI8+Yb8pKPevh+STJjpx/F7Qrlg8cvoG7EP5CWz3R0xgVlGrN3x1KsNz0d3woBhfUiuYf1Fydn4RuwUeNUXNzr9fL+x+MAX/gpVTLvEbTH0/Oy3ixZupgvp3zMoLseyjUb+hdTPsZaS8PoxkTXyz6fXLu2F/HmO6+ybuMatm7bRIg7hPZtO2fuDw+PoHWLdiz8eS6z5/v+s+e8VZsfl8tFg/qNWPn3n3g8x6aS69H9at9qFT9+x8bN66lft8EJ/85FaCBQOpAF/EsZ78/81yqT4uwyfIN8iqtIfL9DKU5stY4i43a7eXX881zWoTc/zl7E159Oo/eNx8ZK1W/o++K7ZfM29u7eR8XKFQJRZi7RDX13Xdb9s4Ejh4+c8OoYZ4JAj64FmOZJSyJ+79+Y0o0wHo+/X5z/0ePx3T5NS8OmxFO3fCzvvdqTcmVLMWPeAfoPXc/QZzez+LeDkJqGScuyedIg888e/589/p89vp89ab7r+18nbncqE6Yn03tEEqOmphEWFsZbj1xAdMQaSDvOFHKl60JISTZs/dlrYGH/hxJPZA3C2kA5h2zjT6BugOXAo0BdYMgJnptNSmoKGzaty9yWrVjC5Omf0bNv58zBDE888kIBV8nbTdffQd3a9dmybRP3PHBzZoucx5PG/775gtHjfJ12H3/4+VzN+q1i2hIaGsaadX/z16o/aRHTJtcgj47tugAwefpnQO5btTNnTeOt98ewLTb7nKu//fFzZjCMad7q2Gue35Yre1yL1+vlpv49mfPjdyQkJgCQnHyUP/78jSeee5ANm/LpOlC0ehH492VRvD9vPMFzpXgYQeDflxnbQydY+0Z//edYazsBAZs0vHHTRgwc4hv49sywF9m3Jz5zX0yr5pzbohmeNA+vvfBmvtfweHLPfXcqte/cjuiG9UhMTOKNkW/ne9zprstJAt6S1/+hxPUfjCq1dvPWnxpWqXyuMSVqw9EtvtY3rwW3vzXP5fK1wKXHU7O0h7df7M7gp39k34FkVq5N4Z8Ne6hWKYyL25akUd1womuFUiYSsP51bm3uW7YJiV4OHExn/2Eve/ens2hlOks3ePF4ID3dTZnSIYz5z7k0LLkWUgvoI1u+JWmeFLbG/e6yJzjxa6A74GblnxS5IHHAJ/j64/2T5dzy/+a1129cQ+fL8x5kUKpkKR4e/BRdO+VeOqcwQtwhTBjzMfcMvpnvZk/nu9nTqVSxCgcO7ictLRW3283Q/zyRa3kygIiIEsQ0b5k5wXHW/ngZOvhDXkYYzTmn3959uxnx6hOMePUJKpxVkXJlz+LAwf2Z06hc07MvF7Xvmu2cZx9/lYOHDrBo8TxuH3gdxhgiwktwNDkp85gbrut3Un8fJyjBWnuw4MNOvUK+P3dw7P35d5ZzC3urV4qX5GL2/jyAr2vLJGtt3hNdBsgDwwYy838/sHH9Zp548Dne+sQ3R7QxhqdfeYxeF9/IJ+9/wc64XfS6/kpq161JUtJRNq3fwqwZ82jdrgX3Db37tNUbEuLm/14axq3XDGDC6PfZtnk7l19zCbVq1yAhIZENazfx/fTZ9Liqe54DSs4EAQ95ABbeW7N+1sjm5/ShVKWusPFtX588l/X3zfNiXW6My+Xrg+c9QLWINN57qSsfTl7LD4u2AS72HrB8NSsZtzsNl8tF+TJuyke5KVUCIksa3C44eAT2H7EcOAzJqWCtK3NBi/R03/qebjdULh/B/93akEYl10JqAZ8fJapC+XPZuGYaXk9yKvD16fh7O0VyL7/gkwhMwTcybF5RLqNWvWrNzMl9swoNDaNWjTrUrV2fa3r2pWKF3CtnlStbnrat2hNZOqrA1znn7OZ8N2Uxb749kiXLFrN67Spq16pL08bncvP1/WnTMv+lLq/p2TfzFm/3PPr0NW7UlIsvupTEpAQqVqicawLjHt16Er9/H3N+/I5du3ewZdsmKlWoTPu2nejb+zYuv7RXrmtWOKsi/31vGp9P/og5P37P3/+sYPfeXVSuVJUmZzfjsu5X0TDHAJRzm7WgXNnyx50qppjL7/2ZBPwP3/tzTjAv8yeO1jSf59Pw9VueBHxrrU05fSUVXlh4GK+Me47el9zCjKk/8N302fTo6ftS27JNDJOmvM3Qe59g7vcLmPv9glznX3hRm9NdMp27d+Tt/77OY/95mhlTf2DG1B9yHdOzd+7P7DOFI0Ie8Kb1eob8ueLjqu0vGGJMZBM4vApcGX3p3ODygtu/Pq3LBekHKev+iwdurE/vS6OZOHktv6/ch9vtxu1243K5SDjqJinFhTEmc8tgsYSEePF6vdn2GWPo0aYSd15alZJJKyG1EHddq/cgJSWR31Z8ioXX+z+UWOSTIJ8OxphLgDpZnvICP+L7YJpsrU04Fa/b68rr6XXl9Sd1bkzzVnw1qfArupUuVZpHH3zmhF/npj53cFOfO/Ldb4zhw7fyz/aVKlbhwfuGn/BIWWMMN/Tuxw29+xXq+FEjJpzQ9YsTY0w3sr8/LbAA3/vz60AuySdijDkbyNkZdym+EbSfWWv3nf6qsnO5XQwZPgiAWrXz7kLdul0LRr31ArFbtxO/d3+2fZ27d2Tu79/wwzdzWLViNZs2bKFkqRLUb1CXTt060LZ9q2zHt+/cjvCIcBrkMVFxVt0v60LV6lVodt45ee6vXacGQ4YPomTJvBcmueyq7rRu14JZ385j1YrVbN28jdKlS1G/UT0uvuQiWrQ5L9vxF1/aiYqVK9C0eePj1hUMHBHy+j+UmPz+qFLD122a+2HTxr0oW6Ub5sBKrDcNY/0Bz+UCrxvrMpiMoOc5jE1dTo2wsvzfnfX5e0sdJn2zhc1xiZlBL2PLGuSs9bUQulwuvF4v6enphLhdXHBuea5oexbR5Y7AocXgLUT3iMgGENmAVX9+YNPTEg/jG0RRXD3vf1yD7x/OT6y1sQGsRySrZ/2P6zj2/sx/mn4HSE1JZdWK1cRt30FUVCS16takbv38B2AmJCTy17JVpCSn0CzmnFyLsWfYuH4zpUqVpEo1X+v2yuX/sHHdJqrXrEqNWtVJTEyiUuWKRJXJuyP69m07SE5Oplr1qrkWpI+L3cnfK/6hQqUKnNP8bMIjcg/8T0hIZNeO3VSpVpnSpUuRfDSZJYuXsj/+AC3bnp9t2aszyNP4+rlvx7c6S7buLE7gdrt5cPh9BR533U1X57uvbLkyXH/rtRTma3mHzhfQofMFBR7X7fIudLu8S777a9WtWWDdFSqexY23X1eIqqBrj0507dGpUMcWd44IeX4fY+3QX5d90OSSLs+6TKWOmF3zwOvFujNu1Xr9LXm+yY+tMRiXAc8+SI7nnAoVeXlQfVbHelm5/jD/bE5g2+4UjMk75EWWcNGoZgQNa4TTKro05UIPQsI/cCC+gFL9XKFQ/XISEvfw1+rpxsLzJzjgwjGMMR2BX4F7rbW/B7oekayMMRcAfwJDrLW/FnS8E0z5fDrPPz6SvbuzN+DUb1CXMe++lG15KI8nnReeGMn74z/G6z12p/nKa3vwypvPUjoy++Dmi1teSZsLW/LJ1HcZ1O8hvpvuW1av22WdObtpI9545S1uuqMPL43N3WqdkpxC11ZXkpR0lCVr52eGvLjYnQy4eTArlq7MPDaiRARPvfQoN/fP/k/6gjk/cc/ND/DyG8/Sos159LtuINu3+qa/enPiq2dcyDPG1AWSgW4UcXcWkX/DMSGv/0OJ3vdHlXokbsfSmbFbF1GrVlc4uhMOrfaNfDUG63L5wp4xvpY8Y45tGPDsgMRdNClTgSbtzsJ0qo7HhnM4MZ3DiR4SjnoJD4XIEi4iI1yUCvNiUg5CSjwcXQWHT+RupIHa1+ENr8gvC571WuvZCbxxiv56Tjlr7UJgYaDrEMmLv4O6ozqpH8/KP//mwQHDCQkN4aY7+tDsvHM4sP8gy5Ys57ef/6Beg7rZjh9024PMnDaLpuc25tobr8Zay/8+n843k78jLnYH0+Z9nus1dsbt4ulhLzJ/9iKuvKYHNetUp3HTRpx9TkPeeOUtfvhmLi+MeSrXdEEL5vxEYmISrdu1yGwJ3LNrL5e1v5b98Qe4pm9P2lzYku2xO5g44RMeG/w0KSkp9L/31lw1bFi7kbGvTMDldnPLnX2JiAinWUzet9yCmbV2M9Av0HWI5OSYkAfQ/6HE794fVWrc3MWvDeoVVY2ydfrCuglwdJdvVKx/Hjtc/la8rCEvs7+dgbQd+AbYWULcEZQ3oZQPCYUot291i7RUOJoC6Un+Jc9OQtWLoWwzViz/0MbG/e610Kf/Q4mO7EwrIqfXj7MXkZ6ezs39r+f5157Mti/5aDIRJY4ty7dsyQpmTptFk2ZnM23e55lLQt16Z1+6tu6Zuf+yq7pnu87WzbHs++Jbpsz6JFewim5Unw1rN/Lbz3/kmiR2xtRZAPTsfVnmc+Nfe4/98Qd4cPh9mX22ADp3a8813W7m9ZcmcP0t1+RqUZz41ie0bHs+E7+eQOnSJz5/pYicWk6YJy+nB7zpqQtnzn/Gm5yaCPVuhZAcHx7+5ciMf/68nHPjZW5paZjkI5ikeEzCTsyR7ZjEnZAcD56Ekw94ZZtDlS5s2bqAZau+Milp3Hvn0KRicQtJRE69kFDfnLZH8rg7kDXgAZlrbg4YfHu2NT/DI8K59gbfhLSzvp2b6zper5f+g27Js+Ws57W+qYZm+gNdhrTUNGbPnI/b7ebyq33TBSUfTeaTD74gJDSEex7on+34Vhe0ILpRfQ7sP8iSxctyvU56upcXxjylgCfiUI4KecYY951Dk9x/rEi/5Whi/M7ZC0akp7tLQ/07IPQ4U2RYS7Z5UDK2jOeKctnKsk2hdm/279/I/J9Hk5xiJ9z7WNKXQAljTLgx5sxcIE9EMl1y5cVElIhgyufTue/2oZkLvOdl3eoNmT8vW7Ii2+ZJ9y1pumHd5jzP7dI975VVruztC3nfTZ9N1mV7f/rxF44cPkLb9q2oUMk3qCN2axwpySlUqVaZ1avW5qohqoyv9W7j+tw1VK9VjQZnH3/kpIgEjiNu1xpjXPhqCQFC3vo4JeGBu8JvbmpWfzd/0UvmogsfdoU2ug82fwyJARzsWeViqHox++PXMWPeMzY5OW3RE68kv4hv3Vl3xmaM8QAedb4VOTPVi67DB1+MY8iAx5j21Qy+mfwd3S7vzMAH7sw1ncO2Lb7PtMF3Dsv3ent27c3z+ZwLyWeIbliPxk0bsXrVWpYuWU7LNjHAsVu1V157bFLxjAC6fWscV3Xpe0I11Kh5Zg2wECluAh7y/C1fIfjW7MsIeqFj3k3ZOLh/+G3W/vrO4e8fKn1Jl6fcpaLvhm2T4cDy01ukKxRq9YZyzdm6ZT7zFo8hKcnzy9ufpDxw8JDN+Dt042sZzdiMMcZjrT1z11MROYN16NKORSt+4PNJX/PO2In88M1cZn07j0EP3cWwp30rAKanp5Oa6puqacjwQUSVyfuORX63Q0PD8l/q9Mpre7B61VpmTp1FyzYxeDzpzPp2LiEhbi67+lj/vuTkZABq1qnBHQNvyfd6556f+7ZwaGjA/wkRkeMI6P+h+QW8jMfX309ZflmX0D49u2+ZMGXGf+r26PykqVDneohqADt+gLTDp77IqIZQ/TK84RVZsXyiXbbqa7N3v/38udHJLycdtan+ek1+pxtjrFr0RM5MJUpGcPs9N3PLnTfw3w++4NnHXubNV9+hRZsYuvbohNvtpnadmmxcv5nO3ToS06p5wRctpJ69e/DKM2OYM3M+//fiMH796XcO7D9Ip24dKFf+2Apv9aKPjfS9c1DuEbQiUnwFrE+e8U1a5+JYC5g7r23mvLRdr4xPvi1+/+Gfp89+jLVrp5Netjk0GQpVu4Mr90SdRaJEVYi+A+rfTpLXzbwFz3mXrfrau3p9+iuPvXB0dNJRm2/NefxOZwovkK0PkBRfWf47BsuXFA8E5v0ZEuLmtrtv5O77+wHwy6IlmfvqN6oHwPKlfxXpa9auW4vm5zdl88atbNm0jbnfzQeODcrIULd+bVwuFztid+aa08/JXK7M79bB8v4E8OrzMzg45fMz0AMvTI7Nldfjpm3e5EdGHH102/a0ST/9/nb619MHeLfH/Yq3cic452Go1AHCimLtcQOlakPt66DR/aSEVWHpnxPtF1PvZOOWJfFzFnmGjno75Vuy35bNs+asm8m6nlpwiweI3198/qGQ/O0/Nil4IWcHdzZrbYIxJvFA/Kmfrzz5aHKez7tDfN/5ypY9dlu23903AvD26x9wYH/e62Tnd72CZAS6BXN+4qcffyU0LJRLruya7ZiSpUrQ+6arSU9PZ+Rzr+cZgo8mndzrn0pHDmWOXA6K96dffPy+A0p5QWB/fOb/ywF9fzqtQ4XN49ECNi0NzzOvJb97YauQH67strN/QsJLXcqXr+9t36K/q0K1Hpjql/knT/4HDq2GpLjCvaIrDCKjoUwTKHM2hJQiNS2Zjasns2TlF6SmJCWt2Zj+6fufpk49dMQezVpTAVvW3+NMsRvw7tqzI9BfHqQI7N6zM+PHncc7rjix1sbt2rGnAcfpYlEUBt46hNKRpbmoa3tq1q5O8tFklv+xkrdfn0hIaAidunXIPLZDl3Z07dGJOd/9SLc2V3FL/77Uja5NSnIKG9ZtYta383j4qcG55skrjCuu6cGIJ15l+tczWfvPerr26JTnUmfDnnqAmVNn8dmHX7N1UyyXXHkxlapUZP++AyxfupK53/3Iiq2L/9XfSVHbvWtPxo/Fcq3wfOxIPprc9MjhI0RG5b0knRQPu3c64/0ZsJBnrbXGGC++psy8tnR8rWLZnvv5d0/sz797RlzSKfTr7h033Ll///DzS5WqbBvV7Whq1WhL2UqdcVe5GLypvj57mdsRSE+G0EjfdCyhkRDifzQuklMT2LtjGeu3/uzdFve7y5OWnLZpm3fqR1+mfrljt/cwvls9Hn9dOevMWbvN+mjPkPZ3a63H5XLt2x63rVKga5F/b/uObQAHrbVJga6lCO2M3xffMOeExEWtYuUKfPbh10z98ttsz59VoTzPjByea267tz95nZHPjeWdsRN59fmx2faVKlWSipXyXsO2INVrVuX81ueyZPFSIPuo2qwqVanI94snM+Sux1i88DcWL/wt2/7ohvVO6vVPpditmV/kg+ZLCP7fZfu2HTRu2ijQtci/sH1bZrYL6PvTBDJ/+G9j5hx4UZg/Z249u4eec9457ouqVHRdGBZGpbDwKKJrXWgrnVXflCxRjvCIcoSHlyU0vAwudxietARSUw6RknyA5OSDJCbtY3PcMu/O3X+5rDfde+iIXb0tzrv4+/meBWs3pu/DF+I8+WxphfnzmRLyAIwxU4GrZk/7jbMbnnnLGwWLzVs30vHScwG+t9bmnQyKIWPMS8CwD74cT7fLOp/S19qwbhOrV65l7559hIWFUb1mVS7o0Pq44XLv7n38/ddqNqzbjMtlqN+gLq0uaJG5vmyGX3/6HWstrS5oQUjI8bv9btm0jZ1xuwCIadn8uK/v9XrZuG4zq1etZWfcLqLKRNH0vCY0PbcxWXudxO/bz7rVGyhbrkxAwsjRpGSa1WprU5JT1llrzz7tBZwixpgBwFuPPz8018TUUry0b9adbVu2H7XWnmWtPRqoOgIa8iBb0Msa6Nx5PGY9xp3Ho7vzhSH1Wp8bcmG1Kq42JSKo7nJRKvtrucgx0DXd4+HwwcN2zaZt3l9nLUj7bUus9yDHWugyWu6y/pw1xKXjC3IZj7mC4JkU8ACMMbcDHzz8nyf5z8D85/0SZ3v7g9d5fuTjAHdba98NdD1FxRhzAbC4723XMnLc84EuR07SjKk/cM/NDwC8Yq0Nmg8aY0xlYEertue7psz5b6DLkZO09p/1dG3dE2CKtfbaQNYS8JCXIct0KnmFubyCnSvLn7OO0s342ZSJNOGNol1nVa/sqnBWOVMhIsKUPHTY7t8Tb/dv3e7dt25T+kGvN9st1qy3XtOz/Dlr2Msr8GUNex4g7UydNsUYUxHYWa9uA/e8b/7A7T6TBhcHB6/XyyW9LmDNur+9QFVr7Z4CTyomjDEuY8zOqDKRFX9dPdfkXItViod+ve9h7vcLAC601jqrs+C/ZIz5ybjMhT8um0m96DqBLkdOwnPDX+GdsRMBbrHWfhLIWhwT8jLkCHvZWuqybBkhL+t0JZnhjrxHuubFm+Uxaz+6woa9nEFPkx8Dxpj3gP6jRkygzzX5T64qzjR5+mc8MOwugI+ttUE3cZoxZjgw4v6HB/DIUw8Euhw5QX/89ie9Lr4RYIm1tk2g6ylqxphrgMk9enbjnU/HFni8OEtc7E46nnepTUtNi7PWNrDWBnRouuNCXgb/bdz8Ql5eAS+/KU0yAl7OoJdzFGzWoJfXIJD0HD9na9U7027LHo8xpoYxZkOVytXCfpy5zJQsocXLi4uko4l0vryF3bkrLs1a29BauzXQNRU1Y0xJY8yGsPCwKgv+/M7ktzSYOI+1lmu738LvvywF6GStXRDomk4FY8wvQNvJsz6hdbsWgS5HTsB/+j/C/774BqCftfajQNfj2KkurI/HWptsrU0EEvxbon9LyLEdybEdLsR2JMfPWbes1831utbaRH9tZ1y/u4JYa7dba0fv3BVnHhh2lyZHLkYefGwAO3ZuN9ba14Mx4AFYa5OstU+kJKeYO2+476TnoJPT7+WnR2cEvG+CNeD5PQzYQf0essVpguoz3Wcffp0R8FYAHwe4HMDBLXkFybJiRmFa8HK25uWchy/rz3m26CnInRhjTAgwC+h8/z2P8Mjg/wt0SVKAMeNfZNQbIwB+BrpYa1MDXNIpZYyZCPS7oteljPtoFC6XY7/zCjD5s+k8cNcwjDGbrLWtrbXBNAlyLhndClq2ieHzGRMJjzhFqztJkfj9l6Vcf1k/6/GkH7LWtrLWbgh0TVCMQ15h5FhpIq/btb4fgvkvIYCMMWcZY3631ta9pe+dPPv4SEJC8l9QXQLD40nj6ReH8dGn72CM2W6tbRFMgy3yY4wJA+YBF3bu3pE3J76a50TBEnjjRr3Ly8+MBssRa20ba+3qQNd0OhhjPgeuP69lM9799A2qVKsc6JIkD99M/o4H7xluk48me4FLrbVzAl1TBkeFPGNMY6A2sMtau7yAY6/A11/vF2vt7tNRn5w4Y0wdYDrQrHWLdrw6YgJ1a9cPbFGSaeu2zQwZfje/L/0Fg1ltsT2d8g30dDDGlAe+ArrUqV/bvvnBSHNui2aBLkv84vft54khz/Ht/77HGLPTWnuVtfb3QNd1uhhjwoF3gVsqVq5gR00YYTp37xjossQvMTGJ10a8yTtjJ2KMSbTW3mStnRbourJyWsgbB9wLfGWt7VPAsYlASeBya+3M01GfnBxjTGlgEtDL7XbbG/vcbh4Y+CiVKlYJdGlnrH3xe3jj7ZFM+uxd6/F4DDADuNFaezjQtZ1u/q4FrwKDjTFceW0PHv6/wdSpVyvQpZ2xEhISmTj+Y8aNetcmJiYZ4Degl7U2mFa3KDRjzFDgZcDVvlNbHn3mQfRlJHBSU1L54uPJvDZinN23N94YY7ZYa6+01q4KdG05KeTJaeG/dX6dMeZFa209l3HRIqY13btcQcy5rahcqSqVKlZGI3GLXtLRRPbs3c3uPTv5a9Uyfpj7Lb8v+wWv14v/w2k48PmZ3m3BGNMV3z+k5wM0bhGKHQcAABhXSURBVNqIiy+9iHYd21C1RhUqV6mo9URPgdSUVPbs2suuXXv4a9kq5n6/gF8XLbGpqWnGGBNvrX0OmBDsfUQLYow5D9/7sztA3fq16X7FxbTv1Jaq1atQuWolypYrE9gig5AnzcPePfvYvXMPG9ZtZs7M+cyfvdAmJR41/ta7V4FR1tojga41Lwp5cloZY0KBu4CbgAs4xQvFS54svpaRT4B3z/R/PLPyfxm5HrjDGNPZWhuw9b3PcKuAL4HXz8TW5ePxfxm51xhzqbW2RIEnSJHzfzn+Cl+4c3R3saD+APN/YJcBDhW2lcIYU/JkFmT3h5coYP+Z3iJyPNbaNGA8MN4YUwW4EmgAVPNvJQNYXrBKwrdI9g5gHfDtmXrbqyD+/3c/Bz43xpQFLgeacuz96bSmvJJAaWAvWQaT5aMUvrk9nTZnTArH3p9bgJlnUr/QE+Xv1D/HGFMSuARfy3PG+7Ms+uJc1NKA3fjen7HAbK/Xe9wxA04SlCHPGNMXeAhoBoQDKcaYLcD/gLdyzv9ljCkBvIivGfxsY8xB4A/gCWvtkjyu/wHQDuiAb8qWcfjCShjwPPDkqfnNgou1dhe+TsUijmOtPQg4egFRY8yj+D67zvfXe7xjX8f3hVfzGQUBf2PE//ybSJ6CbmIoY8z9wGdAS3zfEP8ADgKNgEeB6jmOr4ZvXrDBQE1gGb7k3g34yRiT17JONf3Xawh8C1yLb/WNFGBjkf9SIiL/gn9wyT3A0BxTS4lIEAu6kAdkfEt9ECjnn5SwCr5Q9mAei1mPBGLwhbVq1tqW1trKwNNAKDDWGFMhn9d6A6gHXAeUsdZG4JBZrkVEsrgU352GEvjuQIjIGSCoQp4xphKQEcg+sNZ6M/ZZa9dZa0fnOP4coC9wCN86c1lHxzwLrMHXp+/+fF4yBrjKWvu1f+k1rLXpRfLLiIgUnVvz+VlEglhQhTx8nY8P+X9+1BhT0O/XE9/fwaKcS+T4O2BnrI3YPJ/zF1prfzrZYkVETjX/AJKeWZ66zt8PWUSCXFANvLDWWmPMQ8B7+PrfXWKMeRX40lrryeOUjKUXahpjxuSx/3z/Y3Q+L+m4iQ9FRHLog28AWoYo4Cp8o4hFJIgFVcgDsNa+b4zZD7yJ73bqf4EXjDHPWWvfz3F4Rsg717/lJ79pExw9P46ICHBLHs/dhkKeSNBzWsjL6BN33FsJxhg3vk7EAIk591tr/2eMmQHcAAwDGgPvGWMuA3pnmccuYxLY/+Jr/ctPfvNKefN5XkQk4Iwx9YD2eezqZoyp4p/GSESClNNC3jr/Y4MCjqvJsdrznDTTP4v/R8aYj4EBwFjgGqAfMNF/2Fp8c+MZa+2PJ121iIgz5dWKB74pn24CRp3GWkTkNHPawIs1/sdoY0x+/eDAN/EwQAK+WajzZa31WmsnAB/6n2qXZfda/2Nbf+ugiEgwyS/kgUbZigQ9p4W8X4Gl+L5lvpHXCDBjTEPgCf8fR2ddQsw/4Wd+MgZeHMjy3KfAfnxz3fXP78QCrisi4jjGmHYc63ecl+bGmOP1RRaRYs5RIc8/r919+NZgvBRYYYx5xBhztTHmBmPMS/hCYCV8axy+lOMSVxlj5hljBhpjmhljKhpjGhhjHgDu8B8zO8vrHeDY5MlvGWPe8r9WK2NMd2PMQ8aYJUDXU/ZLi4icGoVpqVNrnkgQc1wLlbX2V2NMF+ADfH3zXs7jsKnAAP/afVmlA539W05e4Flr7ewcz4/3P76Cr+/egDzOTSlk+SIiAWeMCcc3dUpBbjTGPKJJ3EWCk+NCHoC19kdjTHN8czk1BZrgu826Elia3yAJa+1UY0xboBdQA6iIb4LkTcAn1tp1eZxjgXHGmJnA5cB5QC18t3FXAdOstStznDYJ33q3C//lryoicipcAZQrxHFV8A0+++7UliMigWCydGkTEZFixBjzKPAivnW6D2Z5fhrZV7k4ni+stX1PRX0iEliObMkTEZGTY4ypAPTAN9jsS2AJsBp4Bt/KF4/imzu0Nb5bulcZY8pYaw/lfUURKa4cNfBCRET+td7AJ0Aja+1N1trXrbWz8PUtTrXWzvI/dxNwNr4g2CuA9YrIKaKWPBGR4PKRtfatwhxord0I3JbXdFUiUvypJU9EJIhYa4+ejnNExPkU8kRERESCkEKeiIiISBBSyBMREREJQgp5IiIiIkFIIU9EREQkCCnkiYiIiAQhhTwRERGRIKSQJyIiIhKEFPJEREREgpBCnoiIiEgQUsgTERERCUIKeSIiIiJBSCFPREREJAgp5ImIiIgEIYU8ERGHM8bUNca8bYypVcjjuxtjxp3qukTE2RTyREQczlq7GTgXWG+MGWeM6WmMacixz/AoY0xrY8xtxpifgB+AdYGqV0ScwVhrA12DiIgUwBhzL5CzdS4dcOdxuAeobq3dk+X8H4EIa23bU1akiDiKWvJERIqHz4HUHM/lFfAAvs8a8ETkzKSQJyJSDFhr9wMzCnn4R6eyFhEpHhTyRESKj0mFOOYA8M2pLkREnE8hT0Sk+JgJxBdwzJfW2pTTUYyIOJtCnohIMWGtTQW+KOAw3aoVEUAhT0SkuDneLdv11tpfTlslIuJoCnkiIsWItfY38p8D7+PTWYuIOJtCnohI8ZNXa55FIU9EslDIExEpfj7BF+qyWmit3RKAWkTEoRTyRESKGWvtVmBBjqcLM72KiJxBFPJERIqnrLdmk4GvAlWIiDiTQp6ISPH0FZDm//lba+2RQBYjIs6jkCciUgz5Q90//j9+HshaRMSZFPJERIqv54A9wPRAFyIizmOszTlAS0RETpYx5gLgttP1ckAz4K9CHNsc3xf75SfxOmnAs9bavSdxrogEiEKeiEgRMsb0AyYGuo6i5nK5Vnm93i4KeiLFR0igCxARCUY//PADHTt2DHQZReKDDz7gvvvua+pyueYZYxT0RIoJhTwRkVMgLCyMiIiIQJdRJO69916MMQwaNEhBT6QY0cALEREp0MCBAxk3bhzW2oygVzHQNYnI8SnkiYhIoSjoiRQvCnkiIlJoCnoixYdCnoiInBAFPZHiQSFPREROmIKeiPMp5ImIyElR0BNxNoU8ERGH+vnnn4+7Pz09nREjRpCamnqaKspNQU/EuRTyREQc6pZbbmHkyJF57lu7di0XXnghTzzxBIFeuUhBT8SZFPJERBzqqquu4pFHHuHOO+8kLS0NAGstY8eOJSYmhjVr1vD+++8THh4e4EoV9EScSCFPRMShRo8ezYQJE5g0aRKXXHIJy5cv5+KLL2bw4MF07tyZv//+mzvuuCPQZWbKCHper7cpMCbQ9Yic6RTyREQc7J577mHOnDmsWrWKmJgY/vzzTyZOnMiMGTOoXr16oMvLZeDAgURGRnqBUoGuReRMp7VrRUQcrmPHjvzxxx9cc801hIaGcvPNNwe6JBEpBhTyREQcICUlhcqVK+e5z+12U6pUKUJDQ9m0aRPlypXD7XZn7t+9e7cj+uWJiLMo5ImIOIDL5aJTp04FHtesWbM8zxURyUkhT0TEAUJDQ5k6dWqgyxCRIKKvfyIiIiJBSC15IiIOt27dOr7//nvWr1+Px+MhOjqaTp060aJFi0CXJiIOppAnIuJgjz76KK+99lrmZMhZ9enTh48++oiIiIgAVCYiTqfbtSIiDjV+/HhefvllBgwYwO+//86ePXuIj4/nr7/+4qmnnmLq1Kk8+OCDgS5TRBxKLXkiIg715ptvMmDAAN54441sz5cvX55mzZoRFRXF8OHDef311wkNDQ1QlSLiVGrJExFxoLS0NNauXUuXLl3yPaZDhw6kpKSwYcOG01iZiBQXCnkiIg4UGhpK5cqV2bp1a77HxMbGYoyhWrVqp7EyESkuFPJERBzqkksu4aWXXmLRokW59i1fvpyHH36Y9u3bU6ZMmQBUJyJOpz55IiIO9dprr7Fo0SI6duxIjRo1qFWrFm63m7i4ODZt2kTFihX58MMPA12miDiUWvJERByqXLly/PXXX4wYMYLatWuzZcsW1q9fT7Vq1XjyySdZv3499erVC3SZIuJQaskTEXGwkiVLMnz4cIYPHx7oUkSkmFFLnohIMREfH8/u3bsDXYaIFBMKeSIiDhYXF0e/fv0oX748FSpUoEqVKkRFRdGzZ09WrlwZ6PJExMF0u1ZExKG2bdtGTEwMCQkJdOvWjdq1a2cOvJg9ezbnn38+8+fPp3379oEuVUQcSCFPRMSh7r//fsqVK8fKlStzzYV35MgRLr/8cm6//XbWr18foApFxMl0u1ZExKEWLlzIgw8+mOdkx5GRkTz++ONs2LCBPXv2BKA6EXE6hTwREQfyeDwkJSVRtmzZfI+JiooC4NChQ6erLBEpRhTyREQcKCQkhKZNm/LVV1/le8xXX31FVFQU0dHRp7EyESku1CdPRMShnnzySXr16kXXrl254YYbsq14MXnyZKZNm8bo0aMxxgS6VBFxIIU8ERGHuvrqq3nrrbd4/PHHmTt3brZ9UVFRvPDCCwwePDhA1YmI0ynkiYg42IABA7jhhhtYsmQJmzdvxuv1Eh0dTYsWLY7bX68wdu3axYsvvsiCBQvYuHEjjz76KI8//jgA//3vfzl48CCDBg0qil9DRAJAIU9ExOGioqLo2rVrkV4zYw6+/fv3U6NGDTweDx6PJ3N/YmIigwcPpm/fvpx11llF+toicnpo4IWIiMNZa4mNjWXRokXMnz+fzZs3k56e/q+u+eijjxIWFsayZcuIjY2lUaNG2fa3bt2a9PR0li5d+q9eR0QCRyFPRMTB3nvvPRo0aECtWrXo2LEjXbp0oV69etSsWZNXXnkFr9d7UtddsmQJAwYMICYmJs/9GbeCk5KSTrp2EQks3a4VEXGo8ePHM2jQIOrXr8/QoUOpVasWLpeL2NhYvvnmG4YNG0Z8fDwvv/zyCV87MjKS1atX57s/Y13cJk2anHT9IhJYCnkiIg41cuRIunfvzowZMwgJyf5x/eKLL3L77bczZswYnnnmGSIiIk7o2l26dGHs2LFMnjyZyy+/PNu+f/75h2HDhtG4cWPNwSdSjOl2rYiIA6WmprJlyxZuu+22XAEPwBjDXXfdRWpqKhs2bDjh6z///PPExMTQu3dvKlWqxPr165k4cSLR0dE0bdqU2NhYPvzwQ1wu/TMhUlzp/14REQcKCwujevXq7Nu3L99j9u3bR1hYGLVq1Trh65coUYLFixfzzjvv0LJlSypXrsyRI0coX7489957L+vWraN169b/5lcQkQDT7VoREYd67LHHeOmll+jSpQtNmzbNtm/z5s088cQTDBkyJHMN2xMVEhLCXXfdxV133VUU5YqIwyjkiYg4QGpqKj179sz2XFhYGCkpKTRr1ozo6GiqVq2Ky+Vi9+7drFmzhqioKGrVqoXX69VtVRHJxVhrA12DiEjQMMb0AybOnz+fTp06Ffq8lJQUKleufFKvuXv3bsLDw497zBtvvEFiYuIJX/vuu++mfPnyJ3ROVFSU98iRI99Ya68+4RcUkSKjljwREQcIDw/n4MGDp+z6I0aMYPfu3Sd8Xq9evU445ImIMyjkiYicAebOnUtaWtoJn1enTp2iL0ZETguFPBERB/N4PHz66af89ttvbNmyhSFDhmSuY7to0SLKlSuXa1BGXs4555xTXaqIOIxCnoiIQ+3du5cOHTqwdu3azOf69OmT+fOkSZNYsGAB69atC0R5IuJwCnkiIg517733snPnTiZOnEjnzp1p3rx5tv1dunThvffeIzY2lpo1ax73Wlu2bGHs2LHcfffdnH322Xz66ads27atwBpOZuCFiDiDQp6IiANZa5k9ezZPPfUU/fr1A3yrXGRVrVo1APbs2VNgyJs3bx6jR4+mXLlyPPnkk7zzzjssWLCgwDo08EKk+FLIExFxoPT0dFJSUnC73fkeExsbCxRucMStt95KkyZNOO+88wB47733SEhIKPA8DbwQKb4U8kREHCgkJISYmBjGjRvHHXfcQWRkZLb9iYmJjB49msaNG3PWWWcVeL2uXbvSr18/2rZtC0Dp0qWpVKnSSa+WISLOpynSRUQc6o033iA2NpZKlSrRq1cvkpOTmTJlCrfddhu1a9fmzz//ZMKECYW61pIlS8g6+X3fvn0Lfa6IFE8KeSIiDtWiRQsWL15Mly5dmD59OikpKUyfPp1PPvmEBg0asGjRIi666KJCXSsqKoqFCxeSmpp6iqsWEafQsmYiIkXoZJc1K0haWhqxsbEkJydTr149IiIiTuj8IUOGMGbMGEJDQylXrhwHDhwgPDyc0qVLH/e8n376ifr165/Qa2lZMxFnUJ88ERGH2rt3L5GRkURERBAaGkq9evWy7U9MTCQuLo6GDRsWeK1XX32Vpk2b8uOPPxIXF8fSpUupUKFCgaNyw8LC/tXvICKBo5AnIuJQrVq14uWXX+b666/Pc//69euJiYlh+/btVK9e/bjXcrvd9O/fn/79+wMwdOhQKlasyLBhw4q8bhFxBoU8EZFiateuXQDs27evwJCX08CBAzPPF5HgpJAnIuIgu3bt4umnnwYgPj6e9957j/nz5+c6bs+ePSxcuJDIyMgT7jMHUL9+/ZM6T0SKD4U8EREHKVGiBGFhYXz55ZckJCQwZ84c5syZk+u4kJAQGjVqxMsvv1zg4AkROTMp5ImIOEiZMmUYO3Yso0aNomnTpgwbNoxrrrkm13GlS5cmJEQf4SKSP31CiIg4UGhoKJMmTaJcuXKULVs20OWISDGkkCci4lBt2rQBfHPkrV69mvXr1+PxeIiOjqZJkyaUKFEiwBWKiJMp5ImIONiMGTN44IEH2LBhQ7bnK1euzFNPPcXAgQMDVJmIOJ1CnoiIQy1cuJCrrrqKZs2aMXLkSGrXro3b7SYuLo4pU6Zw7733YozhnnvuCXSpIuJAWtZMRKQIFeWyZm3btiUyMpLvv/8et9uda/+AAQP48ssv2b9/P8aYf/VaRUnLmok4gyvQBYiISG4ej4fly5dz66235hnwAG655RYOHjzI5s2bT3N1IlIcKOSJiDiQ2+0mPDyco0eP5ntMcnIyABEREaerLBEpRhTyREQcyBhDmzZtGD9+PIcPH861PzU1lVdffZU6depQrVq1AFQoIk6ngRciIg41ZswYWrZsSe3atbniiiuoVatW5sCLb7/9lr179zJr1qxAlykiDqWQJyLiUE2aNGHx4sU88sgjfPrpp3i9XsB3K7ddu3ZMmTKFCy+8MMBViohTKeSJiDjYeeedx6xZs0hNTWXr1q14vV7q1q1LWFhYoEsTEYdTyBMRKQbCwsJo0KBBoMsQkWJEAy9EREREgpBa8kREHGLlypV06NDhhM/bvXs34eHhp6AiESnOFPJERByiTJkyXH318ReJCA0NJTIyklmzZvH333+fpspEpDhSyBMRcYhatWrx4YcfHveYNWvWcP/99/P3339Tp04dRo8erVY8EcmT+uSJiBQDCQkJPPLIIzRv3pyffvqJp556in/++afAlj8ROXOpJU9ExOE+/fRTHn74YXbs2EHPnj0ZM2YMdevWDXRZIuJwaskTEXGolStXctFFF3HTTTdRqlQpZs6cybRp0xTwRKRQFPJERBzm0KFDDB48mJiYGJYuXcoLL7zAqlWr6NGjR6BLE5FiRLdrRUQcwlrLRx99xLBhw9izZw99+vRh1KhR1KhRI9CliUgxpJAnIuIQK1as4PbbbwegW7du1KtXj9dee42UlBTS09PzPe/NN98kJEQf5yKSnT4VREQcwu12U6ZMGQCWLFnCkiVLCnXe66+/rpAnIrnoU0FExCGaNWvGwYMHA12GiAQJDbwQERERCUIKeSIiIiJBSCFPREREJAgp5ImIiIgEIYU8ERERkSCkkCciIiIShDSFiojIKbBr1y62bNkS6DICwuv1BroEEUEhT0TklLjhhhsCXUIg6S6RiAMo5ImIFK3lwDOBLsIB1gS6AJEznbHWBroGERERESlialIXERERCUIKeSIiIiJBSCFPREREJAgp5ImIiIgEIYU8ERERkSCkkCciIiIShBTyRERERIKQQp6IiIhIEFLIExEREQlCCnkiIiIiQUghT0RERCQIKeSJiIiIBCGFPBEREZEgpJAnIiIiEoQU8kRERESCkEKeiIiISBBSyBMREREJQgp5IiIiIkFIIU9EREQkCCnkiYiIiAQhhTwRERGRIKSQJyIiIhKEFPJEREREgpBCnoiIiEgQUsgTERERCUIKeSIiIiJBSCFPREREJAgp5ImIiIgEIYU8ERERkSCkkCciIiIShP4fkFDcXD6pQ5gAAAAASUVORK5CYII="
    }
   },
   "cell_type": "markdown",
   "metadata": {
    "id": "R5Xep4W9lq-Z"
   },
   "source": [
    "### Restart the kernel\n",
    "- To use the newly installed packages, we must restart the kernel. \n",
    "- Restarting the kernel after installs so that our environment can access the new packages\n",
    "- To know more about the kernel, please check the image below ([image source](https://ipython.org/ipython-doc/3/development/how_ipython_works.html)).\n",
    "![notebook_components.png](attachment:e882374e-dcbc-46b4-b351-d3c3b2791d00.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FtsU9Bw9h2rL"
   },
   "source": [
    "## Authenticate your notebook environment (Colab only)\n",
    "\n",
    "If you are running this notebook on Google Colab, run the following cell to authenticate your environment. This step is not required if you are using [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "GpYEyLsOh2rL",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "    \n",
    "if \"google.colab\" in sys.modules:\n",
    "    print(\"I am in Colab\")\n",
    "    from google.colab import auth\n",
    "    auth.authenticate_user() # Authenticate user to Google Cloud"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "O1vKZZoEh2rL",
    "tags": []
   },
   "source": [
    "## Define Google Cloud project information\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 3,
     "status": "ok",
     "timestamp": 1717649555830,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "gJqZ76rJh2rM",
    "outputId": "eef6a25d-f66d-403c-b420-82c541f52ec1",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cmd Output: ['ai-gcp-demos']\n",
      "Project ID: ai-gcp-demos\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "\n",
    "PROJECT_ID = \"\"  \n",
    "LOCATION = \"us-central1\" \n",
    "\n",
    "# if not running on Colab, try to get the PROJECT_ID automatically\n",
    "if \"google.colab\" not in sys.modules:\n",
    "    CMD_OUTPUT = !gcloud config get-value project\n",
    "    PROJECT_ID = CMD_OUTPUT[0]\n",
    "\n",
    "print(f\"Cmd Output: {CMD_OUTPUT}\")\n",
    "print(f\"Project ID: {PROJECT_ID}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "D48gUW5-h2rM",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import vertexai\n",
    "\n",
    "vertexai.init(project=PROJECT_ID, location=LOCATION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r-TX_R_xh2rM",
    "tags": []
   },
   "source": [
    "## Load Gemini models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "SvMwSRJJh2rM",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from vertexai.generative_models import GenerativeModel\n",
    "from vertexai.language_models import TextEmbeddingModel\n",
    "from vertexai.vision_models import MultiModalEmbeddingModel\n",
    "\n",
    "# Text model: works with: \n",
    "#   1) text, code\n",
    "model_10p   = GenerativeModel(\"gemini-1.0-pro\") \n",
    "\n",
    "# Multimodal models: works with \n",
    "#   1) text, code, \n",
    "#   2) video(without audio), and \n",
    "#   3) images \n",
    "# with 16k input context\n",
    "model_10pv  = GenerativeModel(\"gemini-1.0-pro-vision-001\") \n",
    "\n",
    "# Multimodal models: works with \n",
    "#   1) text, code, \n",
    "#   2) images, \n",
    "#   3) video(with or without audio) and \n",
    "#   4) audio(mp3) \n",
    "# with 2M or 1M input context \n",
    "# Choose based on performance/cost needs\n",
    "model_15p = GenerativeModel(\"gemini-1.5-pro-001\")   \n",
    "model_15f = GenerativeModel(\"gemini-1.5-flash-001\") \n",
    "\n",
    "# Load text embedding model from pre-trained source\n",
    "model_tEMB = TextEmbeddingModel.from_pretrained(\"text-embedding-004\")\n",
    "\n",
    "# Load multimodal embedding model from pre-trained source\n",
    "# works with \n",
    "#   1) image, \n",
    "#   2) image with caption(~32 words), \n",
    "#   3) video, \n",
    "#   4) video with caption(~32 words)\n",
    "model_mEMB = MultiModalEmbeddingModel.from_pretrained(\"multimodalembedding\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g7bKCQMFT7JT"
   },
   "source": [
    "## Get documents and images from GCS\n",
    "**gcloud storage rsync**\n",
    "makes the contents under the destination the same as the contents under the source, by\n",
    "- copying any missing files/objects (or those whose data has changed), and \n",
    "- (if the -d option is specified) deleting any extra files/objects\n",
    "\n",
    "**Options of the command:**\n",
    "- **--quiet (-q)** Top-level option for the gcloud CLI that disables all interactive prompts when running gcloud CLI commands and is useful for scripting. \n",
    "- **-r** to copy an entire directory tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "KwbL89zcY39N",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "At file://./**, worker process 107033 thread 140185273149248 listed 43...\n",
      "At gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/**, worker process 107033 thread 140185273149248 listed 90...\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/data/med_gemini.pdf to file://./data/med_gemini.pdf\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_11_0_62.jpeg to file://./images/Google Cloud TPU blog.pdf_image_11_0_62.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_11_1_63.jpeg to file://./images/Google Cloud TPU blog.pdf_image_11_1_63.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_12_0_66.jpeg to file://./images/Google Cloud TPU blog.pdf_image_12_0_66.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_13_0_69.jpeg to file://./images/Google Cloud TPU blog.pdf_image_13_0_69.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_14_0_72.jpeg to file://./images/Google Cloud TPU blog.pdf_image_14_0_72.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_15_0_75.jpeg to file://./images/Google Cloud TPU blog.pdf_image_15_0_75.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_1_0_10.jpeg to file://./images/Google Cloud TPU blog.pdf_image_1_0_10.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_3_0_18.jpeg to file://./images/Google Cloud TPU blog.pdf_image_3_0_18.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/Google Cloud TPU blog.pdf_image_6_0_35.jpeg to file://./images/Google Cloud TPU blog.pdf_image_6_0_35.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/gemini_v1_5_report_technical.pdf_image_0_0_1910.jpeg to file://./images/gemini_v1_5_report_technical.pdf_image_0_0_1910.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/gemini_v1_5_report_technical.pdf_image_24_0_488.jpeg to file://./images/gemini_v1_5_report_technical.pdf_image_24_0_488.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/gemini_v1_5_report_technical.pdf_image_44_0_606.jpeg to file://./images/gemini_v1_5_report_technical.pdf_image_44_0_606.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/gemini_v1_5_report_technical.pdf_image_4_0_142.jpeg to file://./images/gemini_v1_5_report_technical.pdf_image_4_0_142.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/gemini_v1_5_report_technical.pdf_image_4_1_143.jpeg to file://./images/gemini_v1_5_report_technical.pdf_image_4_1_143.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/gemini_v1_5_report_technical.pdf_image_5_0_148.jpeg to file://./images/gemini_v1_5_report_technical.pdf_image_5_0_148.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/gemini_v1_5_report_technical.pdf_image_5_1_149.jpeg to file://./images/gemini_v1_5_report_technical.pdf_image_5_1_149.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/gemma_technical_paper.pdf_image_0_0_153.jpeg to file://./images/gemma_technical_paper.pdf_image_0_0_153.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_0_0_46.jpeg to file://./images/med_gemini.pdf_image_0_0_46.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_0_1_48.jpeg to file://./images/med_gemini.pdf_image_0_1_48.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_15_0_464.jpeg to file://./images/med_gemini.pdf_image_15_0_464.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_15_1_465.jpeg to file://./images/med_gemini.pdf_image_15_1_465.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_16_0_480.jpeg to file://./images/med_gemini.pdf_image_16_0_480.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_16_1_481.jpeg to file://./images/med_gemini.pdf_image_16_1_481.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_18_0_573.jpeg to file://./images/med_gemini.pdf_image_18_0_573.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_18_1_575.jpeg to file://./images/med_gemini.pdf_image_18_1_575.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_18_2_576.jpeg to file://./images/med_gemini.pdf_image_18_2_576.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_18_3_578.jpeg to file://./images/med_gemini.pdf_image_18_3_578.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_19_0_613.jpeg to file://./images/med_gemini.pdf_image_19_0_613.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_19_1_615.jpeg to file://./images/med_gemini.pdf_image_19_1_615.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_19_2_617.jpeg to file://./images/med_gemini.pdf_image_19_2_617.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_19_3_618.jpeg to file://./images/med_gemini.pdf_image_19_3_618.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_1_0_73.jpeg to file://./images/med_gemini.pdf_image_1_0_73.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_1_1_75.jpeg to file://./images/med_gemini.pdf_image_1_1_75.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_1_2_77.jpeg to file://./images/med_gemini.pdf_image_1_2_77.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_1_3_79.jpeg to file://./images/med_gemini.pdf_image_1_3_79.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_1_4_81.jpeg to file://./images/med_gemini.pdf_image_1_4_81.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_1_5_83.jpeg to file://./images/med_gemini.pdf_image_1_5_83.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_1_6_84.jpeg to file://./images/med_gemini.pdf_image_1_6_84.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_1_7_86.jpeg to file://./images/med_gemini.pdf_image_1_7_86.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_21_0_681.jpeg to file://./images/med_gemini.pdf_image_21_0_681.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_23_0_726.jpeg to file://./images/med_gemini.pdf_image_23_0_726.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_23_1_727.jpeg to file://./images/med_gemini.pdf_image_23_1_727.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_23_2_728.jpeg to file://./images/med_gemini.pdf_image_23_2_728.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_24_0_752.jpeg to file://./images/med_gemini.pdf_image_24_0_752.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_24_1_754.jpeg to file://./images/med_gemini.pdf_image_24_1_754.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_24_2_780.jpeg to file://./images/med_gemini.pdf_image_24_2_780.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_24_3_782.jpeg to file://./images/med_gemini.pdf_image_24_3_782.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_24_4_783.jpeg to file://./images/med_gemini.pdf_image_24_4_783.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_25_0_812.jpeg to file://./images/med_gemini.pdf_image_25_0_812.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_25_1_813.jpeg to file://./images/med_gemini.pdf_image_25_1_813.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_25_2_815.jpeg to file://./images/med_gemini.pdf_image_25_2_815.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_25_3_817.jpeg to file://./images/med_gemini.pdf_image_25_3_817.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_25_4_819.jpeg to file://./images/med_gemini.pdf_image_25_4_819.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_25_5_820.jpeg to file://./images/med_gemini.pdf_image_25_5_820.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_26_0_849.jpeg to file://./images/med_gemini.pdf_image_26_0_849.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_26_1_851.jpeg to file://./images/med_gemini.pdf_image_26_1_851.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_26_2_853.jpeg to file://./images/med_gemini.pdf_image_26_2_853.jpeg\n",
      "⠛Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_26_3_855.jpeg to file://./images/med_gemini.pdf_image_26_3_855.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_0_890.jpeg to file://./images/med_gemini.pdf_image_27_0_890.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_10_903.jpeg to file://./images/med_gemini.pdf_image_27_10_903.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_11_905.jpeg to file://./images/med_gemini.pdf_image_27_11_905.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_12_907.jpeg to file://./images/med_gemini.pdf_image_27_12_907.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_13_909.jpeg to file://./images/med_gemini.pdf_image_27_13_909.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_14_910.jpeg to file://./images/med_gemini.pdf_image_27_14_910.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_15_911.jpeg to file://./images/med_gemini.pdf_image_27_15_911.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_16_912.jpeg to file://./images/med_gemini.pdf_image_27_16_912.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_17_913.jpeg to file://./images/med_gemini.pdf_image_27_17_913.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_18_914.jpeg to file://./images/med_gemini.pdf_image_27_18_914.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_1_891.jpeg to file://./images/med_gemini.pdf_image_27_1_891.jpeg\n",
      "⠹Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_2_892.jpeg to file://./images/med_gemini.pdf_image_27_2_892.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_3_893.jpeg to file://./images/med_gemini.pdf_image_27_3_893.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_4_894.jpeg to file://./images/med_gemini.pdf_image_27_4_894.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_5_895.jpeg to file://./images/med_gemini.pdf_image_27_5_895.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_6_896.jpeg to file://./images/med_gemini.pdf_image_27_6_896.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_7_897.jpeg to file://./images/med_gemini.pdf_image_27_7_897.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_8_899.jpeg to file://./images/med_gemini.pdf_image_27_8_899.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_27_9_901.jpeg to file://./images/med_gemini.pdf_image_27_9_901.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_55_0_1493.jpeg to file://./images/med_gemini.pdf_image_55_0_1493.jpeg\n",
      "⠼Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_55_1_1494.jpeg to file://./images/med_gemini.pdf_image_55_1_1494.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_55_2_1495.jpeg to file://./images/med_gemini.pdf_image_55_2_1495.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_55_3_1496.jpeg to file://./images/med_gemini.pdf_image_55_3_1496.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_55_4_1497.jpeg to file://./images/med_gemini.pdf_image_55_4_1497.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_55_5_1498.jpeg to file://./images/med_gemini.pdf_image_55_5_1498.jpeg\n",
      "Copying gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2/images/med_gemini.pdf_image_55_6_1499.jpeg to file://./images/med_gemini.pdf_image_55_6_1499.jpeg\n",
      "  Completed files 85/85 | 16.9MiB/16.9MiB                                      \n",
      "\n",
      "Average throughput: 13.5MiB/s\n"
     ]
    }
   ],
   "source": [
    "# download documents and images used in this notebook\n",
    "!gcloud -q storage rsync -r gs://github-repo/rag/intro_multimodal_rag/intro_multimodal_rag_v2 .\n",
    "#!rm -f data/Google\\ Cloud\\ TPU\\ blog.pdf\n",
    "!rm -f data/med_gemini.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V7uv_PVR1T6B",
    "tags": [],
    "toc-hr-collapsed": true
   },
   "source": [
    "# Building metadata of documents containing text and images\n",
    "\n",
    "## The data\n",
    "The source data that we will use in this notebook are:\n",
    "\n",
    "* [Google Cloud TPU Scaling blog](https://storage.googleapis.com/github-repo/rag/intro_multimodal_rag/data/Google%20Cloud%20TPU%20blog.pdf)\n",
    "* [Gemini 1.5 Technical Report](https://storage.googleapis.com/github-repo/rag/intro_multimodal_rag/data/gemini_v1_5_report_technical.pdf)\n",
    "* [Google Gemma Technical Paper](https://storage.googleapis.com/github-repo/rag/intro_multimodal_rag/data/gemma_technical_paper.pdf)\n",
    "* [Med-Gemini Technical Paper](https://storage.googleapis.com/github-repo/rag/intro_multimodal_rag/data/med_gemini.pdf)\n",
    "\n",
    "We can also use other data, by deleting the current files and then placing new files in the `data` folder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5BOAkYN0KlSL"
   },
   "source": [
    "## Extract and store metadata of text and images from a document\n",
    "- The function `get_document_metadata()` extracts text and image metadata from a document, and returns two dataframes, namely *text_metadata* and *image_metadata*. \n",
    "- To know more about how `get_document_metadata()` is implemented using Gemini and the embedding models, please check the [source code](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/retrieval-augmented-generation/utils/intro_multimodal_rag_utils.py).\n",
    "\n",
    "- At the next step, we will use the function to extract and store metadata of text and images from a document. \n",
    "- NOTE: We are loading 4 files with roughly 200 pages and approximately 84 images, the cell below  may take a few minutes to complete. We recommend loading pre-computed metadata instead.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import time\n",
    "from vertexai.generative_models import GenerationConfig, HarmCategory, HarmBlockThreshold\n",
    "from multimodal_qa_with_rag_utils import get_document_metadata\n",
    "from multimodal_qa_with_rag_utils import set_global_variable\n",
    "\n",
    "set_global_variable(\"text_embedding_model\", model_tEMB)\n",
    "set_global_variable(\"multimodal_embedding_model\", model_mEMB) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 350026,
     "status": "ok",
     "timestamp": 1717656088393,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "X8hE0tWD-lf8",
    "outputId": "b094eb22-7c78-41eb-d17c-557631142354",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      " Processing the file: --------------------------------- data/gemini_v1_5_report_technical.pdf \n",
      "\n",
      "\n",
      "Processing page: 1\n",
      "Extracting image from page: 1, saved as: images/gemini_v1_5_report_technical.pdf_image_0_0_1910.jpeg\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 2\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 3\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 4\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 5\n",
      "Extracting image from page: 5, saved as: images/gemini_v1_5_report_technical.pdf_image_4_0_142.jpeg\n",
      "Extracting image from page: 5, saved as: images/gemini_v1_5_report_technical.pdf_image_4_1_143.jpeg\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 6\n",
      "Extracting image from page: 6, saved as: images/gemini_v1_5_report_technical.pdf_image_5_0_148.jpeg\n",
      "Extracting image from page: 6, saved as: images/gemini_v1_5_report_technical.pdf_image_5_1_149.jpeg\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 7\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 8\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 9\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 10\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 11\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 12\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 13\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 14\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 15\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 16\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 17\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 18\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 19\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 20\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 21\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 22\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 23\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 24\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 25\n",
      "Extracting image from page: 25, saved as: images/gemini_v1_5_report_technical.pdf_image_24_0_488.jpeg\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 26\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 27\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 28\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 29\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 30\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 31\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 32\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 33\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 34\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 35\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 36\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 37\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 38\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 39\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 40\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 41\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 42\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 43\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 44\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 45\n",
      "Extracting image from page: 45, saved as: images/gemini_v1_5_report_technical.pdf_image_44_0_606.jpeg\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 46\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 47\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 48\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 49\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 50\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 51\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 52\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 53\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 54\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 55\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 56\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 57\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 58\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "\n",
      " \n",
      " Sleeping for  30  sec before processing the next document to avoid quota issues. You can disable it: \"add_sleep_after_document = False\"  \n",
      "\n",
      "\n",
      " Processing the file: --------------------------------- data/gemma_technical_paper.pdf \n",
      "\n",
      "\n",
      "Processing page: 1\n",
      "Extracting image from page: 1, saved as: images/gemma_technical_paper.pdf_image_0_0_153.jpeg\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 2\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 3\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 4\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 5\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 6\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 7\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 8\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 9\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 10\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 11\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 12\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 13\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 14\n",
      "Sleeping for  12  sec before processing the next page to avoid quota issues. You can disable it: \"add_sleep_after_page = False\"  \n",
      "Processing page: 15\n"
     ]
    }
   ],
   "source": [
    "%%time          \n",
    "                # Cell Magic Command %%time to output the execution time of the entire cell\n",
    "\n",
    "image_description_prompt = \"\"\"\n",
    "- You are a technical image analysis expert. \n",
    "- You will be provided with various types of images extracted from docs.\n",
    "- This docs include research papers, technical blogs, and more.\n",
    "- Your task is to generate concise, accurate descriptions of the images.\n",
    "- Please do not add any info you are not confident about.\n",
    "- Focus on capturing the key details, trends, or relationships depicted in the images.\n",
    "\n",
    "Important Guidelines:\n",
    "* Prioritize accuracy:  If you are uncertain about any detail, state \"Not visible\" instead of guessing.\n",
    "* Avoid hallucinations: Do not add info that is not directly supported by the image.\n",
    "* Be specific:          Use precise language to describe shapes, colors, textures, and any interactions depicted.\n",
    "* Consider context:     If the image is a screenshot or contains text, incorporate that info into description.\n",
    "\"\"\"\n",
    "\n",
    "safety_settings = {\n",
    "    HarmCategory.HARM_CATEGORY_HARASSMENT:        HarmBlockThreshold.BLOCK_ONLY_HIGH,\n",
    "    HarmCategory.HARM_CATEGORY_HATE_SPEECH:       HarmBlockThreshold.BLOCK_ONLY_HIGH,\n",
    "    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_ONLY_HIGH,\n",
    "    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,\n",
    "}\n",
    "\n",
    "\n",
    "# Remove pre-exsisting images folder (here we are running the logic from scratch)\n",
    "!rm -rf images/ \n",
    "\n",
    "# Extract text and image metadata from the PDF docs\n",
    "text_metadata_df, image_metadata_df = get_document_metadata(\n",
    "    model_15p,  \n",
    "    pdf_folder_path           = \"data\",\n",
    "    image_save_dir            = \"images\",\n",
    "    embedding_size            = 1408,  # dimension must be one of 128, 256, 512, 1408.\n",
    "    add_sleep_after_page      = True, \n",
    "    sleep_time_after_page     = 12,    # in seconds, increase value if getting quota issues\n",
    "    add_sleep_after_document  = True, \n",
    "    sleep_time_after_document = 30,    # in seconds, increase value if getting quota issues\n",
    "    image_description_prompt  = image_description_prompt,\n",
    "    safety_settings           = safety_settings,        \n",
    "    generation_config         = GenerationConfig(temperature=0.2, max_output_tokens=2048) #fady\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sWtOx1wb86Az"
   },
   "source": [
    "We can check additional Gemini API parameters and safety setting while building metadata:\n",
    "- [Gemini API parameters](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini)\n",
    "- [Safety settings and thresholds](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/configure-safety-attributes)\n",
    "- We can also pass parameters and safety setting to `get_gemini_response` function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tW3Ci1IL8wSW",
    "tags": [],
    "toc-hr-collapsed": true
   },
   "source": [
    "## Load pre-computed metadata of text and images from source document\n",
    "**If you are facing issues with Quota or want to focus on the outputs, we can load pre-computed metadata.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "w1AGYOYb0In7",
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#To use this cell, uncomment by removing \"\"\" at the start and \"\"\" at the end\n",
    "\"\"\"\n",
    "import pickle\n",
    "\n",
    "# Load the pickle file\n",
    "with open(\"mrag_metadata.pkl\", \"rb\") as f:\n",
    "    data = pickle.load(f)\n",
    "\n",
    "# Extract the DataFrames\n",
    "print(type(data))\n",
    "text_metadata_df = data[\"text_metadata\"]\n",
    "image_metadata_df = data[\"image_metadata\"]\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "miBBoEXwh2rN"
   },
   "source": [
    "### Inspect the processed text metadata\n",
    "\n",
    "\n",
    "The following cell will produce a metadata table which describes the different parts of text metadata, including:\n",
    "\n",
    "- **text**: the page text\n",
    "- **text_embedding_page**: the embedding of the page text\n",
    "- **chunk_text**: the page text divided into smaller chunks\n",
    "- **chunk_number**: the index of each text chunk\n",
    "- **text_embedding_chunk**: the embedding of each text chunk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(text_metadata_df.shape)\n",
    "print(\"*****************************************************************************\")\n",
    "print(text_metadata_df.iloc[0, :]);\n",
    "print(\"*****************************************************************************\")\n",
    "print(text_metadata_df.iloc[1, :]);\n",
    "print(\"*****************************************************************************\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(text_metadata_df.iloc[0, :][\"text\"]);\n",
    "print(\"*****************************************************************************\")\n",
    "print(text_metadata_df.iloc[0, :][\"text_embedding_page\"]);\n",
    "print(\"*****************************************************************************\")\n",
    "print(type(text_metadata_df.iloc[0, :][\"text_embedding_page\"]));\n",
    "print( len(text_metadata_df.iloc[0, :][\"text_embedding_page\"]));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(text_metadata_df.iloc[0, :][\"chunk_text\"]);\n",
    "print(\"*****************************************************************************\")\n",
    "print(text_metadata_df.iloc[0, :][\"text_embedding_chunk\"]);\n",
    "print(\"*****************************************************************************\")\n",
    "print(type(text_metadata_df.iloc[0, :][\"text_embedding_chunk\"]));\n",
    "print( len(text_metadata_df.iloc[0, :][\"text_embedding_chunk\"]));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NjIQYI3mh2rO"
   },
   "source": [
    "### Inspect the processed image metadata\n",
    "The following cell will show the following metadata:\n",
    "- **img_desc**: Gemini-generated textual description of the image\n",
    "- **text_embedding_from_image_description**: text embedding of the generated textual description\n",
    "- **mm_embedding_from_img_only**: image embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(image_metadata_df.shape)\n",
    "print(\"*****************************************************************************\")\n",
    "print(image_metadata_df.iloc[0, :]);\n",
    "print(\"*****************************************************************************\")\n",
    "print(image_metadata_df.iloc[1, :]);\n",
    "print(\"*****************************************************************************\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(image_metadata_df.iloc[0, :].img_desc)\n",
    "print(\"*****************************************************************************\")\n",
    "print(image_metadata_df.iloc[0, :].text_embedding_from_image_description)\n",
    "print(\"*****************************************************************************\")\n",
    "print(type(image_metadata_df.iloc[0, :].text_embedding_from_image_description))\n",
    "print( len(image_metadata_df.iloc[0, :].text_embedding_from_image_description))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(image_metadata_df.iloc[0, :].mm_embedding_from_img_only)\n",
    "print(\"*****************************************************************************\")\n",
    "print(type(image_metadata_df.iloc[0, :].mm_embedding_from_img_only))\n",
    "print( len(image_metadata_df.iloc[0, :].mm_embedding_from_img_only))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iBhoOkutUtPr"
   },
   "source": [
    "## Import the helper functions to implement RAG\n",
    "We will import the following functions to implement RAG:\n",
    "1) **get_similar_text_from_query():** \n",
    "    - Given a text query, finds text from the doc which are relevant. \n",
    "    - It uses text embeddings from the metadata\n",
    "2) **print_text_to_text_citation():** \n",
    "    - Prints the source (citation) and details of the retrieved text.\n",
    "3) **get_similar_image_from_query():** \n",
    "    - Given an image, finds images from the doc which are relevant. \n",
    "    - It uses image embeddings from the metadata.\n",
    "4) **print_text_to_image_citation():** \n",
    "    - Prints the source (citation) and the details of retrieved images.\n",
    "5) **get_gemini_response():** \n",
    "    - Interacts with a Gemini to answer questions based on text and image inputs.\n",
    "6) **display_images():**  \n",
    "    - Displays a series of images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Tngn_vrIKdE1",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from multimodal_qa_with_rag_utils import (\n",
    "    get_similar_text_from_query,\n",
    "    print_text_to_text_citation,\n",
    "    \n",
    "    get_similar_image_from_query,\n",
    "    print_text_to_image_citation,\n",
    "    display_images,\n",
    "    \n",
    "    get_gemini_response,    \n",
    "    get_answer_from_qa_system, \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c9jGEj6DY1Rj"
   },
   "source": [
    "- Before implementing a Multimodal QA Sys with Vertex AI, let's explore what we can achieve with just text or image embeddings. \n",
    "- This will set the foundation for implementing an mRAG Sys. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KHuLlEvSKFWt",
    "tags": []
   },
   "source": [
    "# Text Search\n",
    "\n",
    "## Search similar text with text query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1717656148876,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "eEzP6Yyv7N-G",
    "outputId": "4cb21b58-fcec-4109-9258-43faa1017cb6",
    "tags": []
   },
   "outputs": [],
   "source": [
    "query = \"\"\"\n",
    "What are various med-gemini medical benchmarks that show its performance relative to other models?\n",
    "\"\"\"\n",
    "\n",
    "# Matche user text with \"chunk_embedding\" to find relevant chunks.\n",
    "matching_results_text = get_similar_text_from_query(\n",
    "    query, \n",
    "    text_metadata_df,\n",
    "    column_name=\"text_embedding_chunk\",\n",
    "    top_n=3,\n",
    ")\n",
    "print(type(matching_results_text))\n",
    "print( len(matching_results_text))\n",
    "for key, value in matching_results_text.items():\n",
    "    print(f\"{key}: {value}\")\n",
    "print(\"*****************************************************************************\")\n",
    "print_text_to_text_citation(matching_results_text, print_top=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KY1J2sHr-N8f",
    "tags": []
   },
   "source": [
    "## Get answer with RAG"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ORCistIdDWoE",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from rich.markdown import Markdown as rich_Markdown\n",
    "\n",
    "# All relevant text chunk found across docs based on user query\n",
    "all_chunk_text = [value[\"chunk_text\"] for _, value in matching_results_text.items()]\n",
    "print(type(all_chunk_text))\n",
    "print( len(all_chunk_text))\n",
    "\n",
    "context = \"\\n\".join(all_chunk_text)\n",
    "\n",
    "prompt = f\"\"\"\n",
    "    Answer the question with the given context. \n",
    "    If the specific answer is not in the context, please answer \"I don't know\".\n",
    "    Question: {query}\n",
    "    Context: {context}\n",
    "    Answer:\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 136
    },
    "executionInfo": {
     "elapsed": 1478,
     "status": "ok",
     "timestamp": 1717656150350,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "m8jyc5_SAwlF",
    "outputId": "86047b5b-1176-4480-d61e-9e388259bc5e",
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "gemini_response = get_gemini_response(\n",
    "    model_15p,        # Use Gemini 1.5 Pro\n",
    "    model_input       = prompt,\n",
    "    safety_settings   = safety_settings,\n",
    "    generation_config = GenerationConfig(temperature=1, max_output_tokens=8192),\n",
    ")\n",
    "rich_Markdown(gemini_response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 100
    },
    "executionInfo": {
     "elapsed": 1222,
     "status": "ok",
     "timestamp": 1717656151570,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "0rKLLKt1An97",
    "outputId": "105fdca8-b21f-43e8-a69e-db8e740dc942",
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "gemini_response = get_gemini_response(\n",
    "    model_15f,       # Use Gemini 1.5 Flash\n",
    "    model_input       = prompt,\n",
    "    safety_settings   = safety_settings,\n",
    "    generation_config = GenerationConfig(temperature=1, max_output_tokens=2048),\n",
    ")\n",
    "rich_Markdown(gemini_response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uXm271jdD-Rl"
   },
   "source": [
    "## Search similar images with text query based on image description\n",
    "- Sometimes the Plain text search and RAG don't provide the detailed answer\n",
    "- The info may be visually represented in a table or another image format\n",
    "- The goal here also is to find an image similar to the text query\n",
    "- We may also print the citations to verify"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "executionInfo": {
     "elapsed": 854,
     "status": "ok",
     "timestamp": 1717656152420,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "knj4qQ4xni24",
    "outputId": "ce22e3ec-aca8-4915-cfc9-20d1bcda8485",
    "tags": []
   },
   "outputs": [],
   "source": [
    "matched_images = get_similar_image_from_query(\n",
    "    text_metadata_df,\n",
    "    image_metadata_df,\n",
    "    query              = query,\n",
    "    column_name        = \"text_embedding_from_image_description\", \n",
    "    image_emb          = False,  # Use text embedding instead of image embedding\n",
    "    top_n              = 2,\n",
    ")\n",
    "\n",
    "print_text_to_image_citation(matched_images, print_top=False)\n",
    "print(\"*****************************************************************************\")\n",
    "display_images(\n",
    "    [matched_images[0][\"img_path\"], matched_images[1][\"img_path\"]],\n",
    "    #resize_ratio=0.5,\n",
    ")\n",
    "print(\"*****************************************************************************\")\n",
    "print(type(matched_images))\n",
    "print( len(matched_images))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get answer with RAG"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "executionInfo": {
     "elapsed": 14967,
     "status": "ok",
     "timestamp": 1717656167385,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "HuRD1lZ8RNYP",
    "outputId": "9e3ea7be-b760-4771-b028-6da6b661c11f",
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "txt_prompt = f\"\"\"\n",
    "    Answer the question and explain results with the given Images:\n",
    "    Question: {query}\n",
    "    Images:\n",
    "\"\"\"\n",
    "\n",
    "prompt = [\n",
    "    txt_prompt,\n",
    "    \"Image:\",       matched_images[0][\"image_object\"],\n",
    "    \"Description:\", matched_images[0][\"image_description\"],\n",
    "    \"Image:\",       matched_images[1][\"image_object\"],\n",
    "    \"Description:\", matched_images[1][\"image_description\"],\n",
    "]\n",
    "\n",
    "gemini_response = get_gemini_response(\n",
    "    model_15p,        # Use Gemini 1.5 Pro\n",
    "    model_input       = prompt,\n",
    "    safety_settings   = safety_settings,\n",
    "    generation_config = GenerationConfig(temperature=1, max_output_tokens=8192),\n",
    ")\n",
    "rich_Markdown(gemini_response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oDd9rE4NrRod"
   },
   "source": [
    "# Image Search\n",
    "## Search similar image with image input \n",
    "- For demonstration, we will only find similar images in a single doc\n",
    "- However, we can scale to match (find relevant images) across multiple docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 926
    },
    "executionInfo": {
     "elapsed": 8,
     "status": "ok",
     "timestamp": 1717656167385,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "DJhhS5eZw7QI",
    "outputId": "b8aa4e36-289d-4637-b884-430e33d52b45",
    "tags": []
   },
   "outputs": [],
   "source": [
    "image_query_path = \"images/gemini_v1_5_report_technical.pdf_image_4_0_142.jpeg\"\n",
    "display_images([image_query_path]) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 843
    },
    "executionInfo": {
     "elapsed": 2294,
     "status": "ok",
     "timestamp": 1717656169675,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "nZcU7vZC-8vr",
    "outputId": "1cd88a31-99a6-4e84-e3cb-1553e20be884",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Search for similar images based on input image and Image Embedding\n",
    "\n",
    "matched_images = get_similar_image_from_query(\n",
    "    text_metadata_df,\n",
    "    image_metadata_df,\n",
    "    query              = query,\n",
    "    column_name        = \"mm_embedding_from_img_only\",\n",
    "    image_emb          = True, #Previous: False\n",
    "    image_query_path   = image_query_path, \n",
    "    embedding_size     = 1408, \n",
    "    top_n              = 2,\n",
    ")\n",
    "print_text_to_image_citation(matched_images, print_top=True)   \n",
    "print(\"*****************************************************************************\")\n",
    "display_images([matched_images[0][\"img_path\"]]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lUnsv5Co6pJF"
   },
   "source": [
    "## Another Example\n",
    "In this example, we will do the following:\n",
    "1) Search all the images for a specific query\n",
    "2) Send those images to Gemini 1.5 Pro to ask multiple questions, where it has to compare among those images and provide answers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "E6AHCSwojyX0",
    "tags": []
   },
   "outputs": [],
   "source": [
    "matched_images = get_similar_image_from_query(\n",
    "    text_metadata_df,\n",
    "    image_metadata_df,\n",
    "    query              = \"Show me all the images that can describe LLMs and TPU v5e scaling\",\n",
    "    column_name        = \"text_embedding_from_image_description\", \n",
    "    image_emb          = False,    \n",
    "    top_n              = 2,\n",
    ")\n",
    "print_text_to_image_citation(matched_images, print_top=False)\n",
    "print(\"*****************************************************************************\")\n",
    "display_images([ \n",
    "    matched_images[0][\"img_path\"], \n",
    "    matched_images[1][\"img_path\"]\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "executionInfo": {
     "elapsed": 20161,
     "status": "ok",
     "timestamp": 1717656189834,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "wYkzpB4PTSfm",
    "outputId": "89e33976-ee7f-4f92-9ca7-93cc7204cbed",
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "query = \"\"\"\n",
    " - How does the scaling efficiency of TPU v5e compare to the overall growth in LLM model size over time?\n",
    " - How does the model size impact the observed Per-chip performance and EMFU \n",
    "    for a fixed number of TPU v5e chips (e.g., 256)?\n",
    " - For the INT8 Quant training with 32B parameters, \n",
    "    how does its high EMFU relate to the observed TFLOP/chip/s?\n",
    " - How does the \"per device batch (seq)\" for a 16B model compare to a 128B model, \n",
    "    and how does this affect the \"Total observed Perf\"?\n",
    " - How might the MFU be impacted by increasing LLM model size?\n",
    " \"\"\"\n",
    "\n",
    "txt_prompt = f\"\"\"\n",
    "Task: Answer the following questions in detail, \n",
    "providing clear reasoning and evidence from the images in bullet points.\n",
    "\n",
    "Instructions:\n",
    "1. Analyze the provided images focusing on the relationship between \n",
    "       a) TPU v5e scaling efficiency, \n",
    "       b) LLM model size growth, \n",
    "       c) performance metrics, and \n",
    "       d) quantization effects.\n",
    "2. Answer the following questions in detail, \n",
    "       providing clear reasoning and evidence from the images in bullet points\n",
    "3. Cite the image sources to support your explanations. Mention the file name.\n",
    "\n",
    "Additional Considerations:\n",
    "* Clearly define any technical terms within your answers for better understanding.\n",
    "* Use specific examples and data points from the images to support your explanations.\n",
    "* Feel free to request additional info or clarification if the images are unclear or ambiguous.\n",
    "\n",
    "Question:\n",
    "{query}\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "prompt = [\n",
    "    txt_prompt,\n",
    "    \"Images:\",  matched_images[0][\"image_object\"], matched_images[1][\"image_object\"],\n",
    "]\n",
    "\n",
    "gemini_response = get_gemini_response(\n",
    "    model_15p,        # Use Gemini 1.5 Pro\n",
    "    model_input       = prompt,\n",
    "    safety_settings   = safety_settings,\n",
    "    generation_config = GenerationConfig(temperature=1, max_output_tokens=8192),\n",
    ")\n",
    "rich_Markdown(gemini_response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "efJPPrzRhvIT",
    "tags": []
   },
   "source": [
    "# Building Multimodal QA System with mRAG\n",
    "\n",
    "Let's bring everything together to implement multimodal RAG. We will use all the elements that we have explored in previous sections to implement the multimodal RAG. These are the steps:\n",
    "\n",
    "1) The user gives a query in text format where the expected info is available in the docs.\n",
    "2) Find all text chunks from the pages in the docs.\n",
    "3) Find all similar images from the pages based on the user query.\n",
    "4) Combine all similar text and images found in steps 2 and 3.\n",
    "5) With the help of Gemini, we can pass the user query with text and image context. We can also add a specific instructions the model should remember while answering the user query.\n",
    "6) Gemini produces the answer, and we can print the citations to check all relevant text and images used to address the query."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EI62Hzuw_0_b"
   },
   "source": [
    "## Step 1: User query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XvTKFwOPHLQ_",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# this time we are not passing any images, but just a simple text query.\n",
    "print(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UUqlkKUaYvZA"
   },
   "source": [
    "## Step 2: Get all relevant text chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "r65yBb5gR_NG",
    "tags": []
   },
   "outputs": [],
   "source": [
    "matched_text = get_similar_text_from_query(\n",
    "    query,\n",
    "    text_metadata_df,\n",
    "    column_name      = \"text_embedding_chunk\",\n",
    "    top_n            = 20,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mIgXgVIpYzxj"
   },
   "source": [
    "## Step 3: Get all relevant images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wzu5Gf4yR_J4",
    "tags": []
   },
   "outputs": [],
   "source": [
    "matched_images = get_similar_image_from_query(\n",
    "    text_metadata_df,\n",
    "    image_metadata_df,\n",
    "    query       = query,\n",
    "    column_name = \"text_embedding_from_image_description\",\n",
    "    image_emb   = False,\n",
    "    top_n       = 10,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RhUpWlGAY2uG"
   },
   "source": [
    "## Step 4: Create context_text and context_images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "B_EEuuLCe6Y5",
    "tags": []
   },
   "outputs": [],
   "source": [
    "txt_prompt = \"\"\"\n",
    "Task: Answer the following questions in detail, \n",
    "        providing clear reasoning and evidence from the images and text in bullet points.\n",
    "\n",
    "Instructions:\n",
    "1. **Analyze:** Carefully examine the provided images and text context.\n",
    "2. **Synthesize:** Integrate information from both the visual and textual elements.\n",
    "3. **Reason:**  Deduce logical connections and inferences to address the questions.\n",
    "4. **Respond:** Provide a concise, accurate answers in the following format:\n",
    "   * **Question:** [Question]\n",
    "   * **Answer:** [Direct response to the question]\n",
    "   * **Explanation:** [Bullet-point reasoning steps if applicable]\n",
    "   * **Source** [name of the file, page, image from where the information is citied]\n",
    "\n",
    "5. **Ambiguity:** If the context is insufficient to answer, respond \"Not enough context to answer.\"\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "prompt = [\n",
    "    txt_prompt,\n",
    "    \"Questions: \", query,\n",
    "    \"Image Context: \",\n",
    "]\n",
    "for _, value in matched_images.items():\n",
    "    prompt.extend([\n",
    "        \"Image Path: \",        value[\"img_path\"],\n",
    "        \"Image Description: \", value[\"image_description\"],\n",
    "        \"Image:\",              value[\"image_object\"],\n",
    "    ])\n",
    "\n",
    "text_context = [\"Text Context: \"]\n",
    "for key, value in matched_text.items():\n",
    "    text_context.extend([\n",
    "        \"Text Source: \", f\"\"\"file_name: \"{value[\"file_name\"]}\", Page: \"{value[\"page_num\"]}\" \"\"\",\n",
    "        \"Text:\",      value[\"chunk_text\"],\n",
    "    ])\n",
    "    \n",
    "prompt.extend(text_context)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rHrtodcBAEu9",
    "tags": []
   },
   "source": [
    "## Step 5: Pass context to Gemini"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 881
    },
    "executionInfo": {
     "elapsed": 23441,
     "status": "ok",
     "timestamp": 1717656213273,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "aZuhtJu7fW4n",
    "outputId": "7dd25f28-f32e-4f30-c402-9f3755f31f2f",
    "tags": []
   },
   "outputs": [],
   "source": [
    "gemini_response = get_gemini_response(\n",
    "    model_15p,        # Use Gemini 1.5 Pro\n",
    "    model_input       = prompt, \n",
    "    safety_settings   = safety_settings,\n",
    "    generation_config = GenerationConfig(temperature=1, max_output_tokens=8192),\n",
    ")\n",
    "rich_Markdown(gemini_response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L0FtXYl1fzKh"
   },
   "source": [
    "## Step 6: Print citations and references [Optional]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "IYRLQ47or1I8",
    "tags": []
   },
   "outputs": [],
   "source": [
    "display_images([\n",
    "    matched_images[0][\"img_path\"],\n",
    "    matched_images[1][\"img_path\"],\n",
    "    matched_images[2][\"img_path\"],\n",
    "    matched_images[3][\"img_path\"],\n",
    "    matched_images[5][\"img_path\"],\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "buwd_gp6HJ5K",
    "tags": []
   },
   "outputs": [],
   "source": [
    "print_text_to_image_citation(matched_images, print_top=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "06vYM4MOHJ1-",
    "tags": []
   },
   "outputs": [],
   "source": [
    "print_text_to_text_citation(matched_text, print_top=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U82wS4nIB8IS"
   },
   "source": [
    "# More questions with Multimodal QA System"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "executionInfo": {
     "elapsed": 24067,
     "status": "ok",
     "timestamp": 1717656237338,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "ZAyQVQ54B8X0",
    "outputId": "b79c75b7-0a53-44b0-eb6b-ff565b8b3fca",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# this time we are not passing any images, but just a simple text query.\n",
    "\n",
    "query = \"\"\"\n",
    "Question 1: \n",
    "Imagine a patient presents with new onset prurigo nodularis.\n",
    "Could Med-Gemini-M 1.5 be used to analyze dermatological images \n",
    "  of the patient's lesions in conjunction with a comprehensive history taken\n",
    "  from an EHR dialogue to help a clinician reach a diagnosis and develop a treatment plan?\n",
    "What are the limitations and potential ethical considerations of using the model in this way?\n",
    "\n",
    "Question 2: \n",
    "The paper focuses on uncertainty-guided search for text-based reasoning tasks.\n",
    "How could this approach be extended to multimodal tasks?\n",
    "For instance, if Med-Gemini-M 1.5 encounters uncertainty when analyzing a dermatology image, \n",
    "  could it generate queries to search for \n",
    "  relevant visual examples or supplemental clinical info to refine its interpretation?\n",
    "\n",
    "Question 3:  \n",
    "Considering the potential benefits and risks highlighted in the paper, \n",
    "  what specific steps should be taken during the development, validation, and deployment \n",
    "  of Med-Gemini models to ensure they are used safely, fairly, and effectively \n",
    "  in real-world clinical settings?\n",
    "How can these steps be informed by ongoing collaboration between \n",
    " researchers, clinicians, regulators, and patient communities?\n",
    "\"\"\"\n",
    "\n",
    "(response, matched_text, matched_images) = get_answer_from_qa_system(\n",
    "    query,\n",
    "    text_metadata_df,\n",
    "    image_metadata_df,\n",
    "    top_n_text        = 10,\n",
    "    top_n_image       = 5,\n",
    "    model             = model_15p,\n",
    "    safety_settings   = safety_settings,\n",
    "    generation_config = GenerationConfig(temperature=1, max_output_tokens=8192),\n",
    ")\n",
    "\n",
    "rich_Markdown(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "executionInfo": {
     "elapsed": 18207,
     "status": "ok",
     "timestamp": 1717656255541,
     "user": {
      "displayName": "",
      "userId": ""
     },
     "user_tz": -330
    },
    "id": "dTHdui6jCHzc",
    "outputId": "316d45cd-b4e3-4e20-db47-b052f32eef24",
    "tags": []
   },
   "outputs": [],
   "source": [
    "query = \"\"\"\n",
    "Question 1: \n",
    "How does the mixture-of-experts architecture in Gemini 1.5 Pro contribute to \n",
    "  its ability to handle long context while maintaining performance on core capabilities? \n",
    "Discuss the potential trade-offs involved.\n",
    "\n",
    "Question 2: \n",
    "Gemini 1.5 Pro incorporates various safety mitigations, \n",
    "  including supervised fine-tuning and reinforcement learning.\n",
    "Discuss the effectiveness of these mitigations \n",
    "  in addressing content safety and representational harms in both text-to-text and\n",
    "image-to-text modalities. How can these evaluations be improved?\n",
    "\n",
    "Question 3: \n",
    "Gemini 1.5 Pro demonstrates surprising in-context language learning capabilities for Kalamang,\n",
    "  a low-resource language. \n",
    "What are the implications of this finding for language preservation and revitalization?\n",
    "What challenges need to be addressed for broader applicability of this approach?\n",
    "\"\"\"\n",
    "\n",
    "(response, matched_text, matched_images) = get_answer_from_qa_system(\n",
    "    query,\n",
    "    text_metadata_df,\n",
    "    image_metadata_df,\n",
    "    top_n_text        = 10,\n",
    "    top_n_image       = 5,\n",
    "    model             = model_15p,\n",
    "    safety_settings   = safety_settings,\n",
    "    generation_config = GenerationConfig(temperature=1, max_output_tokens=8192),\n",
    ")\n",
    "\n",
    "rich_Markdown(response)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "building_DIY_multimodal_qa_system_with_mRAG.ipynb",
   "provenance": []
  },
  "environment": {
   "kernel": "python3",
   "name": "tf2-cpu.2-11.m123",
   "type": "gcloud",
   "uri": "us-docker.pkg.dev/deeplearning-platform-release/gcr.io/tf2-cpu.2-11:m123"
  },
  "kernelspec": {
   "display_name": "Python 3 (Local)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  },
  "toc-autonumbering": true,
  "toc-showtags": false
 },
 "nbformat": 4,
 "nbformat_minor": 4
}