examples/dad_jokes/Build Dad Jokes KB.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b704bb97-c14d-44fd-8b85-14696f0f5859",
   "metadata": {},
   "source": [
    "# Dad Jokes Knowledge Base\n",
    "\n",
    "Chosen Dataset:\n",
    "- **Source:** https://www.kaggle.com/datasets/usamabuttar/dad-jokes\n",
    "- **License:** CC0: Public Domain\n",
    "- (version from 2024-07-13)\n",
    "\n",
    "PS: I also considered [this dataset](https://www.kaggle.com/datasets/oktayozturk010/reddit-dad-jokes/data) but it is larger than I wanted (i.e. the resulting file size would be too large for easily sharing)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e1f87ee8-44c4-4b37-9556-dc13c7219533",
   "metadata": {},
   "outputs": [],
   "source": [
    "import svs\n",
    "import pandas as pd\n",
    "\n",
    "# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n",
    "# !!! You must set your OPENAI_API_KEY environment varaible !!!\n",
    "# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n",
    "\n",
    "import os\n",
    "from dotenv import load_dotenv; load_dotenv()\n",
    "assert os.environ.get('OPENAI_API_KEY')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1fb74c5b-0efc-421e-a9b6-da1983d3f891",
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(\n",
    "    level=logging.INFO,\n",
    "    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "302bd601-1171-4e21-89b6-fcb3e981abcc",
   "metadata": {},
   "source": [
    "# Step 1: Load the Knowledge Base\n",
    "\n",
    "We'll read the data and load the KB. As part of that process, the embeddings will be queried from OpanAI and stored in the database.\n",
    "\n",
    "Note: This is _not_ the final step; we'll do more cleaning in later steps..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8416ca5f-77a9-4b0b-bd10-5a4292a7a172",
   "metadata": {},
   "outputs": [],
   "source": [
    "DB_FILE_PATH = './dad_jokes.sqlite'\n",
    "\n",
    "kb = svs.KB(DB_FILE_PATH, svs.make_openai_embeddings_func(), force_fresh_db=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "31177265-81b5-4ee6-a5ff-694d022e488f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10548,\n",
       " ['A steak pun is a rare medium well done.',\n",
       "  'They say that breakfast is the most important meal of the day. Well, not if it’s poisoned. Then the antidote becomes the most important.',\n",
       "  'What do you get if you cross an angry sheep with a moody cow? An animal that’s in a baaaaad mooood.',\n",
       "  'An apple a day keeps the doctor away. At least it does if you throw it hard enough.',\n",
       "  'What sounds like a sneeze and is made of leather? A shoe.',\n",
       "  'A man found a magic lamp with a genie that offered him three wishes. The man said, “For my first wish, I’d like to be rich.” “Okay, Rich,” said the genie. “What would you like for your second wish?”',\n",
       "  'When does a joke become a dad joke? When it becomes apparent.',\n",
       "  'I finally watched that documentary on clocks. It was about time.',\n",
       "  'What did the sink tell the toilet? You look flushed!',\n",
       "  'Why did the nose feel sad? It was always getting picked on.'])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('./dad_jokes.csv')\n",
    "\n",
    "jokes = df['joke'].str.strip()\n",
    "jokes = list(jokes.unique())\n",
    "\n",
    "len(jokes), jokes[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "35658a40-0bf3-4a5f-8f54-d1d3ee677b1f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-17 02:27:09,092 - svs.kb - INFO - starting bulk-add (as new database transaction)\n",
      "2024-07-17 02:27:09,141 - svs.kb - INFO - getting 10548 document embeddings...\n",
      "2024-07-17 02:28:42,125 - svs.kb - INFO - *DONE*: got 10548 document embeddings\n",
      "2024-07-17 02:28:42,125 - svs.kb - INFO - invalidating cached vectors; they'll be re-built next time you `retrieve()`\n",
      "2024-07-17 02:28:42,126 - svs.kb - INFO - ending bulk-add (committing the database transaction)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 14 s, sys: 2.1 s, total: 16.1 s\n",
      "Wall time: 1min 33s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "with kb.bulk_add_docs() as add_doc:\n",
    "    for j in jokes:\n",
    "        add_doc(j)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea01ecd9-ca0e-4b8d-8325-c94fdee3dd05",
   "metadata": {},
   "source": [
    "# Step 2: Remove \"not safe for work\" jokes\n",
    "\n",
    "IMO dad jokes should be funny for _any_ audience (e.g. young kids). Also, I'd like to be able to present this work in professional contexts without anyone blushing. So we'll use semantic similarity to find-and-remove jokes that are, as they say, \"not safe for work\".\n",
    "\n",
    "Also, the final file size (from Step 1) is a bit too big anyway (I want to be able to easily share the resulting file), so removing some jokes will help that too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "bfe517e9-0f02-470f-ab29-6e585987a8db",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-17 02:28:42,186 - svs.kb - INFO - retrieving 10548 documents with query string: not safe for work (NSFW), racist, misogynistic, discriminatory, explicit, profanity\n",
      "2024-07-17 02:28:42,188 - svs.kb - INFO - re-building cached vectors...\n",
      "2024-07-17 02:28:43,230 - svs.kb - INFO - re-building cached vectors... DONE!\n",
      "2024-07-17 02:28:43,514 - svs.kb - INFO - got embedding for query!\n",
      "2024-07-17 02:28:43,550 - svs.kb - INFO - computed 10548 cosine similarities\n",
      "2024-07-17 02:28:43,628 - svs.kb - INFO - retrieved top 10548 documents\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.32 s, sys: 56.8 ms, total: 1.37 s\n",
      "Wall time: 1.44 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "query = 'not safe for work (NSFW), racist, misogynistic, discriminatory, explicit, profanity'\n",
    "\n",
    "docs = kb.retrieve(query, n=len(kb))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1dd85667-4790-41c5-82bc-e51f6ad3b3b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: >"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGdCAYAAAAMm0nCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAqiElEQVR4nO3df3AU933/8dfpx50sggCBpZMmQmD8NZgfAgxB1tRgCCAhNCSuaVMDNsSl4DDCmaDExYoxCEgtRfDFOC6Nxxlj0gYKdsclKTBYBw6GGgFGtooBlzEUh6TWiSYYDtD4OKT9/pGvLr5KAp24095H93zMaKT97Gd337ufW/Rid0/nsCzLEgAAgEES7C4AAAAgXAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxkuwuIFpaWlr02WefqXfv3nI4HHaXAwAAOsGyLF29elXZ2dlKSOj4OkuPDTCfffaZcnJy7C4DAAB0wW9/+1t99atf7XB+jw0wvXv3lvTHA5CWlmZzNfEjEAiopqZGhYWFSk5OtrsciDGJVYxLbGJc7Ofz+ZSTkxP8Pd6RHhtgWm8bpaWlEWC6USAQUGpqqtLS0jj5YwRjEpsYl9jEuMSO2z3+wUO8AADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMZJsrsAAJ0z6NndXVrOlWipeoI0suJt+Zvb/3j6T6tK7qQ0AOh2XIEBAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADBO2AHm4MGDmjVrlrKzs+VwOLRz586Q+Q6Ho92vdevWBfsMGjSozfyqqqqQ9Zw4cUITJ05USkqKcnJyVF1d3bU9BAAAPU7YAeb69esaPXq0Nm3a1O78hoaGkK/NmzfL4XBo9uzZIf3WrFkT0u/pp58OzvP5fCosLFRubq7q6uq0bt06VVRU6NVXXw23XAAA0AOF/YfsiouLVVxc3OF8t9sdMv3LX/5SU6ZM0T333BPS3rt37zZ9W23dulU3btzQ5s2b5XQ6NWLECNXX12vDhg1avHhxuCUDAIAeJqrPwDQ2Nmr37t1auHBhm3lVVVXq37+/xo4dq3Xr1unmzZvBebW1tZo0aZKcTmewraioSGfOnNHnn38ezZIBAIABovpRAj//+c/Vu3dvPfrooyHt3/3ud/XAAw8oPT1dhw8fVnl5uRoaGrRhwwZJktfr1eDBg0OWyczMDM7r169fm235/X75/f7gtM/nkyQFAgEFAoGI7hc61nqsOeaR50q0urZcghXyvT2MV/fjXIlNjIv9OnvsoxpgNm/erHnz5iklJSWkvaysLPhzXl6enE6nnnrqKVVWVsrlcnVpW5WVlVq9enWb9pqaGqWmpnZpneg6j8djdwk9TvWEO1t+7fiWDuft2bPnzlaOLuNciU2Mi32ampo61S9qAebQoUM6c+aMduzYcdu++fn5unnzpj799FMNHTpUbrdbjY2NIX1apzt6bqa8vDwkGPl8PuXk5KiwsFBpaWl3sCcIRyAQkMfj0fTp05WcnGx3OT3KyIq3u7ScK8HS2vEtev54gvwt7X+Y48mKojspDV3AuRKbGBf7td5BuZ2oBZjXXntN48aN0+jRo2/bt76+XgkJCcrIyJAkFRQU6LnnnlMgEAi+gDwej4YOHdru7SNJcrlc7V69SU5O5kVoA4575HX0SdKdXr7F0eE6/s/zNXe07o7wKde3x7kSmxgX+3T2uIf9EO+1a9dUX1+v+vp6SdL58+dVX1+vCxcuBPv4fD69+eab+pu/+Zs2y9fW1mrjxo36j//4D/3Xf/2Xtm7dqmXLlunxxx8PhpO5c+fK6XRq4cKFOnXqlHbs2KGXXnop5AoLAACIX2FfgTl+/LimTJkSnG4NFQsWLNCWLVskSdu3b5dlWZozZ06b5V0ul7Zv366Kigr5/X4NHjxYy5YtCwknffr0UU1NjUpLSzVu3DgNGDBAK1eu5C3UAABAUhcCzOTJk2VZt343xOLFizsMGw888ICOHDly2+3k5eXp0KFD4ZYHAADiAJ+FBAAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcZLsLgDoaQY9u9vuEgCgx+MKDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwTtgB5uDBg5o1a5ays7PlcDi0c+fOkPnf/va35XA4Qr5mzJgR0ufSpUuaN2+e0tLS1LdvXy1cuFDXrl0L6XPixAlNnDhRKSkpysnJUXV1dfh7BwAAeqSwA8z169c1evRobdq0qcM+M2bMUENDQ/Drn//5n0Pmz5s3T6dOnZLH49GuXbt08OBBLV68ODjf5/OpsLBQubm5qqur07p161RRUaFXX3013HIBAEAPFPZnIRUXF6u4uPiWfVwul9xud7vzPv74Y+3du1fvv/++xo8fL0l6+eWXNXPmTK1fv17Z2dnaunWrbty4oc2bN8vpdGrEiBGqr6/Xhg0bQoIOAACIT1H5MMcDBw4oIyND/fr109e//nX96Ec/Uv/+/SVJtbW16tu3bzC8SNK0adOUkJCgo0eP6s///M9VW1urSZMmyel0BvsUFRXpxz/+sT7//HP169evzTb9fr/8fn9w2ufzSZICgYACgUA0dhPtaD3W8XzMXYmW3SWEcCVYId+7Uzy/Dm6HcyU2MS726+yxj3iAmTFjhh599FENHjxY586d0w9/+EMVFxertrZWiYmJ8nq9ysjICC0iKUnp6enyer2SJK/Xq8GDB4f0yczMDM5rL8BUVlZq9erVbdpramqUmpoaqd1DJ3k8HrtLsE31BLsraN/a8S3dvs09e/Z0+zZNE8/nSixjXOzT1NTUqX4RDzCPPfZY8OdRo0YpLy9PQ4YM0YEDBzR16tRIby6ovLxcZWVlwWmfz6ecnBwVFhYqLS0tattFqEAgII/Ho+nTpys5OdnucmwxsuJtu0sI4UqwtHZ8i54/niB/i6Nbt32yoqhbt2cSzpXYxLjYr/UOyu1E5RbSl91zzz0aMGCAzp49q6lTp8rtduvixYshfW7evKlLly4Fn5txu91qbGwM6dM63dGzNS6XSy6Xq017cnIyL0IbxPNx9zd3b0joLH+Lo9tri9fXQDji+VyJZYyLfTp73KP+d2B+97vf6Q9/+IOysrIkSQUFBbp8+bLq6uqCfd555x21tLQoPz8/2OfgwYMh98E8Ho+GDh3a7u0jAAAQX8IOMNeuXVN9fb3q6+slSefPn1d9fb0uXLiga9eu6ZlnntGRI0f06aefav/+/frmN7+pe++9V0VFf7yUfP/992vGjBlatGiRjh07pvfee09Lly7VY489puzsbEnS3Llz5XQ6tXDhQp06dUo7duzQSy+9FHKLCAAAxK+wA8zx48c1duxYjR07VpJUVlamsWPHauXKlUpMTNSJEyf0jW98Q/fdd58WLlyocePG6dChQyG3d7Zu3aphw4Zp6tSpmjlzph566KGQv/HSp08f1dTU6Pz58xo3bpy+//3va+XKlbyFGgAASOrCMzCTJ0+WZXX8dsy33779A4zp6enatm3bLfvk5eXp0KFD4ZYHAADiAJ+FBAAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYJwkuwsA0HMNenZ31Nb9aVVJ1NYNIPZxBQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgnLADzMGDBzVr1ixlZ2fL4XBo586dwXmBQEDLly/XqFGj1KtXL2VnZ2v+/Pn67LPPQtYxaNAgORyOkK+qqqqQPidOnNDEiROVkpKinJwcVVdXd20PAQBAjxN2gLl+/bpGjx6tTZs2tZnX1NSkDz74QM8//7w++OADvfXWWzpz5oy+8Y1vtOm7Zs0aNTQ0BL+efvrp4Dyfz6fCwkLl5uaqrq5O69atU0VFhV599dVwywUAAD1QUrgLFBcXq7i4uN15ffr0kcfjCWn7+7//e02YMEEXLlzQwIEDg+29e/eW2+1udz1bt27VjRs3tHnzZjmdTo0YMUL19fXasGGDFi9eHG7JAACghwk7wITrypUrcjgc6tu3b0h7VVWV1q5dq4EDB2ru3LlatmyZkpL+WE5tba0mTZokp9MZ7F9UVKQf//jH+vzzz9WvX7822/H7/fL7/cFpn88n6Y+3tQKBQBT2DO1pPdbxfMxdiZbdJYRwJVgh33sK019jnCuxiXGxX2ePfVQDzBdffKHly5drzpw5SktLC7Z/97vf1QMPPKD09HQdPnxY5eXlamho0IYNGyRJXq9XgwcPDllXZmZmcF57AaayslKrV69u015TU6PU1NRI7hY64X9fiYsn1RPsrqB9a8e32F1CRO3Zs8fuEiIins+VWMa42KepqalT/aIWYAKBgL71rW/Jsiz99Kc/DZlXVlYW/DkvL09Op1NPPfWUKisr5XK5urS98vLykPX6fD7l5OSosLAwJDwhugKBgDwej6ZPn67k5GS7y7HFyIq37S4hhCvB0trxLXr+eIL8LQ67y4mYkxVFdpdwRzhXYhPjYr/WOyi3E5UA0xpefvOb3+idd965bYDIz8/XzZs39emnn2ro0KFyu91qbGwM6dM63dFzMy6Xq93wk5yczIvQBvF83P3NsRkS/C2OmK2tK3rK6yuez5VYxrjYp7PHPeJ/B6Y1vHzyySfat2+f+vfvf9tl6uvrlZCQoIyMDElSQUGBDh48GHIfzOPxaOjQoe3ePgIAAPEl7Csw165d09mzZ4PT58+fV319vdLT05WVlaW/+Iu/0AcffKBdu3apublZXq9XkpSeni6n06na2lodPXpUU6ZMUe/evVVbW6tly5bp8ccfD4aTuXPnavXq1Vq4cKGWL1+ukydP6qWXXtKLL74Yod0GAAAmCzvAHD9+XFOmTAlOtz53smDBAlVUVOhXv/qVJGnMmDEhy/3617/W5MmT5XK5tH37dlVUVMjv92vw4MFatmxZyPMrffr0UU1NjUpLSzVu3DgNGDBAK1eu5C3UAABAUhcCzOTJk2VZHb8d81bzJOmBBx7QkSNHbrudvLw8HTp0KNzyAABAHOCzkAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADBOkt0FAHYY9Oxuu0sAANwBrsAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxgk7wBw8eFCzZs1Sdna2HA6Hdu7cGTLfsiytXLlSWVlZuuuuuzRt2jR98sknIX0uXbqkefPmKS0tTX379tXChQt17dq1kD4nTpzQxIkTlZKSopycHFVXV4e/dwAAoEcKO8Bcv35do0eP1qZNm9qdX11drZ/85Cd65ZVXdPToUfXq1UtFRUX64osvgn3mzZunU6dOyePxaNeuXTp48KAWL14cnO/z+VRYWKjc3FzV1dVp3bp1qqio0KuvvtqFXQQAAD1NUrgLFBcXq7i4uN15lmVp48aNWrFihb75zW9Kkv7xH/9RmZmZ2rlzpx577DF9/PHH2rt3r95//32NHz9ekvTyyy9r5syZWr9+vbKzs7V161bduHFDmzdvltPp1IgRI1RfX68NGzaEBB0AABCfIvoMzPnz5+X1ejVt2rRgW58+fZSfn6/a2lpJUm1trfr27RsML5I0bdo0JSQk6OjRo8E+kyZNktPpDPYpKirSmTNn9Pnnn0eyZAAAYKCwr8DcitfrlSRlZmaGtGdmZgbneb1eZWRkhBaRlKT09PSQPoMHD26zjtZ5/fr1a7Ntv98vv98fnPb5fJKkQCCgQCBwJ7uFMLQe61g/5q5Ey+4Suo0rwQr53lPE+mvsdkw5V+IN42K/zh77iAYYO1VWVmr16tVt2mtqapSammpDRfHN4/HYXcItVU+wu4Lut3Z8i90lRNSePXvsLiEiYv1ciVeMi32ampo61S+iAcbtdkuSGhsblZWVFWxvbGzUmDFjgn0uXrwYstzNmzd16dKl4PJut1uNjY0hfVqnW/v8b+Xl5SorKwtO+3w+5eTkqLCwUGlpaXe2Y+i0QCAgj8ej6dOnKzk52e5yOjSy4m27S+g2rgRLa8e36PnjCfK3OOwuJ2JOVhTZXcIdMeVciTeMi/1a76DcTkQDzODBg+V2u7V///5gYPH5fDp69KiWLFkiSSooKNDly5dVV1encePGSZLeeecdtbS0KD8/P9jnueeeUyAQCL6APB6Phg4d2u7tI0lyuVxyuVxt2pOTk3kR2iDWj7u/uef8Iu8sf4ujR+13LL++whHr50q8Ylzs09njHvZDvNeuXVN9fb3q6+sl/fHB3fr6el24cEEOh0Pf+9739KMf/Ui/+tWv9NFHH2n+/PnKzs7WI488Ikm6//77NWPGDC1atEjHjh3Te++9p6VLl+qxxx5Tdna2JGnu3LlyOp1auHChTp06pR07duill14KucICAADiV9hXYI4fP64pU6YEp1tDxYIFC7Rlyxb97d/+ra5fv67Fixfr8uXLeuihh7R3716lpKQEl9m6dauWLl2qqVOnKiEhQbNnz9ZPfvKT4Pw+ffqopqZGpaWlGjdunAYMGKCVK1fyFmoAACCpCwFm8uTJsqyO383gcDi0Zs0arVmzpsM+6enp2rZt2y23k5eXp0OHDoVbHgAAiAM95l1IAOLLoGd3R2W9n1aVRGW9ACKLD3MEAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYJyIB5hBgwbJ4XC0+SotLZUkTZ48uc2873znOyHruHDhgkpKSpSamqqMjAw988wzunnzZqRLBQAAhkqK9Arff/99NTc3B6dPnjyp6dOn6y//8i+DbYsWLdKaNWuC06mpqcGfm5ubVVJSIrfbrcOHD6uhoUHz589XcnKyXnjhhUiXCwAADBTxAHP33XeHTFdVVWnIkCF6+OGHg22pqalyu93tLl9TU6PTp09r3759yszM1JgxY7R27VotX75cFRUVcjqdkS4ZAAAYJuIB5stu3LihX/ziFyorK5PD4Qi2b926Vb/4xS/kdrs1a9YsPf/888GrMLW1tRo1apQyMzOD/YuKirRkyRKdOnVKY8eObXdbfr9ffr8/OO3z+SRJgUBAgUAgGruHdrQe61g/5q5Ey+4Suo0rwQr5jlvrrteuKedKvGFc7NfZYx/VALNz505dvnxZ3/72t4Ntc+fOVW5urrKzs3XixAktX75cZ86c0VtvvSVJ8nq9IeFFUnDa6/V2uK3KykqtXr26TXtNTU3ILSp0D4/HY3cJt1Q9we4Kut/a8S12l2CEPXv2dOv2Yv1ciVeMi32ampo61S+qAea1115TcXGxsrOzg22LFy8O/jxq1ChlZWVp6tSpOnfunIYMGdLlbZWXl6usrCw47fP5lJOTo8LCQqWlpXV5vQhPIBCQx+PR9OnTlZycbHc5HRpZ8bbdJXQbV4KlteNb9PzxBPlbHLdfIM6drCjqlu2Ycq7EG8bFfq13UG4nagHmN7/5jfbt2xe8stKR/Px8SdLZs2c1ZMgQud1uHTt2LKRPY2OjJHX43IwkuVwuuVyuNu3Jycm8CG0Q68fd3xx/v8j9LY643O9wdffrNtbPlXjFuNins8c9an8H5vXXX1dGRoZKSkpu2a++vl6SlJWVJUkqKCjQRx99pIsXLwb7eDwepaWlafjw4dEqFwAAGCQqV2BaWlr0+uuva8GCBUpK+tMmzp07p23btmnmzJnq37+/Tpw4oWXLlmnSpEnKy8uTJBUWFmr48OF64oknVF1dLa/XqxUrVqi0tLTdKywAACD+RCXA7Nu3TxcuXNBf//Vfh7Q7nU7t27dPGzdu1PXr15WTk6PZs2drxYoVwT6JiYnatWuXlixZooKCAvXq1UsLFiwI+bsxAAAgvkUlwBQWFsqy2r5lMycnR+++++5tl8/Nze32dwIAAABz8FlIAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDhJdhcA3MqgZ3fbXQIAIAZxBQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOEl2FwAAsWTQs7ujtu5Pq0qitm4g3nAFBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcSIeYCoqKuRwOEK+hg0bFpz/xRdfqLS0VP3799dXvvIVzZ49W42NjSHruHDhgkpKSpSamqqMjAw988wzunnzZqRLBQAAhorK26hHjBihffv2/WkjSX/azLJly7R79269+eab6tOnj5YuXapHH31U7733niSpublZJSUlcrvdOnz4sBoaGjR//nwlJyfrhRdeiEa5AADAMFEJMElJSXK73W3ar1y5otdee03btm3T17/+dUnS66+/rvvvv19HjhzRgw8+qJqaGp0+fVr79u1TZmamxowZo7Vr12r58uWqqKiQ0+mMRskAAMAgUQkwn3zyibKzs5WSkqKCggJVVlZq4MCBqqurUyAQ0LRp04J9hw0bpoEDB6q2tlYPPvigamtrNWrUKGVmZgb7FBUVacmSJTp16pTGjh3b7jb9fr/8fn9w2ufzSZICgYACgUA0dhPtaD3WkTrmrkQrIuuJZ64EK+Q77PPl8yLS5woig3GxX2ePfcQDTH5+vrZs2aKhQ4eqoaFBq1ev1sSJE3Xy5El5vV45nU717ds3ZJnMzEx5vV5JktfrDQkvrfNb53WksrJSq1evbtNeU1Oj1NTUO9wrhMvj8URkPdUTIrIaSFo7vsXuEuLenj172rRF6lxBZDEu9mlqaupUv4gHmOLi4uDPeXl5ys/PV25urt544w3dddddkd5cUHl5ucrKyoLTPp9POTk5KiwsVFpaWtS2i1CBQEAej0fTp09XcnLyHa9vZMXbEagqvrkSLK0d36LnjyfI3+Kwu5y4drKiKPhzpM8VRAbjYr/WOyi3E/XPQurbt6/uu+8+nT17VtOnT9eNGzd0+fLlkKswjY2NwWdm3G63jh07FrKO1ncptfdcTSuXyyWXy9WmPTk5mRehDSJ13P3N/MKNFH+Lg+Nps/bOCf6Nik2Mi306e9yj/ndgrl27pnPnzikrK0vjxo1TcnKy9u/fH5x/5swZXbhwQQUFBZKkgoICffTRR7p48WKwj8fjUVpamoYPHx7tcgEAgAEifgXmBz/4gWbNmqXc3Fx99tlnWrVqlRITEzVnzhz16dNHCxcuVFlZmdLT05WWlqann35aBQUFevDBByVJhYWFGj58uJ544glVV1fL6/VqxYoVKi0tbfcKCwAAiD8RDzC/+93vNGfOHP3hD3/Q3XffrYceekhHjhzR3XffLUl68cUXlZCQoNmzZ8vv96uoqEj/8A//EFw+MTFRu3bt0pIlS1RQUKBevXppwYIFWrNmTaRLBQAAhop4gNm+ffst56ekpGjTpk3atGlTh31yc3PbfVofAABA4rOQAACAgQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABgn4gGmsrJSX/va19S7d29lZGTokUce0ZkzZ0L6TJ48WQ6HI+TrO9/5TkifCxcuqKSkRKmpqcrIyNAzzzyjmzdvRrpcAABgoKRIr/Ddd99VaWmpvva1r+nmzZv64Q9/qMLCQp0+fVq9evUK9lu0aJHWrFkTnE5NTQ3+3NzcrJKSErndbh0+fFgNDQ2aP3++kpOT9cILL0S6ZAAAYJiIB5i9e/eGTG/ZskUZGRmqq6vTpEmTgu2pqalyu93trqOmpkanT5/Wvn37lJmZqTFjxmjt2rVavny5Kioq5HQ6I102AAAwSNSfgbly5YokKT09PaR969atGjBggEaOHKny8nI1NTUF59XW1mrUqFHKzMwMthUVFcnn8+nUqVPRLhkAAMS4iF+B+bKWlhZ973vf05/92Z9p5MiRwfa5c+cqNzdX2dnZOnHihJYvX64zZ87orbfekiR5vd6Q8CIpOO31etvdlt/vl9/vD077fD5JUiAQUCAQiOh+oWOtxzpSx9yVaEVkPfHMlWCFfId9vnxeRPpcQWQwLvbr7LGPaoApLS3VyZMn9e///u8h7YsXLw7+PGrUKGVlZWnq1Kk6d+6chgwZ0qVtVVZWavXq1W3aa2pqQp6vQffweDwRWU/1hIisBpLWjm+xu4S4t2fPnjZtkTpXEFmMi32+fEfmVqIWYJYuXapdu3bp4MGD+upXv3rLvvn5+ZKks2fPasiQIXK73Tp27FhIn8bGRknq8LmZ8vJylZWVBad9Pp9ycnJUWFiotLS0O9kV3MbIireDP7sSLK0d36LnjyfI3+KwsSq0YkxiUyTH5WRFUYSqQiAQkMfj0fTp05WcnGx3OXGp9Q7K7UQ8wFiWpaefflr/+q//qgMHDmjw4MG3Xaa+vl6SlJWVJUkqKCjQ3/3d3+nixYvKyMiQ9Mc0nJaWpuHDh7e7DpfLJZfL1aY9OTmZF2GU+Zvb/uPrb3G02w77MCaxKRLjwr9xkcfvDvt09rhHPMCUlpZq27Zt+uUvf6nevXsHn1np06eP7rrrLp07d07btm3TzJkz1b9/f504cULLli3TpEmTlJeXJ0kqLCzU8OHD9cQTT6i6ulper1crVqxQaWlpuyEFAADEl4i/C+mnP/2prly5osmTJysrKyv4tWPHDkmS0+nUvn37VFhYqGHDhun73/++Zs+erX/7t38LriMxMVG7du1SYmKiCgoK9Pjjj2v+/PkhfzcGAADEr6jcQrqVnJwcvfvuu7ddT25ubrsPvAEAAPBZSAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMk2V0AAODODHp2d9TW/WlVSdTWDdwJrsAAAADjEGAAAIBxCDAAAMA4PAMTJ6J5jxwAgO7GFRgAAGAcAgwAADAOAQYAABiHZ2AAAB3ib8wgVnEFBgAAGIcAAwAAjEOAAQAAxiHAAAAA4/AQLwDAFtF6QJiHg+MDASbG8BdzAQC4vZi+hbRp0yYNGjRIKSkpys/P17Fjx+wuCQAAxICYvQKzY8cOlZWV6ZVXXlF+fr42btyooqIinTlzRhkZGXaXBwCIUXdyJduVaKl6gjSy4m35mx1t5nN7KnbE7BWYDRs2aNGiRXryySc1fPhwvfLKK0pNTdXmzZvtLg0AANgsJq/A3LhxQ3V1dSovLw+2JSQkaNq0aaqtrW13Gb/fL7/fH5y+cuWKJOnSpUsKBAIRrS+/cn9E1/dlMTkgYUhqsdTU1KKkQIKaW9r+7wXdjzGJTYxLbLrduNz7gzdsqOrOHC2fancJYbl69aokybKsW/aLyd+Xv//979Xc3KzMzMyQ9szMTP3nf/5nu8tUVlZq9erVbdoHDx4clRrRsbl2F4A2GJPYxLjEpp42LgP+r90VdM3Vq1fVp0+fDufHZIDpivLycpWVlQWnW1padOnSJfXv318OB/+76S4+n085OTn67W9/q7S0NLvLgRiTWMW4xCbGxX6WZenq1avKzs6+Zb+YDDADBgxQYmKiGhsbQ9obGxvldrvbXcblcsnlcoW09e3bN1ol4jbS0tI4+WMMYxKbGJfYxLjY61ZXXlrF5EO8TqdT48aN0/79f3rWpKWlRfv371dBQYGNlQEAgFgQk1dgJKmsrEwLFizQ+PHjNWHCBG3cuFHXr1/Xk08+aXdpAADAZjEbYP7qr/5K//M//6OVK1fK6/VqzJgx2rt3b5sHexFbXC6XVq1a1eZ2HuzDmMQmxiU2MS7mcFi3e58SAABAjInJZ2AAAABuhQADAACMQ4ABAADGIcAAAADjEGAQlk2bNmnQoEFKSUlRfn6+jh07dsv+b775poYNG6aUlBSNGjVKe/bs6aZK40s443Lq1CnNnj1bgwYNksPh0MaNG7uv0DgTzrj87Gc/08SJE9WvXz/169dP06ZNu+35ha4JZ1zeeustjR8/Xn379lWvXr00ZswY/dM//VM3VouOEGDQaTt27FBZWZlWrVqlDz74QKNHj1ZRUZEuXrzYbv/Dhw9rzpw5WrhwoT788EM98sgjeuSRR3Ty5MlurrxnC3dcmpqadM8996iqqqrDv2yNOxfuuBw4cEBz5szRr3/9a9XW1ionJ0eFhYX67//+726uvGcLd1zS09P13HPPqba2VidOnNCTTz6pJ598Um+//XY3V442LKCTJkyYYJWWlganm5ubrezsbKuysrLd/t/61reskpKSkLb8/Hzrqaeeimqd8Sbccfmy3Nxc68UXX4xidfHrTsbFsizr5s2bVu/eva2f//zn0SoxLt3puFiWZY0dO9ZasWJFNMpDGLgCg065ceOG6urqNG3atGBbQkKCpk2bptra2naXqa2tDekvSUVFRR32R/i6Mi6IvkiMS1NTkwKBgNLT06NVZty503GxLEv79+/XmTNnNGnSpGiWik4gwKBTfv/736u5ubnNX0LOzMyU1+ttdxmv1xtWf4SvK+OC6IvEuCxfvlzZ2dlt/hOAruvquFy5ckVf+cpX5HQ6VVJSopdfflnTp0+Pdrm4jZj9KAEAiFdVVVXavn27Dhw4oJSUFLvLiXu9e/dWfX29rl27pv3796usrEz33HOPJk+ebHdpcY0Ag04ZMGCAEhMT1djYGNLe2NjY4YOgbrc7rP4IX1fGBdF3J+Oyfv16VVVVad++fcrLy4tmmXGnq+OSkJCge++9V5I0ZswYffzxx6qsrCTA2IxbSOgUp9OpcePGaf/+/cG2lpYW7d+/XwUFBe0uU1BQENJfkjweT4f9Eb6ujAuir6vjUl1drbVr12rv3r0aP358d5QaVyJ1vrS0tMjv90ejRITD7qeIYY7t27dbLpfL2rJli3X69Glr8eLFVt++fS2v12tZlmU98cQT1rPPPhvs/95771lJSUnW+vXrrY8//thatWqVlZycbH300Ud27UKPFO64+P1+68MPP7Q+/PBDKysry/rBD35gffjhh9Ynn3xi1y70SOGOS1VVleV0Oq1/+Zd/sRoaGoJfV69etWsXeqRwx+WFF16wampqrHPnzlmnT5+21q9fbyUlJVk/+9nP7NoF/H8EGITl5ZdftgYOHGg5nU5rwoQJ1pEjR4LzHn74YWvBggUh/d944w3rvvvus5xOpzVixAhr9+7d3VxxfAhnXM6fP29JavP18MMPd3/hPVw445Kbm9vuuKxatar7C+/hwhmX5557zrr33nutlJQUq1+/flZBQYG1fft2G6rG/+awLMuy6+oPAABAV/AMDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADG+X/c/DU267Mg7QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "scores = pd.Series([doc['score'] for doc in docs])\n",
    "\n",
    "scores.hist(bins=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6dd782e2-38c7-4e4d-91c2-618f0607e3bd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    10548.000000\n",
       "mean         0.106808\n",
       "std          0.051627\n",
       "min         -0.069786\n",
       "25%          0.071219\n",
       "50%          0.104501\n",
       "75%          0.140160\n",
       "max          0.372656\n",
       "dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scores.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "358027b0-b78a-4591-aa26-5d3f1f081bce",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(0.46217292377701935)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "THRESH = 0.1\n",
    "\n",
    "(scores < THRESH).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a9684cb1-ce70-4718-918b-b5508196e8e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# with kb.bulk_del_docs() as del_doc:\n",
    "#     for doc in sorted(docs, key = lambda doc: doc['score'], reverse=True):\n",
    "#         if doc['score'] < THRESH:\n",
    "#             break\n",
    "#         print(f\"{doc['score']:.4f}: {doc['doc']['text']}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "42587de4-83d5-4af8-b6c4-1d4ec3a68321",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-17 02:28:44,105 - svs.kb - INFO - starting bulk-delete (as new database transaction)\n",
      "2024-07-17 02:28:44,928 - svs.kb - INFO - invalidating cached vectors; they'll be re-built next time you `retrieve()`\n",
      "2024-07-17 02:28:44,933 - svs.kb - INFO - ending bulk-delete (committing the database transaction)\n"
     ]
    }
   ],
   "source": [
    "with kb.bulk_del_docs() as del_doc:\n",
    "    for doc in sorted(docs, key = lambda doc: doc['score'], reverse=True):\n",
    "        if doc['score'] < THRESH:\n",
    "            break\n",
    "        del_doc(doc['doc']['id'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b819fe7b-a1ae-4ead-a61d-1d763d16ec44",
   "metadata": {},
   "source": [
    "# Step 3: Remove duplicates\n",
    "\n",
    "We'll look at documents' pairwise semantic similarity scores to find _duplicate_ jokes! I don't want duplicate jokes!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9a225f42-2f13-4f62-bee6-fc1883a25e2f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-17 02:28:45,125 - svs.kb - INFO - re-building cached vectors...\n",
      "2024-07-17 02:28:45,611 - svs.kb - INFO - re-building cached vectors... DONE!\n",
      "2024-07-17 02:28:45,612 - svs.kb - INFO - computing pairwise similarity over 4875 documents\n",
      "2024-07-17 02:28:46,078 - svs.kb - INFO - computed 23765625 pairwise cosine similarities\n",
      "2024-07-17 02:28:46,108 - svs.kb - INFO - retrieved top 10000 document pairs\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Axes: >"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGdCAYAAAAMm0nCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAwzklEQVR4nO3dfXRU9Z3H8U8IkwlRBgqaTLIEjFCByKNYYKpF5CERsiyunLNSFVhFOLDBcyBdQCpiADUuLSJVhHpE4h6hKhatAiUZwgZEgmgkCwRlBXHRlQlbEIbHyZDc/aMns0x5yjyR/Mj7dU5Ouff+5ne/99tJ8+l9mImzLMsSAACAQZo1dAEAAAChIsAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIzTvKELiJXa2lr98MMPatmypeLi4hq6HAAAUA+WZenkyZNKS0tTs2aXP89y3QaYH374Qenp6Q1dBgAACMN3332ndu3aXXb7dRtgWrZsKemvDXA4HA1cTePn9/tVXFysrKws2Wy2hi7HOPQvMvQvfPQuMvQvMrHon9frVXp6euDv+OVctwGm7rKRw+EgwNSD3+9XUlKSHA4Hv8RhoH+RoX/ho3eRoX+RiWX/rnb7BzfxAgAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABineUMXYKJbnlwXs7m/fSEnZnMDAHC94AwMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjhBRgli5dqh49esjhcMjhcMjlcunPf/5zYPvAgQMVFxcX9DNp0qSgOQ4dOqScnBwlJSUpOTlZ06dP1/nz54PGlJaW6o477pDdblenTp1UWFgY/hECAIDrTkiPUbdr104vvPCCfvrTn8qyLL355psaOXKkdu7cqdtvv12SNGHCBM2bNy/wmqSkpMC/a2pqlJOTI6fTqW3btunw4cMaO3asbDabnn/+eUnSwYMHlZOTo0mTJmnlypUqKSnR448/rtTUVGVnZ0fjmAEAgOFCCjAjRowIWn7uuee0dOlSbd++PRBgkpKS5HQ6L/n64uJi7d27Vxs3blRKSop69eql+fPna+bMmcrPz1dCQoKWLVumjIwMLVy4UJLUtWtXbd26VYsWLSLAAAAASRF8kF1NTY1Wr16t06dPy+VyBdavXLlSb731lpxOp0aMGKGnn346cBamrKxM3bt3V0pKSmB8dna2Jk+erMrKSvXu3VtlZWUaMmRI0L6ys7M1derUK9bj8/nk8/kCy16vV5Lk9/vl9/vDPcxLssdbUZ3vQtGuNdT9NtT+TUf/IkP/wkfvIkP/IhOL/tV3rpADzO7du+VyuXTu3DndeOONev/995WZmSlJeuihh9ShQwelpaVp165dmjlzpvbt26c1a9ZIkjweT1B4kRRY9ng8Vxzj9Xp19uxZtWjR4pJ1FRQUaO7cuRetLy4uDrqMFQ0L+kZ1uiDr16+P3eT14Ha7G3T/pqN/kaF/4aN3kaF/kYlm/86cOVOvcSEHmM6dO6uiokInTpzQe++9p3Hjxmnz5s3KzMzUxIkTA+O6d++u1NRUDR48WAcOHFDHjh1D3VVIZs2apby8vMCy1+tVenq6srKy5HA4orqvbvlFUZ3vQnvyG+Yymd/vl9vt1tChQ2Wz2RqkBpPRv8jQv/DRu8jQv8jEon91V1CuJuQAk5CQoE6dOkmS+vTpo88++0yLFy/W73//+4vG9uvXT5K0f/9+dezYUU6nUzt27AgaU1VVJUmB+2acTmdg3YVjHA7HZc++SJLdbpfdbr9ovc1mi/qb0lcTF9X5LtTQv0Cx6FdTQv8iQ//CR+8iQ/8iE83+1XeeiD8Hpra2NujekwtVVFRIklJTUyVJLpdLu3fv1pEjRwJj3G63HA5H4DKUy+VSSUlJ0DxutzvoPhsAANC0hXQGZtasWRo2bJjat2+vkydPatWqVSotLVVRUZEOHDigVatWafjw4Wrbtq127dqladOmacCAAerRo4ckKSsrS5mZmRozZowWLFggj8ej2bNnKzc3N3D2ZNKkSXrllVc0Y8YMPfbYY9q0aZPeffddrVsXu2+ABgAAZgkpwBw5ckRjx47V4cOH1apVK/Xo0UNFRUUaOnSovvvuO23cuFEvvfSSTp8+rfT0dI0aNUqzZ88OvD4+Pl5r167V5MmT5XK5dMMNN2jcuHFBnxuTkZGhdevWadq0aVq8eLHatWun119/nUeoAQBAQEgBZvny5Zfdlp6ers2bN191jg4dOlz1SZuBAwdq586doZQGAACaEL4LCQAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4IQWYpUuXqkePHnI4HHI4HHK5XPrzn/8c2H7u3Dnl5uaqbdu2uvHGGzVq1ChVVVUFzXHo0CHl5OQoKSlJycnJmj59us6fPx80prS0VHfccYfsdrs6deqkwsLC8I8QAABcd0IKMO3atdMLL7yg8vJyff755xo0aJBGjhypyspKSdK0adP00UcfafXq1dq8ebN++OEHPfDAA4HX19TUKCcnR9XV1dq2bZvefPNNFRYWas6cOYExBw8eVE5Oju69915VVFRo6tSpevzxx1VUVBSlQwYAAKZrHsrgESNGBC0/99xzWrp0qbZv36527dpp+fLlWrVqlQYNGiRJWrFihbp27art27erf//+Ki4u1t69e7Vx40alpKSoV69emj9/vmbOnKn8/HwlJCRo2bJlysjI0MKFCyVJXbt21datW7Vo0SJlZ2dH6bABAIDJQgowF6qpqdHq1at1+vRpuVwulZeXy+/3a8iQIYExXbp0Ufv27VVWVqb+/furrKxM3bt3V0pKSmBMdna2Jk+erMrKSvXu3VtlZWVBc9SNmTp16hXr8fl88vl8gWWv1ytJ8vv98vv94R7mJdnjrajOd6Fo1xrqfhtq/6ajf5Ghf+Gjd5Ghf5GJRf/qO1fIAWb37t1yuVw6d+6cbrzxRr3//vvKzMxURUWFEhIS1Lp166DxKSkp8ng8kiSPxxMUXuq212270hiv16uzZ8+qRYsWl6yroKBAc+fOvWh9cXGxkpKSQj3MK1rQN6rTBVm/fn3sJq8Ht9vdoPs3Hf2LDP0LH72LDP2LTDT7d+bMmXqNCznAdO7cWRUVFTpx4oTee+89jRs3Tps3bw65wGibNWuW8vLyAster1fp6enKysqSw+GI6r665cfufpw9+Q1zmczv98vtdmvo0KGy2WwNUoPJ6F9k6F/46F1k6F9kYtG/uisoVxNygElISFCnTp0kSX369NFnn32mxYsX68EHH1R1dbWOHz8edBamqqpKTqdTkuR0OrVjx46g+eqeUrpwzN8+uVRVVSWHw3HZsy+SZLfbZbfbL1pvs9mi/qb01cRFdb4LNfQvUCz61ZTQv8jQv/DRu8jQv8hEs3/1nSfiz4Gpra2Vz+dTnz59ZLPZVFJSEti2b98+HTp0SC6XS5Lkcrm0e/duHTlyJDDG7XbL4XAoMzMzMObCOerG1M0BAAAQ0hmYWbNmadiwYWrfvr1OnjypVatWqbS0VEVFRWrVqpXGjx+vvLw8tWnTRg6HQ0888YRcLpf69+8vScrKylJmZqbGjBmjBQsWyOPxaPbs2crNzQ2cPZk0aZJeeeUVzZgxQ4899pg2bdqkd999V+vWrYv+0QMAACOFFGCOHDmisWPH6vDhw2rVqpV69OihoqIiDR06VJK0aNEiNWvWTKNGjZLP51N2drZeffXVwOvj4+O1du1aTZ48WS6XSzfccIPGjRunefPmBcZkZGRo3bp1mjZtmhYvXqx27drp9ddf5xFqAAAQEFKAWb58+RW3JyYmasmSJVqyZMllx3To0OGqT9oMHDhQO3fuDKU0AADQhPBdSAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGCSnAFBQU6Gc/+5latmyp5ORk3X///dq3b1/QmIEDByouLi7oZ9KkSUFjDh06pJycHCUlJSk5OVnTp0/X+fPng8aUlpbqjjvukN1uV6dOnVRYWBjeEQIAgOtOSAFm8+bNys3N1fbt2+V2u+X3+5WVlaXTp08HjZswYYIOHz4c+FmwYEFgW01NjXJyclRdXa1t27bpzTffVGFhoebMmRMYc/DgQeXk5Ojee+9VRUWFpk6dqscff1xFRUURHi4AALgeNA9l8IYNG4KWCwsLlZycrPLycg0YMCCwPikpSU6n85JzFBcXa+/evdq4caNSUlLUq1cvzZ8/XzNnzlR+fr4SEhK0bNkyZWRkaOHChZKkrl27auvWrVq0aJGys7NDPUYAAHCdCSnA/K0TJ05Iktq0aRO0fuXKlXrrrbfkdDo1YsQIPf3000pKSpIklZWVqXv37kpJSQmMz87O1uTJk1VZWanevXurrKxMQ4YMCZozOztbU6dOvWwtPp9PPp8vsOz1eiVJfr9ffr8/ksO8iD3eiup8F4p2raHut6H2bzr6Fxn6Fz56Fxn6F5lY9K++c4UdYGprazV16lTddddd6tatW2D9Qw89pA4dOigtLU27du3SzJkztW/fPq1Zs0aS5PF4gsKLpMCyx+O54hiv16uzZ8+qRYsWF9VTUFCguXPnXrS+uLg4EJ6iZUHfqE4XZP369bGbvB7cbneD7t909C8y9C989C4y9C8y0ezfmTNn6jUu7ACTm5urPXv2aOvWrUHrJ06cGPh39+7dlZqaqsGDB+vAgQPq2LFjuLu7qlmzZikvLy+w7PV6lZ6erqysLDkcjqjuq1t+7O7F2ZPfMJfI/H6/3G63hg4dKpvN1iA1mIz+RYb+hY/eRYb+RSYW/au7gnI1YQWYKVOmaO3atdqyZYvatWt3xbH9+vWTJO3fv18dO3aU0+nUjh07gsZUVVVJUuC+GafTGVh34RiHw3HJsy+SZLfbZbfbL1pvs9mi/qb01cRFdb4LNfQvUCz61ZTQv8jQv/DRu8jQv8hEs3/1nSekp5Asy9KUKVP0/vvva9OmTcrIyLjqayoqKiRJqampkiSXy6Xdu3fryJEjgTFut1sOh0OZmZmBMSUlJUHzuN1uuVyuUMoFAADXqZACTG5urt566y2tWrVKLVu2lMfjkcfj0dmzZyVJBw4c0Pz581VeXq5vv/1WH374ocaOHasBAwaoR48ekqSsrCxlZmZqzJgx+s///E8VFRVp9uzZys3NDZxBmTRpkr755hvNmDFDX331lV599VW9++67mjZtWpQPHwAAmCikALN06VKdOHFCAwcOVGpqauDnnXfekSQlJCRo48aNysrKUpcuXfSrX/1Ko0aN0kcffRSYIz4+XmvXrlV8fLxcLpceeeQRjR07VvPmzQuMycjI0Lp16+R2u9WzZ08tXLhQr7/+Oo9QAwAASSHeA2NZV358OD09XZs3b77qPB06dLjq0zYDBw7Uzp07QykPAAA0EXwXEgAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjhBRgCgoK9LOf/UwtW7ZUcnKy7r//fu3bty9ozLlz55Sbm6u2bdvqxhtv1KhRo1RVVRU05tChQ8rJyVFSUpKSk5M1ffp0nT9/PmhMaWmp7rjjDtntdnXq1EmFhYXhHSEAALjuhBRgNm/erNzcXG3fvl1ut1t+v19ZWVk6ffp0YMy0adP00UcfafXq1dq8ebN++OEHPfDAA4HtNTU1ysnJUXV1tbZt26Y333xThYWFmjNnTmDMwYMHlZOTo3vvvVcVFRWaOnWqHn/8cRUVFUXhkAEAgOmahzJ4w4YNQcuFhYVKTk5WeXm5BgwYoBMnTmj58uVatWqVBg0aJElasWKFunbtqu3bt6t///4qLi7W3r17tXHjRqWkpKhXr16aP3++Zs6cqfz8fCUkJGjZsmXKyMjQwoULJUldu3bV1q1btWjRImVnZ0fp0AEAgKkiugfmxIkTkqQ2bdpIksrLy+X3+zVkyJDAmC5duqh9+/YqKyuTJJWVlal79+5KSUkJjMnOzpbX61VlZWVgzIVz1I2pmwMAADRtIZ2BuVBtba2mTp2qu+66S926dZMkeTweJSQkqHXr1kFjU1JS5PF4AmMuDC912+u2XWmM1+vV2bNn1aJFi4vq8fl88vl8gWWv1ytJ8vv98vv94R7mJdnjrajOd6HOT62Nybx78q985qquR9HuVVNB/yJD/8JH7yJD/yITi/7Vd66wA0xubq727NmjrVu3hjtFVBUUFGju3LkXrS8uLlZSUlJU97Wgb1SnuybWr19fr3FutzvGlVzf6F9k6F/46F1k6F9kotm/M2fO1GtcWAFmypQpWrt2rbZs2aJ27doF1judTlVXV+v48eNBZ2GqqqrkdDoDY3bs2BE0X91TSheO+dsnl6qqquRwOC559kWSZs2apby8vMCy1+tVenq6srKy5HA4wjnMy+qWb97NxPU5A+N2uzV06FDZbLZrVNX1g/5Fhv6Fj95Fhv5FJhb9q7uCcjUhBRjLsvTEE0/o/fffV2lpqTIyMoK29+nTRzabTSUlJRo1apQkad++fTp06JBcLpckyeVy6bnnntORI0eUnJws6a/JzeFwKDMzMzDmb88YuN3uwByXYrfbZbfbL1pvs9mi/qb01cRFdb5rob49iEW/mhL6Fxn6Fz56Fxn6F5lo9q++84QUYHJzc7Vq1Sr96U9/UsuWLQP3rLRq1UotWrRQq1atNH78eOXl5alNmzZyOBx64okn5HK51L9/f0lSVlaWMjMzNWbMGC1YsEAej0ezZ89Wbm5uIIBMmjRJr7zyimbMmKHHHntMmzZt0rvvvqt169aFUi4AALhOhfQU0tKlS3XixAkNHDhQqampgZ933nknMGbRokX6+7//e40aNUoDBgyQ0+nUmjVrAtvj4+O1du1axcfHy+Vy6ZFHHtHYsWM1b968wJiMjAytW7dObrdbPXv21MKFC/X666/zCDUAAJAUxiWkq0lMTNSSJUu0ZMmSy47p0KHDVW8qHThwoHbu3BlKeQAAoIngu5AAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjBNygNmyZYtGjBihtLQ0xcXF6YMPPgja/s///M+Ki4sL+rnvvvuCxhw7dkwPP/ywHA6HWrdurfHjx+vUqVNBY3bt2qVf/OIXSkxMVHp6uhYsWBD60QEAgOtSyAHm9OnT6tmzp5YsWXLZMffdd58OHz4c+PnDH/4QtP3hhx9WZWWl3G631q5dqy1btmjixImB7V6vV1lZWerQoYPKy8v1m9/8Rvn5+XrttddCLRcAAFyHmof6gmHDhmnYsGFXHGO32+V0Oi+57csvv9SGDRv02Wef6c4775Qkvfzyyxo+fLh++9vfKi0tTStXrlR1dbXeeOMNJSQk6Pbbb1dFRYVefPHFoKADAACappjcA1NaWqrk5GR17txZkydP1tGjRwPbysrK1Lp160B4kaQhQ4aoWbNm+vTTTwNjBgwYoISEhMCY7Oxs7du3Tz/++GMsSgYAAAYJ+QzM1dx333164IEHlJGRoQMHDujXv/61hg0bprKyMsXHx8vj8Sg5OTm4iObN1aZNG3k8HkmSx+NRRkZG0JiUlJTAtp/85CcX7dfn88nn8wWWvV6vJMnv98vv90f1GO3xVlTnuxau1oO67dHuVVNB/yJD/8JH7yJD/yITi/7Vd66oB5jRo0cH/t29e3f16NFDHTt2VGlpqQYPHhzt3QUUFBRo7ty5F60vLi5WUlJSVPe1oG9Up7sm1q9fX69xbrc7xpVc3+hfZOhf+OhdZOhfZKLZvzNnztRrXNQDzN+69dZbddNNN2n//v0aPHiwnE6njhw5EjTm/PnzOnbsWOC+GafTqaqqqqAxdcuXu7dm1qxZysvLCyx7vV6lp6crKytLDocjmoekbvlFUZ3vWtiTn33F7X6/X263W0OHDpXNZrtGVV0/6F9k6F/46F1k6F9kYtG/uisoVxPzAPP999/r6NGjSk1NlSS5XC4dP35c5eXl6tOnjyRp06ZNqq2tVb9+/QJjnnrqKfn9/kBD3G63OnfufMnLR9Jfbxy22+0XrbfZbFF/U/pq4qI637VQ3x7Eol9NCf2LDP0LH72LDP2LTDT7V995Qr6J99SpU6qoqFBFRYUk6eDBg6qoqNChQ4d06tQpTZ8+Xdu3b9e3336rkpISjRw5Up06dVJ29l/PAHTt2lX33XefJkyYoB07duiTTz7RlClTNHr0aKWlpUmSHnroISUkJGj8+PGqrKzUO++8o8WLFwedYQEAAE1XyAHm888/V+/evdW7d29JUl5ennr37q05c+YoPj5eu3bt0j/8wz/otttu0/jx49WnTx99/PHHQWdHVq5cqS5dumjw4MEaPny47r777qDPeGnVqpWKi4t18OBB9enTR7/61a80Z84cHqEGAACSwriENHDgQFnW5Z/CKSq6+v0hbdq00apVq644pkePHvr4449DLQ8AADQBfBcSAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxmjd0Abg2bnly3RW32+MtLegrdcsvkq8mLqS5v30hJ5LSAAAIGWdgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGCTnAbNmyRSNGjFBaWpri4uL0wQcfBG23LEtz5sxRamqqWrRooSFDhujrr78OGnPs2DE9/PDDcjgcat26tcaPH69Tp04Fjdm1a5d+8YtfKDExUenp6VqwYEHoRwcAAK5LIQeY06dPq2fPnlqyZMklty9YsEC/+93vtGzZMn366ae64YYblJ2drXPnzgXGPPzww6qsrJTb7dbatWu1ZcsWTZw4MbDd6/UqKytLHTp0UHl5uX7zm98oPz9fr732WhiHCAAArjchfxLvsGHDNGzYsEtusyxLL730kmbPnq2RI0dKkv793/9dKSkp+uCDDzR69Gh9+eWX2rBhgz777DPdeeedkqSXX35Zw4cP129/+1ulpaVp5cqVqq6u1htvvKGEhATdfvvtqqio0IsvvhgUdAAAQNMU1a8SOHjwoDwej4YMGRJY16pVK/Xr109lZWUaPXq0ysrK1Lp160B4kaQhQ4aoWbNm+vTTT/WP//iPKisr04ABA5SQkBAYk52drX/7t3/Tjz/+qJ/85CcX7dvn88nn8wWWvV6vJMnv98vv90fzMGWPt6I6X2Ngb2YF/Wcoot1fE9X1gF6Eh/6Fj95Fhv5FJhb9q+9cUQ0wHo9HkpSSkhK0PiUlJbDN4/EoOTk5uIjmzdWmTZugMRkZGRfNUbftUgGmoKBAc+fOvWh9cXGxkpKSwjyiS1vQN6rTNSrz76wN+TXr16+PQSVmcrvdDV2C0ehf+OhdZOhfZKLZvzNnztRr3HXzZY6zZs1SXl5eYNnr9So9PV1ZWVlyOBxR3Ve3/KKoztcY2JtZmn9nrZ7+vJl8taF9meOe/OwYVWUOv98vt9utoUOHymazNXQ5xqF/4aN3kaF/kYlF/+quoFxNVAOM0+mUJFVVVSk1NTWwvqqqSr169QqMOXLkSNDrzp8/r2PHjgVe73Q6VVVVFTSmbrluzN+y2+2y2+0XrbfZbFF/U4b6bc0m8dXGhXx8/NL/v1i835oS+hc+ehcZ+heZaPavvvNE9XNgMjIy5HQ6VVJSEljn9Xr16aefyuVySZJcLpeOHz+u8vLywJhNmzaptrZW/fr1C4zZsmVL0HUwt9utzp07X/LyEQAAaFpCDjCnTp1SRUWFKioqJP31xt2KigodOnRIcXFxmjp1qp599ll9+OGH2r17t8aOHau0tDTdf//9kqSuXbvqvvvu04QJE7Rjxw598sknmjJlikaPHq20tDRJ0kMPPaSEhASNHz9elZWVeuedd7R48eKgS0QAAKDpCvkS0ueff6577703sFwXKsaNG6fCwkLNmDFDp0+f1sSJE3X8+HHdfffd2rBhgxITEwOvWblypaZMmaLBgwerWbNmGjVqlH73u98Ftrdq1UrFxcXKzc1Vnz59dNNNN2nOnDk8Qg0AACSFEWAGDhwoy7r8o7ZxcXGaN2+e5s2bd9kxbdq00apVq664nx49eujjjz8OtTwAANAE8F1IAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjEGAAAIBxCDAAAMA4BBgAAGAcAgwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4zRv6AJgvlueXBezub99ISdmcwMAzMUZGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADGIcAAAADjRD3A5OfnKy4uLuinS5cuge3nzp1Tbm6u2rZtqxtvvFGjRo1SVVVV0ByHDh1STk6OkpKSlJycrOnTp+v8+fPRLhUAABiqeSwmvf3227Vx48b/30nz/9/NtGnTtG7dOq1evVqtWrXSlClT9MADD+iTTz6RJNXU1CgnJ0dOp1Pbtm3T4cOHNXbsWNlsNj3//POxKBcAABgmJgGmefPmcjqdF60/ceKEli9frlWrVmnQoEGSpBUrVqhr167avn27+vfvr+LiYu3du1cbN25USkqKevXqpfnz52vmzJnKz89XQkJCLEoGAAAGiUmA+frrr5WWlqbExES5XC4VFBSoffv2Ki8vl9/v15AhQwJju3Tpovbt26usrEz9+/dXWVmZunfvrpSUlMCY7OxsTZ48WZWVlerdu/cl9+nz+eTz+QLLXq9XkuT3++X3+6N6fPZ4K6rzNQb2ZlbQfzYW0f7vLlbq6jSl3saG/oWP3kWG/kUmFv2r71xRDzD9+vVTYWGhOnfurMOHD2vu3Ln6xS9+oT179sjj8SghIUGtW7cOek1KSoo8Ho8kyePxBIWXuu112y6noKBAc+fOvWh9cXGxkpKSIjyqYAv6RnW6RmX+nbUNXUKQ9evXN3QJIXG73Q1dgtHoX/joXWToX2Si2b8zZ87Ua1zUA8ywYcMC/+7Ro4f69eunDh066N1331WLFi2ivbuAWbNmKS8vL7Ds9XqVnp6urKwsORyOqO6rW35RVOdrDOzNLM2/s1ZPf95Mvtq4hi4nYE9+dkOXUC9+v19ut1tDhw6VzWZr6HKMQ//CR+8iQ/8iE4v+1V1BuZqYXEK6UOvWrXXbbbdp//79Gjp0qKqrq3X8+PGgszBVVVWBe2acTqd27NgRNEfdU0qXuq+mjt1ul91uv2i9zWaL+pvSV9N4/sBHm682rlEdn2n/gxKL91tTQv/CR+8iQ/8iE83+1XeemH8OzKlTp3TgwAGlpqaqT58+stlsKikpCWzft2+fDh06JJfLJUlyuVzavXu3jhw5EhjjdrvlcDiUmZkZ63IBAIABon4G5l//9V81YsQIdejQQT/88IOeeeYZxcfH65e//KVatWql8ePHKy8vT23atJHD4dATTzwhl8ul/v37S5KysrKUmZmpMWPGaMGCBfJ4PJo9e7Zyc3MveYYFAAA0PVEPMN9//71++ctf6ujRo7r55pt19913a/v27br55pslSYsWLVKzZs00atQo+Xw+ZWdn69VXXw28Pj4+XmvXrtXkyZPlcrl0ww03aNy4cZo3b160SwUAAIaKeoB5++23r7g9MTFRS5Ys0ZIlSy47pkOHDsY9fQIAAK4dvgsJAAAYJ+ZPIQGRuOXJdTGZ99sXcmIyLwDg2uAMDAAAMA4BBgAAGIcAAwAAjEOAAQAAxiHAAAAA4xBgAACAcQgwAADAOAQYAABgHAIMAAAwDgEGAAAYhwADAACMQ4ABAADG4cscAQC4jl2vX4pLgEGTFO1faHu8pQV9pW75Rdr33N9HdW4AwMW4hAQAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcPsgOi7Hr91EsAaEw4AwMAAIzDGRgAABpYrM7cXs84AwMAAIzDGRjAELH8f2jcXwPANAQYALgOdMsvkq8mLurzEm7RWBFgAPDkFADjEGAAGMnE0BWLmu3xlhb0jfq0QKNHgAEAXBb3XqGxIsAAiJn6/vGrO4sQq/s4QsHjrNdOY+h1qO89QlfjQYABAKCeGkPowl/xOTAAAMA4BBgAAGAcAgwAADBOow4wS5Ys0S233KLExET169dPO3bsaOiSAABAI9BoA8w777yjvLw8PfPMM/riiy/Us2dPZWdn68iRIw1dGgAAaGCNNsC8+OKLmjBhgh599FFlZmZq2bJlSkpK0htvvNHQpQEAgAbWKB+jrq6uVnl5uWbNmhVY16xZMw0ZMkRlZWWXfI3P55PP5wssnzhxQpJ07Ngx+f3+qNbX/PzpqM7XGDSvtXTmTK2a+5upprZhP4fDRPQvMvQvfPQuMvQvfEePHpXf79eZM2d09OhR2Wy2qMx78uRJSZJlWVcc1ygDzF/+8hfV1NQoJSUlaH1KSoq++uqrS76moKBAc+fOvWh9RkZGTGq8Hj3U0AUYjv5Fhv6Fj95Fhv6F56aFsZ3/5MmTatWq1WW3N8oAE45Zs2YpLy8vsFxbW6tjx46pbdu2iosjVV+N1+tVenq6vvvuOzkcjoYuxzj0LzL0L3z0LjL0LzKx6J9lWTp58qTS0tKuOK5RBpibbrpJ8fHxqqqqClpfVVUlp9N5ydfY7XbZ7fagda1bt45Vidcth8PBL3EE6F9k6F/46F1k6F9kot2/K515qdMob+JNSEhQnz59VFJSElhXW1urkpISuVyuBqwMAAA0Bo3yDIwk5eXlady4cbrzzjvVt29fvfTSSzp9+rQeffTRhi4NAAA0sEYbYB588EH97//+r+bMmSOPx6NevXppw4YNF93Yi+iw2+165plnLroMh/qhf5Ghf+Gjd5Ghf5FpyP7FWVd7TgkAAKCRaZT3wAAAAFwJAQYAABiHAAMAAIxDgAEAAMYhwDQhS5Ys0S233KLExET169dPO3bsuOzYwsJCxcXFBf0kJiZew2obn1D6J0nHjx9Xbm6uUlNTZbfbddttt2n9+vXXqNrGJ5T+DRw48KL3X1xcnHJycq5hxY1HqO+9l156SZ07d1aLFi2Unp6uadOm6dy5c9eo2sYnlP75/X7NmzdPHTt2VGJionr27KkNGzZcw2objy1btmjEiBFKS0tTXFycPvjgg6u+prS0VHfccYfsdrs6deqkwsLC2BVooUl4++23rYSEBOuNN96wKisrrQkTJlitW7e2qqqqLjl+xYoVlsPhsA4fPhz48Xg817jqxiPU/vl8PuvOO++0hg8fbm3dutU6ePCgVVpaalVUVFzjyhuHUPt39OjRoPfenj17rPj4eGvFihXXtvBGINTerVy50rLb7dbKlSutgwcPWkVFRVZqaqo1bdq0a1x54xBq/2bMmGGlpaVZ69atsw4cOGC9+uqrVmJiovXFF19c48ob3vr1662nnnrKWrNmjSXJev/99684/ptvvrGSkpKsvLw8a+/evdbLL79sxcfHWxs2bIhJfQSYJqJv375Wbm5uYLmmpsZKS0uzCgoKLjl+xYoVVqtWra5RdY1fqP1bunSpdeutt1rV1dXXqsRGLdT+/a1FixZZLVu2tE6dOhWrEhutUHuXm5trDRo0KGhdXl6eddddd8W0zsYq1P6lpqZar7zyStC6Bx54wHr44YdjWmdjV58AM2PGDOv2228PWvfggw9a2dnZMamJS0hNQHV1tcrLyzVkyJDAumbNmmnIkCEqKyu77OtOnTqlDh06KD09XSNHjlRlZeW1KLfRCad/H374oVwul3Jzc5WSkqJu3brp+eefV01NzbUqu9EI9/13oeXLl2v06NG64YYbYlVmoxRO737+85+rvLw8cJnkm2++0fr16zV8+PBrUnNjEk7/fD7fRZfLW7Rooa1bt8a01utBWVlZUK8lKTs7u96/56EiwDQBf/nLX1RTU3PRpxinpKTI4/Fc8jWdO3fWG2+8oT/96U966623VFtbq5///Of6/vvvr0XJjUo4/fvmm2/03nvvqaamRuvXr9fTTz+thQsX6tlnn70WJTcq4fTvQjt27NCePXv0+OOPx6rERiuc3j300EOaN2+e7r77btlsNnXs2FEDBw7Ur3/962tRcqMSTv+ys7P14osv6uuvv1Ztba3cbrfWrFmjw4cPX4uSjebxeC7Za6/Xq7Nnz0Z9fwQYXJLL5dLYsWPVq1cv3XPPPVqzZo1uvvlm/f73v2/o0oxQW1ur5ORkvfbaa+rTp48efPBBPfXUU1q2bFlDl2ac5cuXq3v37urbt29Dl2KE0tJSPf/883r11Vf1xRdfaM2aNVq3bp3mz5/f0KUZYfHixfrpT3+qLl26KCEhQVOmTNGjjz6qZs34c9nYNNrvQkL03HTTTYqPj1dVVVXQ+qqqKjmdznrNYbPZ1Lt3b+3fvz8WJTZq4fQvNTVVNptN8fHxgXVdu3aVx+NRdXW1EhISYlpzYxLJ++/06dN6++23NW/evFiW2GiF07unn35aY8aMCZyx6t69u06fPq2JEyfqqaeealJ/iMPp380336wPPvhA586d09GjR5WWlqYnn3xSt95667Uo2WhOp/OSvXY4HGrRokXU99d03slNWEJCgvr06aOSkpLAutraWpWUlMjlctVrjpqaGu3evVupqamxKrPRCqd/d911l/bv36/a2trAuv/6r/9SampqkwovUmTvv9WrV8vn8+mRRx6JdZmNUji9O3PmzEUhpS5IW03sq+8iee8lJibq7/7u73T+/Hn98Y9/1MiRI2NdrvFcLldQryXJ7XbX++9MyGJyazAanbffftuy2+1WYWGhtXfvXmvixIlW69atA49GjxkzxnryyScD4+fOnWsVFRVZBw4csMrLy63Ro0dbiYmJVmVlZUMdQoMKtX+HDh2yWrZsaU2ZMsXat2+ftXbtWis5Odl69tlnG+oQGlSo/atz9913Ww8++OC1LrdRCbV3zzzzjNWyZUvrD3/4g/XNN99YxcXFVseOHa1/+qd/aqhDaFCh9m/79u3WH//4R+vAgQPWli1brEGDBlkZGRnWjz/+2EBH0HBOnjxp7dy509q5c6clyXrxxRetnTt3Wv/93/9tWZZlPfnkk9aYMWMC4+seo54+fbr15ZdfWkuWLOExakTHyy+/bLVv395KSEiw+vbta23fvj2w7Z577rHGjRsXWJ46dWpgbEpKijV8+PAm+TkIFwqlf5ZlWdu2bbP69etn2e1269Zbb7Wee+456/z589e46sYj1P599dVXliSruLj4Glfa+ITSO7/fb+Xn51sdO3a0EhMTrfT0dOtf/uVfmuQf4Dqh9K+0tNTq2rWrZbfbrbZt21pjxoyx/ud//qcBqm54//Ef/2FJuuinrl/jxo2z7rnnnote06tXLyshIcG69dZbY/rZTXGW1cTOKQIAAONxDwwAADAOAQYAABiHAAMAAIxDgAEAAMYhwAAAAOMQYAAAgHEIMAAAwDgEGAAAYBwCDAAAMA4BBgAAGIcAAwAAjEOAAQAAxvk/ncnktVlDoQIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pairwise_scores = kb.document_top_pairwise_scores(n=10_000)\n",
    "\n",
    "scores = pd.Series([score for score, _, _ in pairwise_scores])\n",
    "\n",
    "scores.hist(bins=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "26212ec4-e66d-4108-ab38-21100eef254e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "!!! 0.9982 !!!\n",
      "> I love telling Dad jokes. Sometimes he even laughs.\n",
      "> I love telling Dad jokes. Sometimes, he even laughs.\n",
      "\n",
      "!!! 0.9976 !!!\n",
      "> Did you hear about the guy who invented the knock-knock joke? He won the 'no-bell' prize.'\n",
      "> Did you hear about the guy who invented the knock-knock joke? He won the 'no-bell' prize.\n",
      "\n",
      "!!! 0.9970 !!!\n",
      "> A cop started crying while he was writing me a ticket. I asked him why and he said, \"It's a moving violation.\"\n",
      "> A cop started crying while he was writing me a ticket. I asked him why and he said, 'It's a moving violation.'\n",
      "\n",
      "!!! 0.9970 !!!\n",
      "> My ex and I had a very amicable divorce. I know this because when I posted on Facebook, 'I’m getting a divorce,' she was the first one to like it.\n",
      "> My ex and I had a very amicable divorce. I know this because when I posted on Facebook, “I’m getting a divorce,” she was the first one to like it.\n",
      "\n",
      "!!! 0.9967 !!!\n",
      "> My wife asked me the other day where I got so much candy. I said, 'I always have a few Twix up my sleeve.'\n",
      "> My wife asked me the other day where I got so much candy. I said, \"I always have a few Twix up my sleeve.\"\n",
      "\n",
      "!!! 0.9964 !!!\n",
      "> A panic-stricken man explained to his doctor, “You have to help me, I think I’m shrinking.” “Now settle down,” the doctor calmly told him. “You'll just have to learn to be a little patient.”\n",
      "> A panic-stricken man explained to his doctor, 'You have to help me, I think I’m shrinking.' 'Now settle down,' the doctor calmly told him. 'You'll just have to learn to be a little patient.'\n",
      "\n",
      "!!! 0.9959 !!!\n",
      "> My wife told me she didn't understand cloning. I told her, 'That makes two of us.'\n",
      "> My wife told me she didn't understand cloning. I told her, \"That makes two of us.\"\n",
      "\n",
      "!!! 0.9957 !!!\n",
      "> A man walks into a magic forest and tries to cut down a talking tree. “You can't cut me down,” the tree complains. “I’m a talking tree!” The man responds, “You may be a talking tree, but you will dialogue.”\n",
      "> A man walks into a magic forest and tries to cut down a talking tree. 'You can't cut me down,' the tree complains. 'I’m a talking tree!' The man responds, 'You may be a talking tree, but you will dialogue.'\n",
      "\n",
      "!!! 0.9956 !!!\n",
      "> I built a model of Mount Everest and my son asked if it was to scale. \"No,\" I said. \"It's to look at.\"\n",
      "> I built a model of Mount Everest and my son asked if it was to scale. 'No,' I said. 'It's to look at.'\n",
      "\n",
      "!!! 0.9955 !!!\n",
      "> A cop started crying while he was writing me a ticket. I asked him why and he said, 'It's a moving violation.'\n",
      "> A cop started crying while he was writing me a ticket. I asked him why and he said, It's a moving violation.\n",
      "\n",
      "!!! 0.9953 !!!\n",
      "> A cop started crying while he was writing me a ticket. I asked him why and he said, \"It's a moving violation.\"\n",
      "> A cop started crying while he was writing me a ticket. I asked him why and he said, It's a moving violation.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for score, doc_1, doc_2 in pairwise_scores:\n",
    "    if score < 0.9952:\n",
    "        break\n",
    "    print(f'!!! {score:.4f} !!!')\n",
    "    print(f\"> {doc_1['text']}\")\n",
    "    print(f\"> {doc_2['text']}\")\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d70eb14b-3825-44de-9730-0ebca38aa0ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "!!! 0.7995 !!!\n",
      "> I decided to remove my racing snail's shell to make him go faster. But if anything, it just made him more sluggish.\n",
      "> I pulled the shell off of my snail to make him faster Turned out it had the opposite effect, now he's a little sluggish.\n",
      "\n",
      "!!! 0.7994 !!!\n",
      "> What’s the difference between an African elephant and an Indian elephant? About 5,000 miles.\n",
      "> What's the difference between an Indian and African elephant? For starters, one of them is an elephant.\n",
      "\n",
      "!!! 0.7994 !!!\n",
      "> How do you make a Kleenex dance? Put a little boogie in it!'\n",
      "> How do you make a tissue dance  You put a little boogie in it.\n",
      "\n",
      "!!! 0.7993 !!!\n",
      "> It was a baby mosquito's first day to fly out from home. When the mosquito came back home later that day, the father mosquito asked, \"How was your journey?\"The baby mosquito replied, \"It went great, everyone was clapping for me!\"\n",
      "> A young mosquito returned to its mother. How was your flight dear? asked mom.It was great mom, everyone clapped for me!\n",
      "\n",
      "!!! 0.7993 !!!\n",
      "> Which U.S. state is famous for its extra-small soft drinks? Minnesota!\n",
      "> Which state gives you the smallest beverages? Minnesota.\n",
      "\n",
      "!!! 0.7991 !!!\n",
      "> I was reading a book on anti gravity last night. I found it quite difficult to put down.\n",
      "> I'm reading a book about anti-gravity. It's impossible to put down!  It's impossible to put down!\n",
      "\n",
      "!!! 0.7986 !!!\n",
      "> Where do you learn to make ice cream? Sundae school.\n",
      "> Where do you learn to make a banana split?' 'Sundae school.'\n",
      "\n",
      "!!! 0.7981 !!!\n",
      "> I decided to sell my vacuum cleaner—it was just gathering dust!'\n",
      "> Why did the man decide to sell his vacuum? It was just collecting dust.\n",
      "\n",
      "!!! 0.7976 !!!\n",
      "> What’s a computer’s favorite snack? Microchips!\n",
      "> What's a robot's favorite snack? Computer chips.\n",
      "\n",
      "!!! 0.7975 !!!\n",
      "> What happens when you don’t pay your exorcist? You get repossessed.\n",
      "> What happened to the couple who didn't pay their exorcist? Their house got repossessed.\n",
      "\n",
      "!!! 0.7970 !!!\n",
      "> My boss asked me why I only get sick on work days. I said it must be my weekend immune system.\n",
      "> I only seem to get sick on weekdays. I must have a weekend immune system.\n",
      "\n",
      "!!! 0.7970 !!!\n",
      "> My toddler is refusing to nap. He’s guilty of resisting a rest.\n",
      "> If a child refuses to nap, are they guilty of resisting a rest?\n",
      "\n",
      "!!! 0.7963 !!!\n",
      "> Vladimir Putin Travels to an Eastern European Country He walks up to the customs agent and the agent asks, “Name?”“Vladimir Putin”“Country of Origin?”“Russia”“Occupation?”“No, no. Just visiting.”\n",
      "> Putin lands in a foreign country and approaches the immigration desk The border official reads through his passport and asks: \"Occupation?\"Putin: \"No, just visiting.\"\n",
      "\n",
      "!!! 0.7960 !!!\n",
      "> I pulled the shell off of my snail to make him faster Turned out it had the opposite effect, now he's a little sluggish.\n",
      "> I wanted to help my pet snail. He was really slow like, so one day I took off his shell, thought it'd make him more streamlined. Turns out it didn't. It made him more sluggish.\n",
      "\n",
      "!!! 0.7953 !!!\n",
      "> My toddler is refusing to nap. He's guilty of resisting a rest.\n",
      "> What was the child who wouldn't nap guilty of? Resisting a rest!\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for score, doc_1, doc_2 in pairwise_scores:\n",
    "    if score > 0.8:\n",
    "        continue\n",
    "    if score < 0.795:\n",
    "        break\n",
    "    print(f'!!! {score:.4f} !!!')\n",
    "    print(f\"> {doc_1['text']}\")\n",
    "    print(f\"> {doc_2['text']}\")\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "e1f843db-691e-4ea3-942b-1a159ffc0daa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "!!! 0.5999 !!!\n",
      "> Did you know turtles have the ability to understand puns? I wish they would have tortoise that in school.\n",
      "> One of my best teachers in high school was a turtle. I remember everything he tortoise.\n",
      "\n",
      "!!! 0.5999 !!!\n",
      "> I went to the shop to buy a foot pump for our new air bed. I was shocked by how much the price had risen since the last pump I purchased. But yer, I suppose that’s the cost of inflation.\n",
      "> It used to be free to fill your tires with air now it costs $1.50! Now that's what you call inflation!\n",
      "\n",
      "!!! 0.5999 !!!\n",
      "> Why can you never hear a pterodactyl going to the bathroom? Because the pee is silent.\n",
      "> Why can't you hear a psychiatrist using the bathroom? Because the 'P' is silent.\n",
      "\n",
      "!!! 0.5999 !!!\n",
      "> Whenever we drive past a graveyard my dad says, 'Do you know why I can't be buried there?' And we all say, 'Why not?' And he says, 'Because I'm not dead yet!'\"\n",
      "> When I was a kid my grandpa asked me when we drove past a cemetery “do you know how many people are dead in that cemetery?”, of course i said nope. Then he said All of them!\n",
      "\n",
      "!!! 0.5998 !!!\n",
      "> What breed of dog can jump higher than buildings? Any dog, because buildings can’t jump.\n",
      "> Can a kangaroo jump higher than a house? Of course. Houses can’t jump\n",
      "\n",
      "!!! 0.5998 !!!\n",
      "> If I die young, I want my girlfriend to cast my ashes headwind Because she never let me come on her face before.\n",
      "> When I die, I want to be cremated. It's my last chance to have a smokin' hot body.\n",
      "\n",
      "!!! 0.5997 !!!\n",
      "> A very long time ago, the ancients struggled to come up with a term for a 24 hour period. It got very late and they said \"lets just call it a day\".\n",
      "> God thought long and hard what to name the period of time when the sun was not visible... Finally, after many hours of trying different sounds and variations he named it night. But when naming the opposite, he was exhaust and called it a day.\n",
      "\n",
      "!!! 0.5996 !!!\n",
      "> Why can't you trust an atom? Because they make up literally everything.\n",
      "> A wise man once said:\"never trust atoms!\" \"They make up everything\"I'll see my way out\n",
      "\n",
      "!!! 0.5996 !!!\n",
      "> What do you call a bundle of hay in a church? Christian Bale.\n",
      "> What do you call Batman when he keeps skipping mass? Christian Bale.\n",
      "\n",
      "!!! 0.5996 !!!\n",
      "> How many baby boomers does it take to change a light bulb? None. They just hire someone to do it and complain how back then a bulb used to cost a nickel\n",
      "> How many Germans does it take to change a lightbulb? Just the one, because they are very efficient and they don't have a sense of humour.\n",
      "\n",
      "!!! 0.5994 !!!\n",
      "> Why are bakers so rich? They make so much dough.\n",
      "> I am the breadwinner of the family I make the most dough\n",
      "\n",
      "!!! 0.5993 !!!\n",
      "> How many mystery writers does it take to change a lightbulb? Two. One to screw it in most of the way and another to give it a surprise twist at the end.\n",
      "> How many actors does it take to change a light bulb? only one cause they don't like to share the spot light\n",
      "\n",
      "!!! 0.5992 !!!\n",
      "> What ruined Tiger Woods’ golf career? His driving game.\n",
      "> Tiger Woods has really picked up his game He used to only flip his golf club\n",
      "\n",
      "!!! 0.5992 !!!\n",
      "> I'm reading a book about anti-gravity. It's impossible to put down!'\n",
      "> I was reading a great book about an immortal dog the other day. It was Impossible to put down.\n",
      "\n",
      "!!! 0.5991 !!!\n",
      "> Where do armies belong? In your sleeves.\n",
      "> Where did Napoleon keep his armies? Up his sleevies!\n",
      "\n",
      "!!! 0.5990 !!!\n",
      "> What reward does a light rain get? A precipitation trophy\n",
      "> My son lost a youth baseball game because of heavy rain... ...he received a precipitation trophy\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for score, doc_1, doc_2 in pairwise_scores:\n",
    "    if score > 0.6:\n",
    "        continue\n",
    "    if score < 0.599:\n",
    "        break\n",
    "    print(f'!!! {score:.4f} !!!')\n",
    "    print(f\"> {doc_1['text']}\")\n",
    "    print(f\"> {doc_2['text']}\")\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "a7bb028c-a915-4813-8023-a6b3d6007418",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-17 02:28:46,329 - svs.kb - INFO - starting bulk-delete (as new database transaction)\n",
      "2024-07-17 02:28:46,497 - svs.kb - INFO - invalidating cached vectors; they'll be re-built next time you `retrieve()`\n",
      "2024-07-17 02:28:46,501 - svs.kb - INFO - ending bulk-delete (committing the database transaction)\n"
     ]
    }
   ],
   "source": [
    "# Um, this is actually graph problem.\n",
    "#\n",
    "# The correct solution would be to:\n",
    "#   1. Build a graph from the edges above.\n",
    "#   2. Compute the \"connected components\" of that graph.\n",
    "#   3. For each connected component, pick one node (i.e. a joke) to keep, then delete the rest!\n",
    "#\n",
    "# However, the below is a dumb \"good enough\" solution, since I'd rather not add *yet more* complexity to this notebook.\n",
    "\n",
    "with kb.bulk_del_docs() as del_doc:\n",
    "    for score, doc_1, doc_2 in pairwise_scores:\n",
    "        if score < 0.9:\n",
    "            break\n",
    "        try:\n",
    "            del_doc(doc_2['id'])\n",
    "        except KeyError:\n",
    "            pass  # we must have already deleted this document in a previous iteration (possible if 3-or-more jokes are all similar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "35fc3b2c-2c10-4627-a7c9-1e53c9a50361",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-17 02:28:47,866 - svs.kb - INFO - invalidating cached vectors; they'll be re-built next time you `retrieve()`\n"
     ]
    }
   ],
   "source": [
    "# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n",
    "# !!! Since we deleted records from the database, it's a good practice to vacuum it. !!!\n",
    "# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n",
    "\n",
    "kb.close(vacuum=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fac205b0-512e-45df-8019-83abd33743b9",
   "metadata": {},
   "source": [
    "# Step 4: Final Sanity Check\n",
    "\n",
    "Let's just make sure it's all still working correctly by querying some jokes!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "9b9c8fcb-eb3e-472e-8421-f23763427f54",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4461"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kb = svs.KB(DB_FILE_PATH)\n",
    "len(kb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "df63adff-733e-4b76-bfb4-5f17b4a14d32",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-17 02:28:47,926 - svs.kb - INFO - retrieving 5 documents with query string: pizza\n",
      "2024-07-17 02:28:47,929 - svs.kb - INFO - re-building cached vectors...\n",
      "2024-07-17 02:28:48,399 - svs.kb - INFO - re-building cached vectors... DONE!\n",
      "2024-07-17 02:28:48,954 - svs.kb - INFO - got embedding for query!\n",
      "2024-07-17 02:28:48,960 - svs.kb - INFO - computed 4461 cosine similarities\n",
      "2024-07-17 02:28:48,970 - svs.kb - INFO - retrieved top 5 documents\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 487 ms, sys: 43 ms, total: 530 ms\n",
      "Wall time: 1.05 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'score': 0.4538998603820801,\n",
       "  'doc': {'id': 2518,\n",
       "   'parent_id': None,\n",
       "   'level': 0,\n",
       "   'text': 'What do you call a couple of ants sharing a slice of pizza in Italy? Romance',\n",
       "   'embedding': True,\n",
       "   'meta': None}},\n",
       " {'score': 0.42194458842277527,\n",
       "  'doc': {'id': 9090,\n",
       "   'parent_id': None,\n",
       "   'level': 0,\n",
       "   'text': 'How do you fix a broken pizza? With tomato paste.',\n",
       "   'embedding': True,\n",
       "   'meta': None}},\n",
       " {'score': 0.4053124785423279,\n",
       "  'doc': {'id': 8676,\n",
       "   'parent_id': None,\n",
       "   'level': 0,\n",
       "   'text': 'I ate a salad for dinner! It was mostly tomatoes and croutons.Really just one, big round crouton covered in tomato sauce. And cheese.... I had a pizza.',\n",
       "   'embedding': True,\n",
       "   'meta': None}},\n",
       " {'score': 0.3984290361404419,\n",
       "  'doc': {'id': 4064,\n",
       "   'parent_id': None,\n",
       "   'level': 0,\n",
       "   'text': 'My sister goes to the pizza place The pizza guy asks: \"would you want me to cut your pizza in 4 or 12 pieces.She said: Please only 4, I can\\'t eat 12 pieces all alone.',\n",
       "   'embedding': True,\n",
       "   'meta': None}},\n",
       " {'score': 0.3854081630706787,\n",
       "  'doc': {'id': 2988,\n",
       "   'parent_id': None,\n",
       "   'level': 0,\n",
       "   'text': \"What is Ravioli's favourite party game? Pasta parcel\",\n",
       "   'embedding': True,\n",
       "   'meta': None}}]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "kb.retrieve('pizza', n=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "1ee8ae31-83f7-4450-badd-b402b30a8395",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-17 02:28:48,986 - svs.kb - INFO - invalidating cached vectors; they'll be re-built next time you `retrieve()`\n"
     ]
    }
   ],
   "source": [
    "kb.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d9e568d3-7a41-4dd5-a376-6ce42c3e39e5",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}