basic_rag_pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f24855df-394f-4b79-96b4-fa5c5b385f2f",
   "metadata": {},
   "source": [
    "## Simple RAG Pipeline\n",
    "\n",
    "Download the \"mini-llama-articales.csv\" for try-outs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7a04de30-b80c-4147-a9fd-9454219bbb42",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File downloaded successfully!\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "url = \"https://raw.githubusercontent.com/AlaFalaki/tutorial_notebooks/main/data/mini-llama-articles.csv\"\n",
    "response = requests.get(url)\n",
    "\n",
    "if response.status_code == 200:\n",
    "    with open(\"mini-llama-articles.csv\", \"wb\") as file:\n",
    "        file.write(response.content)\n",
    "    print(\"File downloaded successfully!\")\n",
    "else:\n",
    "    print(f\"Failed to download file. Status code: {response.status_code}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c9a3ffb-bc81-44d1-8edb-8de9fc42013d",
   "metadata": {},
   "source": [
    "Before loading the data, it’s necessary to define a function for dividing the text into segments. Chunking is an important step before augmenting prompts due to the limited context windows of language models, which prevent the use of multiple full-length articles as context. It also enables providing only relevant information to the model, improving accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "528cd213-4386-415e-bb8a-54efd0947697",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of articles: 14\n",
      "number of chunks: 174\n"
     ]
    }
   ],
   "source": [
    "import csv \n",
    "# Split the input text into chunks of specified size.\n",
    "def split_into_chunks(text, chunk_size=1024):\n",
    "  chunks = []\n",
    "  for i in range(0, len(text), chunk_size):\n",
    "    chunks.append(text[i:i+chunk_size])\n",
    "\n",
    "  return chunks\n",
    "\n",
    "chunks = []\n",
    "# Load the file as a CSV\n",
    "with open(\"./mini-llama-articles.csv\", mode=\"r\", encoding=\"utf-8\") as file:\n",
    "  csv_reader = csv.reader(file)\n",
    "  for idx, row in enumerate(csv_reader):\n",
    "    if idx == 0: continue; # Skip header row\n",
    "    chunks.extend(split_into_chunks(row[1]))\n",
    "\n",
    "print(\"number of articles:\", idx)\n",
    "print(\"number of chunks:\", len(chunks))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "6910b6a3-bd07-4722-9588-16914c17e911",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"LLM Variants and Meta's Open Source Before shedding light on four major trends, I'd share the latest Meta's Llama 2 and Code Llama. Meta's Llama 2 represents a sophisticated evolution in LLMs. This suite spans models pretrained and fine-tuned across a parameter spectrum of 7 billion to 70 billion. A specialized derivative, Llama 2-Chat, has been engineered explicitly for dialogue-centric applications. Benchmarking revealed Llama 2's superior performance over most extant open-source chat models. Human-centric evaluations, focusing on safety and utility metrics, positioned Llama 2-Chat as a potential contender against proprietary, closed-source counterparts. The development trajectory of Llama 2 emphasized rigorous fine-tuning methodologies. Meta's transparent delineation of these processes aims to catalyze community-driven advancements in LLMs, underscoring a commitment to collaborative and responsible AI development. Code Llama is built on top of Llama 2 and is available in three models: Code Llama, the found\""
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chunks[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "7bfe1f32-729d-4480-b023-54cc2539ac77",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://pub.towardsai.net/beyond-gpt-4-whats-new-cbd61a448eb9#dda8\n",
      "https://pub.towardsai.net/building-a-q-a-bot-over-private-documents-with-openai-and-langchain-be975559c1e8#bead\n",
      "https://pub.towardsai.net/enhancing-e-commerce-product-search-using-llms-30d5a2117f71#e5f3\n",
      "https://pub.towardsai.net/exploring-large-language-models-part-3-ab60ee236950#d193\n",
      "https://pub.towardsai.net/fine-tuning-a-llama-2-7b-model-for-python-code-generation-865453afdf73#bf4e\n",
      "https://pub.towardsai.net/foundation-models-37074a2d70a1#7ebc\n",
      "https://pub.towardsai.net/gptq-quantization-on-a-llama-2-7b-fine-tuned-model-with-huggingface-a7b291fbb871#34d2\n",
      "https://pub.towardsai.net/llama-by-meta-leaked-by-an-anonymous-forum-questions-arises-on-meta-e1216e51db6#9001\n",
      "https://pub.towardsai.net/llama-gpt4all-simplified-local-chatgpt-ab7d28d34923#485a\n",
      "https://pub.towardsai.net/inside-code-llama-meta-ais-entrance-in-the-code-llm-space-9f286d13a48d#c9e0\n",
      "https://pub.towardsai.net/metas-llama-2-revolutionizing-open-source-language-models-for-commercial-use-1492bec112b#148f\n",
      "https://pub.towardsai.net/the-generative-ai-revolution-exploring-the-current-landscape-4b89998fcc5f#7d83\n",
      "https://pub.towardsai.net/building-intuition-on-the-concepts-behind-llms-like-chatgpt-part-1-4cb6654ab67#123f\n",
      "https://pub.towardsai.net/wizardcoder-why-its-the-best-coding-model-out-there-46a089c2833#0f8e\n"
     ]
    }
   ],
   "source": [
    "with open(\"./mini-llama-articles.csv\", mode=\"r\", encoding=\"utf-8\") as file:\n",
    "  csv_reader = csv.reader(file)\n",
    "  for idx, row in enumerate(csv_reader):\n",
    "    if idx == 0: continue;\n",
    "    print(row[2])\n",
    "    exit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3706e438-7187-4803-9617-f5617fb691a7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['chunk'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# Convert the list to a Pandas Dataframe\n",
    "df = pd.DataFrame(chunks, columns=['chunk'])\n",
    "\n",
    "print(df.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f8fed682-54c7-4f78-9119-89a6b6e6f22b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "\n",
    "# Defining a function that converts a text to embedding vector using OpenAI's Ada model.\n",
    "def get_embedding(text):\n",
    "  try:\n",
    "    # Remove newlines\n",
    "    text = text.replace(\"\\n\", \" \")\n",
    "    res = openai.Embedding.create(input=[text], model=\"text-embedding-3-small\")\n",
    "\n",
    "    return res.data[0].embedding\n",
    "\n",
    "  except:\n",
    "        return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "53edf402-6192-4734-85a1-37714a061c67",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>chunk</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LLM Variants and Meta's Open Source Before she...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ational code model;Codel Llama - Python specia...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>erm \"multimodal\" refers to their ability to pr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>es it matter? LLM connections, like the LlamaI...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>understand data in the AI-driven future.  Fro...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>versity. In-breadth Evolving solves this probl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>ns are done, the initial instruction dataset (...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>er, the Prompt should be as follows:  Best Use...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>172</th>\n",
       "      <td>sis, and visualization.Machine Learning Pipeli...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>3 pass@1 on the HumanEval Benchmarks, which is...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>174 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 chunk\n",
       "0    LLM Variants and Meta's Open Source Before she...\n",
       "1    ational code model;Codel Llama - Python specia...\n",
       "2    erm \"multimodal\" refers to their ability to pr...\n",
       "3    es it matter? LLM connections, like the LlamaI...\n",
       "4     understand data in the AI-driven future.  Fro...\n",
       "..                                                 ...\n",
       "169  versity. In-breadth Evolving solves this probl...\n",
       "170  ns are done, the initial instruction dataset (...\n",
       "171  er, the Prompt should be as follows:  Best Use...\n",
       "172  sis, and visualization.Machine Learning Pipeli...\n",
       "173  3 pass@1 on the HumanEval Benchmarks, which is...\n",
       "\n",
       "[174 rows x 1 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "bf2f19df-9ae2-460a-a408-9c51521c1c4b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"LLM Variants and Meta's Open Source Before shedding light on four major trends, I'd share the latest Meta's Llama 2 and Code Llama. Meta's Llama 2 represents a sophisticated evolution in LLMs. This suite spans models pretrained and fine-tuned across a parameter spectrum of 7 billion to 70 billion. A specialized derivative, Llama 2-Chat, has been engineered explicitly for dialogue-centric applications. Benchmarking revealed Llama 2's superior performance over most extant open-source chat models. Human-centric evaluations, focusing on safety and utility metrics, positioned Llama 2-Chat as a potential contender against proprietary, closed-source counterparts. The development trajectory of Llama 2 emphasized rigorous fine-tuning methodologies. Meta's transparent delineation of these processes aims to catalyze community-driven advancements in LLMs, underscoring a commitment to collaborative and responsible AI development. Code Llama is built on top of Llama 2 and is available in three models: Code Llama, the found\""
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['chunk'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0a48a1cf-5bc7-4666-8667-c786b707ca74",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating embeddings...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3af150ab616c4fce84eac4451ea3476d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tqdm.notebook import tqdm\n",
    "import numpy as np\n",
    "\n",
    "# Generate embedding\n",
    "print(\"Generating embeddings...\")\n",
    "embeddings = []\n",
    "for index, row in tqdm(df.iterrows()):\n",
    "  embeddings.append(get_embedding(row['chunk']))\n",
    "\n",
    "# Add the \"embedding\" column to the dataframe\n",
    "embeddings_values = pd.Series(embeddings)\n",
    "df.insert(loc=1, column='embedding', value=embeddings_values)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2538011-c16a-47fe-b6b5-1b03bdb2adf9",
   "metadata": {},
   "source": [
    "__Cosine Similarity Metric:__\n",
    "\n",
    "It measures the cosine of the angle between two vectors in a multi-dimensional space, indicating how closely the vectors are oriented regardless of their size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "386b341c-8314-4ce4-9762-919a5d76e1f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "\n",
    "QUESTION = \"How many parameters LLaMA2 model has?\"\n",
    "QUESTION_emb = get_embedding(QUESTION)\n",
    "\n",
    "# The similarity between the questions and each part of the essay.\n",
    "cosine_similarities = cosine_similarity([QUESTION_emb], df['embedding'].tolist())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "26e4e4cd-e3dc-4c5a-b6ca-b29dc6723281",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[114  75  89]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "number_of_chunks_to_retrieve = 3\n",
    "\n",
    "# Sort and find the index of N highest scored chunks\n",
    "indices = np.argsort(cosine_similarities[0])[::-1][:number_of_chunks_to_retrieve]\n",
    "print(indices)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4db0e2e9-ef0e-49db-a59d-32ad51bbcc54",
   "metadata": {},
   "source": [
    "#### Inference from GenAI of Google"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "098cbe44-1a07-4eea-9730-1ac4b00027a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import google.generativeai as genai\n",
    "# Use the Gemini API to answer the questions based on the retrieved pieces of text.\n",
    "try:\n",
    "    # Formulating the system prompt and condition the model to answer only AI-related questions.\n",
    "    system_prompt = (\n",
    "        \"You are an assistant and expert in answering questions from a chunks of content. \"\n",
    "        \"Only answer AI-related question, else say that you cannot answer this question.\"\n",
    "    )\n",
    "\n",
    "    # Create a user prompt with the user's question\n",
    "    prompt = (\n",
    "        \"Read the following informations that might contain the context you require to answer the question. You can use the informations starting from the <START_OF_CONTEXT> tag and end with the <END_OF_CONTEXT> tag. Here is the content:\\n\\n<START_OF_CONTEXT>\\n{}\\n<END_OF_CONTEXT>\\n\\n\"\n",
    "        \"Please provide an informative and accurate answer to the following question based on the avaiable context. Be concise and take your time. \\nQuestion: {}\\nAnswer:\"\n",
    "    )\n",
    "    # Add the retrieved pieces of text to the prompt.\n",
    "    prompt = prompt.format(\"\".join(df.chunk[indices]), QUESTION)\n",
    "\n",
    "    model = genai.GenerativeModel(model_name= \"gemini-1.5-flash\", system_instruction=system_prompt)\n",
    "    \n",
    "    result = model.generate_content(prompt,request_options={\"timeout\": 1000},)\n",
    "    res = result.text\n",
    "\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f83a9ab-c43e-4c4e-8227-7058e29b9f89",
   "metadata": {},
   "source": [
    "#### Inference from OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d80cff01-6a9c-46e1-b931-85aaff4d7fff",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generated Answer: The LLaMA2 model is available in four different sizes with varying parameters: 7 billion, 13 billion, 34 billion, and 70 billion parameters.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import openai\n",
    "\n",
    "openai.api_key=os.getenv(\"OPENAI_API_KEY2\")\n",
    "\n",
    "try:\n",
    "    # Formulating the system prompt to condition the model\n",
    "    system_prompt = (\n",
    "        \"You are an assistant and expert in answering questions from chunks of content. \"\n",
    "        \"Only answer AI-related questions; otherwise, say that you cannot answer this question.\"\n",
    "    )\n",
    "\n",
    "    # Create a user prompt with the user's question\n",
    "    prompt = (\n",
    "        \"Read the following information that might contain the context you require to answer the question. You can use the information starting from the <START_OF_CONTEXT> tag and end with the <END_OF_CONTEXT> tag. Here is the content:\\n\\n<START_OF_CONTEXT>\\n{}\\n<END_OF_CONTEXT>\\n\\n\"\n",
    "        \"Please provide an informative and accurate answer to the following question based on the available context. Be concise and take your time. \\nQuestion: {}\\nAnswer:\"\n",
    "    )\n",
    "    # Add the retrieved pieces of text to the prompt\n",
    "    prompt = prompt.format(\"\".join(df.chunk[indices]), QUESTION)\n",
    "\n",
    "    # Use OpenAI's chat completion API to generate the answer\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=\"gpt-3.5-turbo\",  \n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": system_prompt},\n",
    "            {\"role\": \"user\", \"content\": prompt}\n",
    "        ],\n",
    "        max_tokens=1000,  \n",
    "        temperature=0.7,  \n",
    "        timeout=1000 \n",
    "    )\n",
    "\n",
    "    # Extract the assistant's response\n",
    "    res = response[\"choices\"][0][\"message\"][\"content\"]\n",
    "    print(\"Generated Answer:\", res)\n",
    "\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred: {e}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4934280b-7c4c-4eac-a886-227c9728ba1f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}