cookbooks/python/mistralai/prompting_capabilities.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5d7f6058-d0d1-44af-b43a-cb4b06df03d8",
   "metadata": {},
   "source": [
    "# Prompting Capabilities \n",
    "\n",
    "When you first start using Mistral models, your first interaction will revolve around prompts. The art of crafting effective prompts is essential for generating desirable responses from Mistral models or other LLMs. This guide will walk you through example prompts showing four different prompting capabilities. \n",
    "\n",
    "- Classification \n",
    "- Summarization \n",
    "- Personalization\n",
    "- Evaluation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "daa37d97-10e4-425d-b852-15bd043662b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install mistralai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae93e75a-fd20-4ca6-9eff-bef7d44ab3a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mistralai.client import MistralClient\n",
    "from mistralai.models.chat_completion import ChatMessage\n",
    "import os, dotenv\n",
    "\n",
    "dotenv.load_dotenv()\n",
    "\n",
    "assert \"GITHUB_TOKEN\" in os.environ, \"Please set the GITHUB_TOKEN environment variable.\"\n",
    "github_token = os.environ[\"GITHUB_TOKEN\"]\n",
    "\n",
    "# We can use some defaults for the other two variables\n",
    "endpoint = \"https://models.inference.ai.azure.com\"\n",
    "model_name = \"mistral-large\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e030d9c2-1ecb-4bf0-864d-9f96b41bd016",
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_mistral(user_message, model=model_name):\n",
    "    client = MistralClient(api_key=github_token, endpoint=endpoint)\n",
    "    messages = [\n",
    "        ChatMessage(role=\"user\", content=user_message)\n",
    "    ]\n",
    "    chat_response = client.chat(\n",
    "        model=model,\n",
    "        messages=messages\n",
    "    )\n",
    "    return (chat_response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b98a29ef-4764-40b7-aed8-0e5d0502f985",
   "metadata": {},
   "source": [
    "## Classification\n",
    "Mistral models can easily categorize text into distinct classes. In this example prompt, we can define a list of predefined categories and ask Mistral models to classify user inquiry. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d71e9d1c-ca45-4d19-882c-07e077ea19ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "def user_message(inquiry):\n",
    "    user_message = (\n",
    "        f\"\"\"\n",
    "        You are a bank customer service bot. Your task is to assess customer intent \n",
    "        and categorize customer inquiry after <<<>>> into one of the following predefined categories:\n",
    "        \n",
    "        card arrival\n",
    "        change pin\n",
    "        exchange rate\n",
    "        country support \n",
    "        cancel transfer\n",
    "        charge dispute\n",
    "        \n",
    "        If the text doesn't fit into any of the above categories, classify it as:\n",
    "        customer service\n",
    "        \n",
    "        You will only respond with the predefined category. Do not include the word \"Category\". Do not provide explanations or notes. \n",
    "        \n",
    "        ####\n",
    "        Here are some examples:\n",
    "        \n",
    "        Inquiry: How do I know if I will get my card, or if it is lost? I am concerned about the delivery process and would like to ensure that I will receive my card as expected. Could you please provide information about the tracking process for my card, or confirm if there are any indicators to identify if the card has been lost during delivery?\n",
    "        Category: card arrival\n",
    "        Inquiry: I am planning an international trip to Paris and would like to inquire about the current exchange rates for Euros as well as any associated fees for foreign transactions.\n",
    "        Category: exchange rate \n",
    "        Inquiry: What countries are getting support? I will be traveling and living abroad for an extended period of time, specifically in France and Germany, and would appreciate any information regarding compatibility and functionality in these regions.\n",
    "        Category: country support\n",
    "        Inquiry: Can I get help starting my computer? I am having difficulty starting my computer, and would appreciate your expertise in helping me troubleshoot the issue. \n",
    "        Category: customer service\n",
    "        ###\n",
    "    \n",
    "        <<<\n",
    "        Inquiry: {inquiry}\n",
    "        >>>\n",
    "        \"\"\"\n",
    "    )\n",
    "    return user_message"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8863460a-7670-4b6f-b511-cf1df7693cfa",
   "metadata": {},
   "source": [
    "### Strategies we used: \n",
    "\n",
    "- **Few shot learning**: Few-shot learning or in-context learning is when we give a few examples in the prompts, and the LLM can generate corresponding output based on the example demonstrations. Few-shot learning can often improve model performance especially when the task is difficult or when we want the model to respond in a specific manner. \n",
    "- **Delimiter**: Delimiters like ### <<< >>> specify the boundary between different sections of the text. In our example, we used ### to indicate examples and <<<>>> to indicate customer inquiry. \n",
    "- **Role playing**: Providing LLM a role (e.g., \"You are a bank customer service bot.\") adds personal context to the model and often leads to better performance. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d8a83cc-31e6-4d4b-b252-93d9133ecbf5",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(run_mistral(user_message(\n",
    "    \"I am inquiring about the availability of your cards in the EU, as I am a resident of France and am interested in using your cards. \"\n",
    ")))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6eca06b-7b1d-4663-9a68-2a23116f8c6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(run_mistral(user_message(\"What's the weather today?\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f351f00-683a-4244-974f-5c719812141f",
   "metadata": {},
   "source": [
    "## Summarization\n",
    "\n",
    "Summarization is a common task for LLMs due to their natural language understanding and generation capabilities. Here is an example prompt we can use to generate interesting questions about an essay and summarize the essay.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bbab492-1d18-4832-86a9-2fba645e0e52",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "response = requests.get('https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt')\n",
    "# need to limit the size to meet the token limit of the GitHub Models free service\n",
    "essay = response.text[:20000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eaede63d-7392-4f1c-8a87-507ee31fe246",
   "metadata": {},
   "outputs": [],
   "source": [
    "message = f\"\"\"\n",
    "You are a commentator. Your task is to write a report on an essay. \n",
    "When presented with the essay, come up with interesting questions to ask, and answer each question. \n",
    "Afterward, combine all the information and write a report in the markdown format. \n",
    "\n",
    "# Essay: \n",
    "{essay}\n",
    "\n",
    "# Instructions: \n",
    "## Summarize:\n",
    "In clear and concise language, summarize the key points and themes presented in the essay.\n",
    "\n",
    "## Interesting Questions: \n",
    "Generate three distinct and thought-provoking questions that can be asked about the content of the essay. For each question:\n",
    "- After \"Q: \", describe the problem \n",
    "- After \"A: \", provide a detailed explanation of the problem addressed in the question.\n",
    "- Enclose the ultimate answer in <>.\n",
    "\n",
    "## Write a report \n",
    "Using the essay summary and the answers to the interesting questions, create a comprehensive report in Markdown format. \n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5505b0a5-411b-4804-aaef-ccecfa3d07be",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(run_mistral(message))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3c1535e-1156-46bc-8e28-5f71846dbe65",
   "metadata": {},
   "source": [
    "## Strategies we used: \n",
    "\n",
    "- **Step-by-step instructions**: This strategy is inspired by the chain-of-thought prompting that enables LLMs to use a series of intermediate reasoning steps to tackle complex tasks. It's often easier to solve complex problems when we decompose them into simpler and small steps and it's easier for us to debug and inspect the model behavior.  In our example, we break down the task into three steps: summarize, generate interesting questions, and write a report. This helps the language to think in each step and generate a more comprehensive final report. \n",
    "- **Example generation**: We can ask LLMs to automatically guide the reasoning and understanding process by generating examples with the explanations and steps. In this example, we ask the LLM to generate three questions and provide detailed explanations for each question. \n",
    "- **Output formatting**: We can ask LLMs to output in a certain format by directly asking \"write a report in the Markdown format\".\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1a9cb95-bb08-4929-b16c-eb19877f3c01",
   "metadata": {},
   "source": [
    "## Personlization\n",
    "\n",
    "LLMs excel at personalization tasks as they can deliver content that aligns closely with individual users. In this example, we create personalized email responses to address customer questions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9cf048b4-3e33-4753-af97-25b73c51ee6a",
   "metadata": {},
   "outputs": [],
   "source": [
    "email = \"\"\"\n",
    "Dear mortgage lender, \n",
    "\n",
    "What's your 30-year fixed-rate APR, how is it compared to the 15-year fixed rate?\n",
    "\n",
    "Regards,\n",
    "Anna\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36de7c1e-60c2-4f35-a51a-115b12d65bb6",
   "metadata": {},
   "outputs": [],
   "source": [
    "message = f\"\"\"\n",
    "\n",
    "You are a mortgage lender customer service bot, and your task is to create personalized email responses to address customer questions.\n",
    "Answer the customer's inquiry using the provided facts below. Ensure that your response is clear, concise, and \n",
    "directly addresses the customer's question. Address the customer in a friendly and professional manner. Sign the email with \n",
    "\"Lender Customer Support.\"   \n",
    "\n",
    "\n",
    "      \n",
    "# Facts\n",
    "30-year fixed-rate: interest rate 6.403%, APR 6.484%\n",
    "20-year fixed-rate: interest rate 6.329%, APR 6.429%\n",
    "15-year fixed-rate: interest rate 5.705%, APR 5.848%\n",
    "10-year fixed-rate: interest rate 5.500%, APR 5.720%\n",
    "7-year ARM: interest rate 7.011%, APR 7.660%\n",
    "5-year ARM: interest rate 6.880%, APR 7.754%\n",
    "3-year ARM: interest rate 6.125%, APR 7.204%\n",
    "30-year fixed-rate FHA: interest rate 5.527%, APR 6.316%\n",
    "30-year fixed-rate VA: interest rate 5.684%, APR 6.062%\n",
    "\n",
    "# Email\n",
    "{email}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aaf50774-0f91-4e0e-86c9-1525f6045ebb",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(run_mistral(message))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0af53ac9-8ef1-4840-8b23-779e1d59204d",
   "metadata": {},
   "source": [
    "### Strategies we used: \n",
    "- Providing facts: Incorporating facts into prompts can be useful for developing customer support bots. It’s important to use clear and concise language when presenting these facts. This can help the LLM to provide accurate and quick responses to customer queries.\n",
    "\n",
    "## Evaluation\n",
    "\n",
    "There are many ways to evaluate LLM outputs. Here are three approaches for your reference: include a confidence score, introduce an evaluation step, or employ another LLM for evaluation.\n",
    "\n",
    "\n",
    "\n",
    "## Include a confidence score\n",
    "We can include a confidence score along with the generated output in the prompt. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8bf511a7-1828-465c-92ed-a701e21b23da",
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_mistral(user_message, model=model_name):\n",
    "    client = MistralClient(api_key=github_token, endpoint=endpoint)\n",
    "    messages = [\n",
    "        ChatMessage(role=\"user\", content=user_message)\n",
    "    ]\n",
    "    chat_response = client.chat(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=1\n",
    "    )\n",
    "    return (chat_response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8087cae2-9b3b-407c-b808-31fd765fd6f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "message = f\"\"\"\n",
    "You are a summarization system that can provide summaries with associated confidence scores.\n",
    "In clear and concise language, provide three short summaries of the following essay, along with their confidence scores. \n",
    "You will only respond with a JSON object with the key Summary and Confidence. Do not provide explanations. \n",
    "\n",
    "# Essay: \n",
    "{essay}\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c7bfcee-ff8b-4c14-b9e8-7e00d3d389ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(run_mistral(message))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8649ad18-10c0-4fb0-92cc-052e2ab75342",
   "metadata": {},
   "source": [
    "### Strategies we used: \n",
    "- JSON output: For facilitating downstream tasks, JSON format output is frequently preferred. We can specify in the prompt that “You will only respond with a JSON object with the key Summary and Confidence.” Specifying these keys within the JSON object is beneficial for clarity and consistency.\n",
    "- Higher Temperature: In this example, we increase the temperature score to encourage the model to be more creative and output three generated summaries that are different from each other.  \n",
    "\n",
    "\n",
    "## Introduce an evaluation step \n",
    "We can also add a second step in the prompt for evaluation. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6300a485-43be-4bc4-9088-9e6c2a365b6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "message = f\"\"\"\n",
    "You are given an essay text and need to provide summaries and evaluate them.\n",
    "\n",
    "# Essay: \n",
    "{essay}\n",
    "\n",
    "Step 1: In this step, provide three short summaries of the given essay. Each summary should be clear, concise, and capture the key points of the speech. Aim for around 2-3 sentences for each summary.\n",
    "Step 2: Evaluate the three summaries from Step 1 and rate which one you believe is the best. Explain your choice by pointing out specific reasons such as clarity, completeness, and relevance to the speech content.\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "print(run_mistral(message))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba076c25-9536-4ba1-a730-0a3ce6fe24f0",
   "metadata": {},
   "source": [
    "## Employ another LLM for evaluation\n",
    "In production systems, it is common to employ another LLM for evaluation so that the evaluation step can be separate from the generation step.  \n",
    "\n",
    "- Step 1: use the first LLM to generate three summaries \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5265085-77fd-433a-8d9d-d89ffac76543",
   "metadata": {},
   "outputs": [],
   "source": [
    "message = f\"\"\"\n",
    "Provide three short summaries of the given essay. Each summary should be clear, concise, and capture the key points of the essay.\n",
    "Aim for around 2-3 sentences for each summary.\n",
    "\n",
    "# essay: \n",
    "{essay}\n",
    "\n",
    "\"\"\"\n",
    "summaries = run_mistral(message)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4372a9fe-d596-4f36-86c6-3e3c01c74c38",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(summaries)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58fd5f25-ab9a-4547-83ab-48638517b7ef",
   "metadata": {},
   "source": [
    "- Step 2: use another LLM to rate the generated summaries\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b755adc8-c277-4421-8925-3d70e5ee39e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "message = f\"\"\"\n",
    "You are given an essay and three summaries of the essay. Evaluate the three summaries and rate which one you believe is the best. \n",
    "Explain your choice by pointing out specific reasons such as clarity, completeness, and relevance to the essay content.\n",
    "\n",
    "# Essay: \n",
    "{essay}\n",
    "\n",
    "# Summaries \n",
    "{summaries}\n",
    "\n",
    "\"\"\"\n",
    "print(run_mistral(message))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e96d61a-9dbc-4f8c-9d80-d2260c5f3e61",
   "metadata": {},
   "source": [
    "### Strategies we used: \n",
    "- **LLM chaining**: In this example, we chain two LLMs in a sequence, where the output from the first LLM serves as the input for the second LLM. The method of chaining LLMs can be adapted to suit your specific use cases. For instance, you might choose to employ three LLMs in a chain, where the output of two LLMs is funneled into the third LLM. While LLM chaining offers flexibility, it's important to consider that it may result in additional API calls and potentially increased costs.\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,py:light"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}