samples/extraction/text-based/document-intelligence-phi.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Extraction - Azure AI Document Intelligence + Phi-4\n",
    "\n",
    "This sample demonstrates how to extract structured data from any document using Azure AI Document Intelligence and small language models, such as Microsoft's Phi-4.\n",
    "\n",
    "![Data Extraction](../../../images/extraction-document-intelligence-phi.png)\n",
    "\n",
    "This is achieved by the following process:\n",
    "\n",
    "- Analyze a document using Azure AI Document Intelligence's `prebuilt-layout` model to extract the structure as Markdown.\n",
    "- Construct a system prompt that defines the instruction for extracting structured data from documents.\n",
    "- Construct a user prompt that includes specific extraction instruction for the type of document, the expected JSON schema, and the Markdown content of the document\n",
    "- Use the chat completions API with the Phi-4 model to generate a structured output from the content.\n",
    "\n",
    "## Objectives\n",
    "\n",
    "By the end of this sample, you will have learned how to:\n",
    "\n",
    "- Convert a document to Markdown format using Azure AI Document Intelligence.\n",
    "- Use prompt engineering techniques to instruct Phi-4 to extract structured data from a type of document using example JSON structures.\n",
    "- Use the analysis result from Azure AI Document Intelligence to determine the confidence of the extracted structured output."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import modules\n",
    "\n",
    "This sample takes advantage of the following Python dependencies:\n",
    "\n",
    "- **azure-ai-documentintelligence** to interface with the Azure AI Document Intelligence API for analyzing documents.\n",
    "- **openai** to use the default OpenAI client implementation to interface with a deployed Phi-4 serverless endpoint in Azure AI Foundry. _Note: The serverless endpoint for Phi-4 uses the same API schema as the OpenAI API._\n",
    "- **azure-identity** to securely authenticate with deployed Azure Services using Microsoft Entra ID credentials.\n",
    "\n",
    "The following local modules are also used:\n",
    "\n",
    "- **modules.app_settings** to access environment variables from the `.env` file.\n",
    "- **modules.comparison** to compare the output of the extraction process with expected results.\n",
    "- **modules.document_intelligence_confidence** to evaluate the confidence of the extraction process based on the extracted structured output and the analysis result from Azure AI Document Intelligence.\n",
    "- **modules.document_processing_result** to store the results of the extraction process as a file.\n",
    "- **modules.invoice** to provide the expected structured output JSON schema for invoice documents.\n",
    "- **modules.utils** `Stopwatch` to measure the end-to-end execution time for the classification process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append('../../') # Import local modules\n",
    "\n",
    "from IPython.display import display, Markdown\n",
    "import os\n",
    "import pandas as pd\n",
    "import json\n",
    "from dotenv import dotenv_values\n",
    "from azure.ai.documentintelligence import DocumentIntelligenceClient\n",
    "from azure.ai.documentintelligence.models import AnalyzeResult, DocumentContentFormat\n",
    "from azure.ai.inference import ChatCompletionsClient\n",
    "from azure.identity import DefaultAzureCredential\n",
    "from azure.core.credentials import AzureKeyCredential\n",
    "import json\n",
    "\n",
    "from modules.app_settings import AppSettings\n",
    "from modules.utils import Stopwatch\n",
    "from modules.accuracy_evaluator import AccuracyEvaluator\n",
    "from modules.comparison import get_extraction_comparison\n",
    "from modules.document_intelligence_confidence import evaluate_confidence as di_evaluate_confidence\n",
    "from modules.invoice import Invoice\n",
    "from modules.document_processing_result import DataExtractionResult"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure the Azure services\n",
    "\n",
    "To use Azure AI Document Intelligence and a serverless model endpoint deployment, their SDKs are used to create client instances using a deployed endpoint and authentication credentials.\n",
    "\n",
    "For this sample, the credentials of the Azure CLI are used to authenticate with Azure AI Document Intelligence. The Azure AI Inference client for the Phi model uses an API key to authenticate with the serverless endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the working directory to the root of the repo\n",
    "working_dir = os.path.abspath('../../../')\n",
    "settings = AppSettings(dotenv_values(f\"{working_dir}/.env\"))\n",
    "\n",
    "# Configure the default credential for accessing Azure services using Azure CLI credentials\n",
    "credential = DefaultAzureCredential(\n",
    "    exclude_workload_identity_credential=True,\n",
    "    exclude_developer_cli_credential=True,\n",
    "    exclude_environment_credential=True,\n",
    "    exclude_managed_identity_credential=True,\n",
    "    exclude_powershell_credential=True,\n",
    "    exclude_shared_token_cache_credential=True,\n",
    "    exclude_interactive_browser_credential=True\n",
    ")\n",
    "\n",
    "# Uses the Azure AI Inference SDK to interact with the Phi model for chat completions.\n",
    "inference_client = ChatCompletionsClient(\n",
    "    endpoint=settings.phi_endpoint,\n",
    "    credential=AzureKeyCredential(settings.phi_primary_key),\n",
    ")\n",
    "\n",
    "document_intelligence_client = DocumentIntelligenceClient(\n",
    "    endpoint=settings.ai_services_endpoint,\n",
    "    credential=credential\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Establish the expected output\n",
    "\n",
    "To compare the accuracy of the extraction process, the expected output of the extraction process has been defined in the following code block based on the details of an [Invoice](../../assets/invoices/invoice_1.pdf).\n",
    "\n",
    "> **Note**: More invoice examples can be found in the [assets folder](../../assets/invoices). These examples include the PDF file and an associated JSON metadata file that provides the expected structured output. You can add your own scenarios by following the same structure.\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"fname\": \"<name of the invoice file>\",\n",
    "    \"expected\": {\n",
    "        \"customer_name\": \"\",\n",
    "        \"customer_address\": {\n",
    "            \"street\": \"\",\n",
    "            \"city\": \"\",\n",
    "            \"state\": \"\",\n",
    "            \"postal_code\": \"\",\n",
    "            \"country\": \"\"\n",
    "        },\n",
    "        \"customer_tax_id\": \"\",\n",
    "        \"shipping_address\": \"\",\n",
    "        \"purchase_order\": \"\",\n",
    "        \"invoice_id\": \"\",\n",
    "        \"invoice_date\": \"\",\n",
    "        \"payable_by\": \"\",\n",
    "        \"vendor_name\": \"\",\n",
    "        \"vendor_address\": \"\",\n",
    "        \"vendor_tax_id\": \"\",\n",
    "        \"remittance_address\": \"\",\n",
    "        \"subtotal\": 0,\n",
    "        \"total_discount\": 0,\n",
    "        \"total_tax\": 0,\n",
    "        \"invoice_total\": 0,\n",
    "        \"payment_terms\": \"\",\n",
    "        \"items\": [\n",
    "            {\n",
    "                \"product_code\": \"\",\n",
    "                \"description\": \"\",\n",
    "                \"quantity\": 0,\n",
    "                \"tax\": 0,\n",
    "                \"tax_rate\": \"\",\n",
    "                \"unit_price\": 0,\n",
    "                \"total\": 0,\n",
    "                \"reason\": null\n",
    "            }\n",
    "        ],\n",
    "        \"total_item_quantity\": 0,\n",
    "        \"items_customer_signature\": {\n",
    "            \"signatory\": \"\",\n",
    "            \"is_signed\": true\n",
    "        },\n",
    "        \"items_vendor_signature\": {\n",
    "            \"signatory\": \"\",\n",
    "            \"is_signed\": true\n",
    "        },\n",
    "        \"returns\": [\n",
    "            {\n",
    "                \"product_code\": \"\",\n",
    "                \"description\": \"\",\n",
    "                \"quantity\": 0,\n",
    "                \"tax\": null,\n",
    "                \"tax_rate\": null,\n",
    "                \"unit_price\": null,\n",
    "                \"total\": null,\n",
    "                \"reason\": \"\"\n",
    "            }\n",
    "        ],\n",
    "        \"total_return_quantity\": 0,\n",
    "        \"returns_customer_signature\": {\n",
    "            \"signatory\": \"\",\n",
    "            \"is_signed\": true\n",
    "        },\n",
    "        \"returns_vendor_signature\": {\n",
    "            \"signatory\": \"\",\n",
    "            \"is_signed\": true\n",
    "        }\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "The expected output has been defined by a human evaluating the document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = f\"{working_dir}/samples/assets/invoices/\"\n",
    "metadata_fname = \"invoice_1.json\" # Change this to the file you want to evaluate\n",
    "metadata_fpath = f\"{path}{metadata_fname}\"\n",
    "\n",
    "with open(metadata_fpath, \"r\") as f:\n",
    "    data = json.load(f)\n",
    "    \n",
    "expected = Invoice(**data['expected'])\n",
    "pdf_fname = data['fname']\n",
    "pdf_fpath = f\"{path}{pdf_fname}\"\n",
    "\n",
    "invoice_evaluator = AccuracyEvaluator(match_keys=['product_code', 'description'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract data from the document\n",
    "\n",
    "The following code block executes the data extraction process using Azure AI Document Intelligence and Phi-4.\n",
    "\n",
    "It performs the following steps:\n",
    "\n",
    "1. Get the document bytes from the provided file path. _Note: In this example, we are processing a local document, however, you can use any document storage location of your choice, such as Azure Blob Storage._\n",
    "2. Use Azure AI Document Intelligence to analyze the structure of the document and convert it to Markdown format using the pre-built layout model.\n",
    "3. Using a Phi-4 Serverless Endpoint model deployment in Azure AI Foundry and prompt engineering techniques, extract a structured data transfer object (DTO) from the content of the Markdown."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "with Stopwatch() as di_stopwatch:\n",
    "    with open(pdf_fpath, \"rb\") as f:\n",
    "        poller = document_intelligence_client.begin_analyze_document(\n",
    "            model_id=\"prebuilt-layout\",\n",
    "            body=f,\n",
    "            output_content_format=DocumentContentFormat.MARKDOWN,\n",
    "            content_type=\"application/pdf\"\n",
    "        )\n",
    "        \n",
    "    result: AnalyzeResult = poller.result()\n",
    "\n",
    "markdown = result.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "with Stopwatch() as phi_stopwatch:\n",
    "    completion = inference_client.complete(\n",
    "        messages=[\n",
    "            {\n",
    "                \"role\": \"system\",\n",
    "                \"content\": \"You are an AI assistant that extracts data from documents and returns them as structured JSON objects.\",\n",
    "            },\n",
    "            {\n",
    "                \"role\": \"user\",\n",
    "                \"content\": f\"\"\"Extract the data from this invoice. \n",
    "                - If a value is not present, provide null.\n",
    "                - Dates should be in the format YYYY-MM-DD.\n",
    "                - Strictly use the following JSON schema: {json.dumps(Invoice.model_json_schema())}\n",
    "                - ONLY return the JSON object. DO NOT return as a JSON markdown code block. DO NOT include any other detail in your response.\"\"\",\n",
    "            },\n",
    "            {\n",
    "                \"role\": \"user\",\n",
    "                \"content\": markdown,\n",
    "            }\n",
    "        ],\n",
    "        temperature=0.1,\n",
    "        top_p=0.1\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize the outputs\n",
    "\n",
    "To provide context for the execution of the code, the following code blocks visualize the outputs of the data extraction process.\n",
    "\n",
    "This includes:\n",
    "\n",
    "- The Markdown representation of the document structure as determined by Azure AI Document Intelligence.\n",
    "- The accuracy of the structured data extraction comparing the expected output with the output generated by Phi-4.\n",
    "- The confidence score of the structured data extraction by comparing against the Azure AI Document Intelligence analysis.\n",
    "- The execution time of the end-to-end process.\n",
    "- The total number of tokens consumed by the Phi-4 model.\n",
    "- The side-by-side comparison of the expected output and the output generated by Phi-4.\n",
    "\n",
    "### Understanding Accuracy vs Confidence\n",
    "\n",
    "When using AI to extract structured data, both confidence and accuracy are essential for different but complementary reasons.\n",
    "\n",
    "- **Accuracy** measures how close the AI model's output is to a ground truth or expected output. It reflects how well the model's predictions align with reality.\n",
    "  - Accuracy ensures consistency in the extraction process, which is crucial for downstream tasks using the data.\n",
    "- **Confidence** represents the AI model's internal assessment of how certain it is about its predictions.\n",
    "  - Confidence indicates that the model is certain about its predictions, which can be a useful indicator for human reviewers to step in for manual verification.\n",
    "\n",
    "High accuracy and high confidence are ideal, but in practice, there is often a trade-off between the two. While accuracy cannot always be self-assessed, confidence scores can and should be used to prioritize manual verification of low-confidence predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "NEXGEN\n",
       "\n",
       "Innovation drives progress\n",
       "\n",
       "\n",
       "# INVOICE\n",
       "\n",
       "\n",
       "<table>\n",
       "<tr>\n",
       "<th>Customer:</th>\n",
       "<th colspan=\"2\">Sharp Consulting</th>\n",
       "<th colspan=\"2\">Address:</th>\n",
       "<th colspan=\"3\">73 Regal Way, Leeds, LS1 5AB, UK</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Delv. Date:</td>\n",
       "<td>5/16/2024</td>\n",
       "<td colspan=\"2\">Invoice Number:</td>\n",
       "<td colspan=\"2\">3847193</td>\n",
       "<td>Purchase Order:</td>\n",
       "<td>15931</td>\n",
       "</tr>\n",
       "</table>\n",
       "\n",
       "\n",
       "<table>\n",
       "<tr>\n",
       "<th>Item Code</th>\n",
       "<th>Item Desc.</th>\n",
       "<th>Unit Price</th>\n",
       "<th>Quantity</th>\n",
       "<th>Total</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>MA197</td>\n",
       "<td>STRETCHWRAP ROLL</td>\n",
       "<td>16.62</td>\n",
       "<td>5</td>\n",
       "<td>83.10</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>ST4086</td>\n",
       "<td>BALLPOINT PEN MED.</td>\n",
       "<td>2.49</td>\n",
       "<td>10</td>\n",
       "<td>24.90</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>JF9912413BF</td>\n",
       "<td>BUBBLE FILM ROLL CL.</td>\n",
       "<td>15.46</td>\n",
       "<td>12</td>\n",
       "<td>185.52</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "</tr>\n",
       "</table>\n",
       "\n",
       "\n",
       "NOTES\n",
       "\n",
       "\n",
       "<table>\n",
       "<tr>\n",
       "<th>Total Pcs .:</th>\n",
       "<th>27</th>\n",
       "<th>Total Price:</th>\n",
       "<th>293.52</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td colspan=\"4\">Payable on or before 5/24/2024</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Cust. Sig.</td>\n",
       "<td>Sfr</td>\n",
       "<td>Drivr. Sig.</td>\n",
       "<td></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Cust. Name:</td>\n",
       "<td>Sarah H</td>\n",
       "<td>Drivr. Name:</td>\n",
       "<td>James T</td>\n",
       "</tr>\n",
       "</table>\n",
       "\n",
       "\n",
       "<!-- PageBreak -->\n",
       "\n",
       "\n",
       "<table>\n",
       "<tr>\n",
       "<td colspan=\"6\">NEXGEN Innovation drives progress</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td colspan=\"5\">RETURNS</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Customer:</td>\n",
       "<td>Sharp Consulting</td>\n",
       "<td colspan=\"2\">Address:</td>\n",
       "<td colspan=\"2\">73 Regal Way, Leeds, LS1 5AB, UK</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Coll. Date:</td>\n",
       "<td>5/16/2024</td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"2\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Item Code</td>\n",
       "<td>Item Desc.</td>\n",
       "<td>Pcs.</td>\n",
       "<td colspan=\"3\">Reason</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>MA145</td>\n",
       "<td>POSTAL TUBE BROWN</td>\n",
       "<td>1</td>\n",
       "<td colspan=\"3\">This item was provided in previous order</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>JF7902</td>\n",
       "<td>MAILBOX 25PK</td>\n",
       "<td>1 7</td>\n",
       "<td colspan=\"3\">as replacement</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"3\">Not required</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"3\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td colspan=\"6\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>NOTES</td>\n",
       "<td></td>\n",
       "<td colspan=\"4\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"4\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td colspan=\"5\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td colspan=\"5\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td colspan=\"2\"></td>\n",
       "<td></td>\n",
       "<td colspan=\"3\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td></td>\n",
       "<td></td>\n",
       "<td colspan=\"4\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Total Pcs .:</td>\n",
       "<td>2</td>\n",
       "<td></td>\n",
       "<td colspan=\"3\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td colspan=\"6\"></td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Cust. Sig.</td>\n",
       "<td>8</td>\n",
       "<td></td>\n",
       "<td colspan=\"2\">Drivr. Sig.</td>\n",
       "<td>&amp;</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>Cust. Name:</td>\n",
       "<td>Sarah H</td>\n",
       "<td></td>\n",
       "<td colspan=\"2\">Drivr. Name:</td>\n",
       "<td>James T</td>\n",
       "</tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Displays the output of the Azure AI Document Intelligence pre-built layout analysis in Markdown format.\n",
    "display(Markdown(markdown))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Gets the JSON response from the completion and converts it to an Invoice object.\n",
    "response_json = completion.choices[0].message.content\n",
    "\n",
    "# Remove any JSON markdown code block formatting from the response if present.\n",
    "response_json = response_json.replace(\"```json\", \"\").replace(\"```\", \"\").strip()\n",
    "\n",
    "invoice = Invoice.from_json(response_json)\n",
    "\n",
    "expected_dict = expected.to_dict()\n",
    "invoice_dict = invoice.to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Determines the accuracy of the extracted data against the expected values.\n",
    "accuracy = invoice_evaluator.evaluate(expected=expected_dict, actual=invoice_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Determines the confidence of the extracted data against the expected values using the result of the Azure AI Document Intelligence pre-built layout analysis.\n",
    "confidence = di_evaluate_confidence(invoice_dict, result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Gets the total execution time of the data extraction process.\n",
    "total_elapsed = di_stopwatch.elapsed + phi_stopwatch.elapsed\n",
    "\n",
    "# Gets the prompt tokens and completion tokens from the completion response.\n",
    "prompt_tokens = completion.usage.prompt_tokens\n",
    "completion_tokens = completion.usage.completion_tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the output of the data extraction result.\n",
    "extraction_result = DataExtractionResult(invoice.to_dict(), confidence, accuracy, prompt_tokens, completion_tokens, total_elapsed)\n",
    "\n",
    "with open(f\"{working_dir}/samples/extraction/text-based/document-intelligence-phi.{pdf_fname}.json\", \"w\") as f:\n",
    "    f.write(extraction_result.to_json(indent=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Accuracy</th>\n",
       "      <th>Confidence</th>\n",
       "      <th>Execution Time</th>\n",
       "      <th>Document Intelligence Execution Time</th>\n",
       "      <th>Phi-3.5 Execution Time</th>\n",
       "      <th>Prompt Tokens</th>\n",
       "      <th>Completion Tokens</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>94.59%</td>\n",
       "      <td>95.14%</td>\n",
       "      <td>114.75 seconds</td>\n",
       "      <td>8.71 seconds</td>\n",
       "      <td>106.04 seconds</td>\n",
       "      <td>3934</td>\n",
       "      <td>689</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Accuracy Confidence  Execution Time Document Intelligence Execution Time  \\\n",
       "0   94.59%     95.14%  114.75 seconds                         8.71 seconds   \n",
       "\n",
       "  Phi-3.5 Execution Time  Prompt Tokens  Completion Tokens  \n",
       "0         106.04 seconds           3934                689  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_5e81f_row0_col0, #T_5e81f_row0_col1, #T_5e81f_row0_col2, #T_5e81f_row0_col3, #T_5e81f_row0_col4, #T_5e81f_row1_col0, #T_5e81f_row1_col1, #T_5e81f_row1_col2, #T_5e81f_row1_col3, #T_5e81f_row1_col4, #T_5e81f_row2_col0, #T_5e81f_row2_col1, #T_5e81f_row2_col2, #T_5e81f_row2_col3, #T_5e81f_row2_col4, #T_5e81f_row3_col0, #T_5e81f_row3_col1, #T_5e81f_row3_col2, #T_5e81f_row3_col3, #T_5e81f_row3_col4, #T_5e81f_row4_col0, #T_5e81f_row4_col1, #T_5e81f_row4_col2, #T_5e81f_row4_col3, #T_5e81f_row4_col4, #T_5e81f_row5_col0, #T_5e81f_row5_col1, #T_5e81f_row5_col2, #T_5e81f_row5_col3, #T_5e81f_row5_col4, #T_5e81f_row6_col0, #T_5e81f_row6_col1, #T_5e81f_row6_col2, #T_5e81f_row6_col3, #T_5e81f_row6_col4, #T_5e81f_row7_col0, #T_5e81f_row7_col1, #T_5e81f_row7_col2, #T_5e81f_row7_col3, #T_5e81f_row7_col4, #T_5e81f_row8_col0, #T_5e81f_row8_col1, #T_5e81f_row8_col2, #T_5e81f_row8_col3, #T_5e81f_row8_col4, #T_5e81f_row9_col0, #T_5e81f_row9_col1, #T_5e81f_row9_col2, #T_5e81f_row9_col3, #T_5e81f_row9_col4, #T_5e81f_row10_col0, #T_5e81f_row10_col1, #T_5e81f_row10_col2, #T_5e81f_row10_col3, #T_5e81f_row10_col4, #T_5e81f_row11_col0, #T_5e81f_row11_col1, #T_5e81f_row11_col2, #T_5e81f_row11_col3, #T_5e81f_row11_col4, #T_5e81f_row12_col0, #T_5e81f_row12_col1, #T_5e81f_row12_col2, #T_5e81f_row12_col3, #T_5e81f_row12_col4, #T_5e81f_row13_col0, #T_5e81f_row13_col1, #T_5e81f_row13_col2, #T_5e81f_row13_col3, #T_5e81f_row13_col4, #T_5e81f_row14_col0, #T_5e81f_row14_col1, #T_5e81f_row14_col2, #T_5e81f_row14_col3, #T_5e81f_row14_col4, #T_5e81f_row15_col0, #T_5e81f_row15_col1, #T_5e81f_row15_col2, #T_5e81f_row15_col3, #T_5e81f_row15_col4, #T_5e81f_row16_col0, #T_5e81f_row16_col1, #T_5e81f_row16_col2, #T_5e81f_row16_col3, #T_5e81f_row16_col4, #T_5e81f_row17_col0, #T_5e81f_row17_col1, #T_5e81f_row17_col2, #T_5e81f_row17_col3, #T_5e81f_row17_col4, #T_5e81f_row18_col0, #T_5e81f_row18_col1, #T_5e81f_row18_col2, #T_5e81f_row18_col3, #T_5e81f_row18_col4, #T_5e81f_row19_col0, #T_5e81f_row19_col1, #T_5e81f_row19_col2, #T_5e81f_row19_col3, #T_5e81f_row19_col4, #T_5e81f_row20_col0, #T_5e81f_row20_col1, #T_5e81f_row20_col2, #T_5e81f_row20_col3, #T_5e81f_row20_col4, #T_5e81f_row21_col0, #T_5e81f_row21_col1, #T_5e81f_row21_col2, #T_5e81f_row21_col3, #T_5e81f_row21_col4, #T_5e81f_row22_col0, #T_5e81f_row22_col1, #T_5e81f_row22_col2, #T_5e81f_row22_col3, #T_5e81f_row22_col4, #T_5e81f_row23_col0, #T_5e81f_row23_col1, #T_5e81f_row23_col2, #T_5e81f_row23_col3, #T_5e81f_row23_col4, #T_5e81f_row24_col0, #T_5e81f_row24_col1, #T_5e81f_row24_col2, #T_5e81f_row24_col3, #T_5e81f_row24_col4, #T_5e81f_row25_col0, #T_5e81f_row25_col1, #T_5e81f_row25_col2, #T_5e81f_row25_col3, #T_5e81f_row25_col4, #T_5e81f_row26_col0, #T_5e81f_row26_col1, #T_5e81f_row26_col2, #T_5e81f_row26_col3, #T_5e81f_row26_col4, #T_5e81f_row27_col0, #T_5e81f_row27_col1, #T_5e81f_row27_col2, #T_5e81f_row27_col3, #T_5e81f_row27_col4, #T_5e81f_row28_col0, #T_5e81f_row28_col1, #T_5e81f_row28_col2, #T_5e81f_row28_col3, #T_5e81f_row28_col4, #T_5e81f_row29_col0, #T_5e81f_row29_col1, #T_5e81f_row29_col2, #T_5e81f_row29_col3, #T_5e81f_row29_col4, #T_5e81f_row30_col0, #T_5e81f_row30_col1, #T_5e81f_row30_col2, #T_5e81f_row30_col3, #T_5e81f_row30_col4, #T_5e81f_row31_col0, #T_5e81f_row31_col1, #T_5e81f_row31_col2, #T_5e81f_row31_col3, #T_5e81f_row31_col4, #T_5e81f_row32_col0, #T_5e81f_row32_col1, #T_5e81f_row32_col2, #T_5e81f_row32_col3, #T_5e81f_row32_col4, #T_5e81f_row33_col0, #T_5e81f_row33_col1, #T_5e81f_row33_col2, #T_5e81f_row33_col3, #T_5e81f_row33_col4, #T_5e81f_row34_col0, #T_5e81f_row34_col1, #T_5e81f_row34_col2, #T_5e81f_row34_col3, #T_5e81f_row34_col4, #T_5e81f_row35_col0, #T_5e81f_row35_col1, #T_5e81f_row35_col2, #T_5e81f_row35_col3, #T_5e81f_row35_col4, #T_5e81f_row36_col0, #T_5e81f_row36_col1, #T_5e81f_row36_col2, #T_5e81f_row36_col3, #T_5e81f_row36_col4, #T_5e81f_row37_col0, #T_5e81f_row37_col1, #T_5e81f_row37_col2, #T_5e81f_row37_col3, #T_5e81f_row37_col4, #T_5e81f_row38_col0, #T_5e81f_row38_col1, #T_5e81f_row38_col2, #T_5e81f_row38_col3, #T_5e81f_row38_col4, #T_5e81f_row39_col0, #T_5e81f_row39_col1, #T_5e81f_row39_col2, #T_5e81f_row39_col3, #T_5e81f_row39_col4, #T_5e81f_row40_col0, #T_5e81f_row40_col1, #T_5e81f_row40_col2, #T_5e81f_row40_col3, #T_5e81f_row40_col4, #T_5e81f_row41_col0, #T_5e81f_row41_col1, #T_5e81f_row41_col2, #T_5e81f_row41_col3, #T_5e81f_row41_col4, #T_5e81f_row42_col0, #T_5e81f_row42_col1, #T_5e81f_row42_col2, #T_5e81f_row42_col3, #T_5e81f_row42_col4, #T_5e81f_row43_col0, #T_5e81f_row43_col1, #T_5e81f_row43_col2, #T_5e81f_row43_col3, #T_5e81f_row43_col4, #T_5e81f_row44_col0, #T_5e81f_row44_col1, #T_5e81f_row44_col2, #T_5e81f_row44_col3, #T_5e81f_row44_col4, #T_5e81f_row46_col0, #T_5e81f_row46_col1, #T_5e81f_row46_col2, #T_5e81f_row46_col3, #T_5e81f_row46_col4, #T_5e81f_row47_col0, #T_5e81f_row47_col1, #T_5e81f_row47_col2, #T_5e81f_row47_col3, #T_5e81f_row47_col4, #T_5e81f_row48_col0, #T_5e81f_row48_col1, #T_5e81f_row48_col2, #T_5e81f_row48_col3, #T_5e81f_row48_col4, #T_5e81f_row49_col0, #T_5e81f_row49_col1, #T_5e81f_row49_col2, #T_5e81f_row49_col3, #T_5e81f_row49_col4, #T_5e81f_row50_col0, #T_5e81f_row50_col1, #T_5e81f_row50_col2, #T_5e81f_row50_col3, #T_5e81f_row50_col4, #T_5e81f_row51_col0, #T_5e81f_row51_col1, #T_5e81f_row51_col2, #T_5e81f_row51_col3, #T_5e81f_row51_col4, #T_5e81f_row54_col0, #T_5e81f_row54_col1, #T_5e81f_row54_col2, #T_5e81f_row54_col3, #T_5e81f_row54_col4, #T_5e81f_row55_col0, #T_5e81f_row55_col1, #T_5e81f_row55_col2, #T_5e81f_row55_col3, #T_5e81f_row55_col4, #T_5e81f_row56_col0, #T_5e81f_row56_col1, #T_5e81f_row56_col2, #T_5e81f_row56_col3, #T_5e81f_row56_col4, #T_5e81f_row57_col0, #T_5e81f_row57_col1, #T_5e81f_row57_col2, #T_5e81f_row57_col3, #T_5e81f_row57_col4, #T_5e81f_row58_col0, #T_5e81f_row58_col1, #T_5e81f_row58_col2, #T_5e81f_row58_col3, #T_5e81f_row58_col4, #T_5e81f_row59_col0, #T_5e81f_row59_col1, #T_5e81f_row59_col2, #T_5e81f_row59_col3, #T_5e81f_row59_col4, #T_5e81f_row60_col0, #T_5e81f_row60_col1, #T_5e81f_row60_col2, #T_5e81f_row60_col3, #T_5e81f_row60_col4, #T_5e81f_row61_col0, #T_5e81f_row61_col1, #T_5e81f_row61_col2, #T_5e81f_row61_col3, #T_5e81f_row61_col4, #T_5e81f_row62_col0, #T_5e81f_row62_col1, #T_5e81f_row62_col2, #T_5e81f_row62_col3, #T_5e81f_row62_col4, #T_5e81f_row63_col0, #T_5e81f_row63_col1, #T_5e81f_row63_col2, #T_5e81f_row63_col3, #T_5e81f_row63_col4, #T_5e81f_row64_col0, #T_5e81f_row64_col1, #T_5e81f_row64_col2, #T_5e81f_row64_col3, #T_5e81f_row64_col4, #T_5e81f_row65_col0, #T_5e81f_row65_col1, #T_5e81f_row65_col2, #T_5e81f_row65_col3, #T_5e81f_row65_col4, #T_5e81f_row66_col0, #T_5e81f_row66_col1, #T_5e81f_row66_col2, #T_5e81f_row66_col3, #T_5e81f_row66_col4, #T_5e81f_row67_col0, #T_5e81f_row67_col1, #T_5e81f_row67_col2, #T_5e81f_row67_col3, #T_5e81f_row67_col4, #T_5e81f_row68_col0, #T_5e81f_row68_col1, #T_5e81f_row68_col2, #T_5e81f_row68_col3, #T_5e81f_row68_col4, #T_5e81f_row69_col0, #T_5e81f_row69_col1, #T_5e81f_row69_col2, #T_5e81f_row69_col3, #T_5e81f_row69_col4, #T_5e81f_row70_col0, #T_5e81f_row70_col1, #T_5e81f_row70_col2, #T_5e81f_row70_col3, #T_5e81f_row70_col4 {\n",
       "  background-color: #66ff33;\n",
       "}\n",
       "#T_5e81f_row45_col0, #T_5e81f_row45_col1, #T_5e81f_row45_col2, #T_5e81f_row45_col3, #T_5e81f_row45_col4, #T_5e81f_row52_col0, #T_5e81f_row52_col1, #T_5e81f_row52_col2, #T_5e81f_row52_col3, #T_5e81f_row52_col4, #T_5e81f_row53_col0, #T_5e81f_row53_col1, #T_5e81f_row53_col2, #T_5e81f_row53_col3, #T_5e81f_row53_col4 {\n",
       "  background-color: #ff9999;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_5e81f\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_5e81f_level0_col0\" class=\"col_heading level0 col0\" >Field</th>\n",
       "      <th id=\"T_5e81f_level0_col1\" class=\"col_heading level0 col1\" >Expected</th>\n",
       "      <th id=\"T_5e81f_level0_col2\" class=\"col_heading level0 col2\" >Extracted</th>\n",
       "      <th id=\"T_5e81f_level0_col3\" class=\"col_heading level0 col3\" >Confidence</th>\n",
       "      <th id=\"T_5e81f_level0_col4\" class=\"col_heading level0 col4\" >Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "      <td id=\"T_5e81f_row0_col0\" class=\"data row0 col0\" >customer_address_city</td>\n",
       "      <td id=\"T_5e81f_row0_col1\" class=\"data row0 col1\" >Leeds</td>\n",
       "      <td id=\"T_5e81f_row0_col2\" class=\"data row0 col2\" >Leeds</td>\n",
       "      <td id=\"T_5e81f_row0_col3\" class=\"data row0 col3\" >98.70%</td>\n",
       "      <td id=\"T_5e81f_row0_col4\" class=\"data row0 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "      <td id=\"T_5e81f_row1_col0\" class=\"data row1 col0\" >customer_address_country</td>\n",
       "      <td id=\"T_5e81f_row1_col1\" class=\"data row1 col1\" >UK</td>\n",
       "      <td id=\"T_5e81f_row1_col2\" class=\"data row1 col2\" >UK</td>\n",
       "      <td id=\"T_5e81f_row1_col3\" class=\"data row1 col3\" >98.70%</td>\n",
       "      <td id=\"T_5e81f_row1_col4\" class=\"data row1 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "      <td id=\"T_5e81f_row2_col0\" class=\"data row2 col0\" >customer_address_postal_code</td>\n",
       "      <td id=\"T_5e81f_row2_col1\" class=\"data row2 col1\" >LS1 5AB</td>\n",
       "      <td id=\"T_5e81f_row2_col2\" class=\"data row2 col2\" >LS1 5AB</td>\n",
       "      <td id=\"T_5e81f_row2_col3\" class=\"data row2 col3\" >98.70%</td>\n",
       "      <td id=\"T_5e81f_row2_col4\" class=\"data row2 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "      <td id=\"T_5e81f_row3_col0\" class=\"data row3 col0\" >customer_address_state</td>\n",
       "      <td id=\"T_5e81f_row3_col1\" class=\"data row3 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row3_col2\" class=\"data row3 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row3_col3\" class=\"data row3 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row3_col4\" class=\"data row3 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "      <td id=\"T_5e81f_row4_col0\" class=\"data row4 col0\" >customer_address_street</td>\n",
       "      <td id=\"T_5e81f_row4_col1\" class=\"data row4 col1\" >73 Regal Way</td>\n",
       "      <td id=\"T_5e81f_row4_col2\" class=\"data row4 col2\" >73 Regal Way</td>\n",
       "      <td id=\"T_5e81f_row4_col3\" class=\"data row4 col3\" >98.70%</td>\n",
       "      <td id=\"T_5e81f_row4_col4\" class=\"data row4 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
       "      <td id=\"T_5e81f_row5_col0\" class=\"data row5 col0\" >customer_name</td>\n",
       "      <td id=\"T_5e81f_row5_col1\" class=\"data row5 col1\" >Sharp Consulting</td>\n",
       "      <td id=\"T_5e81f_row5_col2\" class=\"data row5 col2\" >Sharp Consulting</td>\n",
       "      <td id=\"T_5e81f_row5_col3\" class=\"data row5 col3\" >99.40%</td>\n",
       "      <td id=\"T_5e81f_row5_col4\" class=\"data row5 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
       "      <td id=\"T_5e81f_row6_col0\" class=\"data row6 col0\" >customer_tax_id</td>\n",
       "      <td id=\"T_5e81f_row6_col1\" class=\"data row6 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row6_col2\" class=\"data row6 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row6_col3\" class=\"data row6 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row6_col4\" class=\"data row6 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
       "      <td id=\"T_5e81f_row7_col0\" class=\"data row7 col0\" >invoice_date</td>\n",
       "      <td id=\"T_5e81f_row7_col1\" class=\"data row7 col1\" >2024-05-16</td>\n",
       "      <td id=\"T_5e81f_row7_col2\" class=\"data row7 col2\" >2024-05-16</td>\n",
       "      <td id=\"T_5e81f_row7_col3\" class=\"data row7 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row7_col4\" class=\"data row7 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
       "      <td id=\"T_5e81f_row8_col0\" class=\"data row8 col0\" >invoice_id</td>\n",
       "      <td id=\"T_5e81f_row8_col1\" class=\"data row8 col1\" >3847193</td>\n",
       "      <td id=\"T_5e81f_row8_col2\" class=\"data row8 col2\" >3847193</td>\n",
       "      <td id=\"T_5e81f_row8_col3\" class=\"data row8 col3\" >99.80%</td>\n",
       "      <td id=\"T_5e81f_row8_col4\" class=\"data row8 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
       "      <td id=\"T_5e81f_row9_col0\" class=\"data row9 col0\" >invoice_total</td>\n",
       "      <td id=\"T_5e81f_row9_col1\" class=\"data row9 col1\" >293.520000</td>\n",
       "      <td id=\"T_5e81f_row9_col2\" class=\"data row9 col2\" >293.520000</td>\n",
       "      <td id=\"T_5e81f_row9_col3\" class=\"data row9 col3\" >99.40%</td>\n",
       "      <td id=\"T_5e81f_row9_col4\" class=\"data row9 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
       "      <td id=\"T_5e81f_row10_col0\" class=\"data row10 col0\" >items_0_description</td>\n",
       "      <td id=\"T_5e81f_row10_col1\" class=\"data row10 col1\" >STRETCHWRAP ROLL</td>\n",
       "      <td id=\"T_5e81f_row10_col2\" class=\"data row10 col2\" >STRETCHWRAP ROLL</td>\n",
       "      <td id=\"T_5e81f_row10_col3\" class=\"data row10 col3\" >98.90%</td>\n",
       "      <td id=\"T_5e81f_row10_col4\" class=\"data row10 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
       "      <td id=\"T_5e81f_row11_col0\" class=\"data row11 col0\" >items_0_product_code</td>\n",
       "      <td id=\"T_5e81f_row11_col1\" class=\"data row11 col1\" >MA197</td>\n",
       "      <td id=\"T_5e81f_row11_col2\" class=\"data row11 col2\" >MA197</td>\n",
       "      <td id=\"T_5e81f_row11_col3\" class=\"data row11 col3\" >99.70%</td>\n",
       "      <td id=\"T_5e81f_row11_col4\" class=\"data row11 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
       "      <td id=\"T_5e81f_row12_col0\" class=\"data row12 col0\" >items_0_quantity</td>\n",
       "      <td id=\"T_5e81f_row12_col1\" class=\"data row12 col1\" >5</td>\n",
       "      <td id=\"T_5e81f_row12_col2\" class=\"data row12 col2\" >5</td>\n",
       "      <td id=\"T_5e81f_row12_col3\" class=\"data row12 col3\" >99.60%</td>\n",
       "      <td id=\"T_5e81f_row12_col4\" class=\"data row12 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row13\" class=\"row_heading level0 row13\" >13</th>\n",
       "      <td id=\"T_5e81f_row13_col0\" class=\"data row13 col0\" >items_0_reason</td>\n",
       "      <td id=\"T_5e81f_row13_col1\" class=\"data row13 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row13_col2\" class=\"data row13 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row13_col3\" class=\"data row13 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row13_col4\" class=\"data row13 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row14\" class=\"row_heading level0 row14\" >14</th>\n",
       "      <td id=\"T_5e81f_row14_col0\" class=\"data row14 col0\" >items_0_tax</td>\n",
       "      <td id=\"T_5e81f_row14_col1\" class=\"data row14 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row14_col2\" class=\"data row14 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row14_col3\" class=\"data row14 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row14_col4\" class=\"data row14 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row15\" class=\"row_heading level0 row15\" >15</th>\n",
       "      <td id=\"T_5e81f_row15_col0\" class=\"data row15 col0\" >items_0_tax_rate</td>\n",
       "      <td id=\"T_5e81f_row15_col1\" class=\"data row15 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row15_col2\" class=\"data row15 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row15_col3\" class=\"data row15 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row15_col4\" class=\"data row15 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row16\" class=\"row_heading level0 row16\" >16</th>\n",
       "      <td id=\"T_5e81f_row16_col0\" class=\"data row16 col0\" >items_0_total</td>\n",
       "      <td id=\"T_5e81f_row16_col1\" class=\"data row16 col1\" >83.100000</td>\n",
       "      <td id=\"T_5e81f_row16_col2\" class=\"data row16 col2\" >83.100000</td>\n",
       "      <td id=\"T_5e81f_row16_col3\" class=\"data row16 col3\" >99.80%</td>\n",
       "      <td id=\"T_5e81f_row16_col4\" class=\"data row16 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row17\" class=\"row_heading level0 row17\" >17</th>\n",
       "      <td id=\"T_5e81f_row17_col0\" class=\"data row17 col0\" >items_0_unit_price</td>\n",
       "      <td id=\"T_5e81f_row17_col1\" class=\"data row17 col1\" >16.620000</td>\n",
       "      <td id=\"T_5e81f_row17_col2\" class=\"data row17 col2\" >16.620000</td>\n",
       "      <td id=\"T_5e81f_row17_col3\" class=\"data row17 col3\" >99.70%</td>\n",
       "      <td id=\"T_5e81f_row17_col4\" class=\"data row17 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row18\" class=\"row_heading level0 row18\" >18</th>\n",
       "      <td id=\"T_5e81f_row18_col0\" class=\"data row18 col0\" >items_1_description</td>\n",
       "      <td id=\"T_5e81f_row18_col1\" class=\"data row18 col1\" >BALLPOINT PEN MED.</td>\n",
       "      <td id=\"T_5e81f_row18_col2\" class=\"data row18 col2\" >BALLPOINT PEN MED.</td>\n",
       "      <td id=\"T_5e81f_row18_col3\" class=\"data row18 col3\" >99.20%</td>\n",
       "      <td id=\"T_5e81f_row18_col4\" class=\"data row18 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row19\" class=\"row_heading level0 row19\" >19</th>\n",
       "      <td id=\"T_5e81f_row19_col0\" class=\"data row19 col0\" >items_1_product_code</td>\n",
       "      <td id=\"T_5e81f_row19_col1\" class=\"data row19 col1\" >ST4086</td>\n",
       "      <td id=\"T_5e81f_row19_col2\" class=\"data row19 col2\" >ST4086</td>\n",
       "      <td id=\"T_5e81f_row19_col3\" class=\"data row19 col3\" >99.70%</td>\n",
       "      <td id=\"T_5e81f_row19_col4\" class=\"data row19 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row20\" class=\"row_heading level0 row20\" >20</th>\n",
       "      <td id=\"T_5e81f_row20_col0\" class=\"data row20 col0\" >items_1_quantity</td>\n",
       "      <td id=\"T_5e81f_row20_col1\" class=\"data row20 col1\" >10</td>\n",
       "      <td id=\"T_5e81f_row20_col2\" class=\"data row20 col2\" >10</td>\n",
       "      <td id=\"T_5e81f_row20_col3\" class=\"data row20 col3\" >99.80%</td>\n",
       "      <td id=\"T_5e81f_row20_col4\" class=\"data row20 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row21\" class=\"row_heading level0 row21\" >21</th>\n",
       "      <td id=\"T_5e81f_row21_col0\" class=\"data row21 col0\" >items_1_reason</td>\n",
       "      <td id=\"T_5e81f_row21_col1\" class=\"data row21 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row21_col2\" class=\"data row21 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row21_col3\" class=\"data row21 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row21_col4\" class=\"data row21 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row22\" class=\"row_heading level0 row22\" >22</th>\n",
       "      <td id=\"T_5e81f_row22_col0\" class=\"data row22 col0\" >items_1_tax</td>\n",
       "      <td id=\"T_5e81f_row22_col1\" class=\"data row22 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row22_col2\" class=\"data row22 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row22_col3\" class=\"data row22 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row22_col4\" class=\"data row22 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row23\" class=\"row_heading level0 row23\" >23</th>\n",
       "      <td id=\"T_5e81f_row23_col0\" class=\"data row23 col0\" >items_1_tax_rate</td>\n",
       "      <td id=\"T_5e81f_row23_col1\" class=\"data row23 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row23_col2\" class=\"data row23 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row23_col3\" class=\"data row23 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row23_col4\" class=\"data row23 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row24\" class=\"row_heading level0 row24\" >24</th>\n",
       "      <td id=\"T_5e81f_row24_col0\" class=\"data row24 col0\" >items_1_total</td>\n",
       "      <td id=\"T_5e81f_row24_col1\" class=\"data row24 col1\" >24.900000</td>\n",
       "      <td id=\"T_5e81f_row24_col2\" class=\"data row24 col2\" >24.900000</td>\n",
       "      <td id=\"T_5e81f_row24_col3\" class=\"data row24 col3\" >99.60%</td>\n",
       "      <td id=\"T_5e81f_row24_col4\" class=\"data row24 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row25\" class=\"row_heading level0 row25\" >25</th>\n",
       "      <td id=\"T_5e81f_row25_col0\" class=\"data row25 col0\" >items_1_unit_price</td>\n",
       "      <td id=\"T_5e81f_row25_col1\" class=\"data row25 col1\" >2.490000</td>\n",
       "      <td id=\"T_5e81f_row25_col2\" class=\"data row25 col2\" >2.490000</td>\n",
       "      <td id=\"T_5e81f_row25_col3\" class=\"data row25 col3\" >99.20%</td>\n",
       "      <td id=\"T_5e81f_row25_col4\" class=\"data row25 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row26\" class=\"row_heading level0 row26\" >26</th>\n",
       "      <td id=\"T_5e81f_row26_col0\" class=\"data row26 col0\" >items_2_description</td>\n",
       "      <td id=\"T_5e81f_row26_col1\" class=\"data row26 col1\" >BUBBLE FILM ROLL CL.</td>\n",
       "      <td id=\"T_5e81f_row26_col2\" class=\"data row26 col2\" >BUBBLE FILM ROLL CL.</td>\n",
       "      <td id=\"T_5e81f_row26_col3\" class=\"data row26 col3\" >98.60%</td>\n",
       "      <td id=\"T_5e81f_row26_col4\" class=\"data row26 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row27\" class=\"row_heading level0 row27\" >27</th>\n",
       "      <td id=\"T_5e81f_row27_col0\" class=\"data row27 col0\" >items_2_product_code</td>\n",
       "      <td id=\"T_5e81f_row27_col1\" class=\"data row27 col1\" >JF9912413BF</td>\n",
       "      <td id=\"T_5e81f_row27_col2\" class=\"data row27 col2\" >JF9912413BF</td>\n",
       "      <td id=\"T_5e81f_row27_col3\" class=\"data row27 col3\" >99.50%</td>\n",
       "      <td id=\"T_5e81f_row27_col4\" class=\"data row27 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row28\" class=\"row_heading level0 row28\" >28</th>\n",
       "      <td id=\"T_5e81f_row28_col0\" class=\"data row28 col0\" >items_2_quantity</td>\n",
       "      <td id=\"T_5e81f_row28_col1\" class=\"data row28 col1\" >12</td>\n",
       "      <td id=\"T_5e81f_row28_col2\" class=\"data row28 col2\" >12</td>\n",
       "      <td id=\"T_5e81f_row28_col3\" class=\"data row28 col3\" >99.80%</td>\n",
       "      <td id=\"T_5e81f_row28_col4\" class=\"data row28 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row29\" class=\"row_heading level0 row29\" >29</th>\n",
       "      <td id=\"T_5e81f_row29_col0\" class=\"data row29 col0\" >items_2_reason</td>\n",
       "      <td id=\"T_5e81f_row29_col1\" class=\"data row29 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row29_col2\" class=\"data row29 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row29_col3\" class=\"data row29 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row29_col4\" class=\"data row29 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row30\" class=\"row_heading level0 row30\" >30</th>\n",
       "      <td id=\"T_5e81f_row30_col0\" class=\"data row30 col0\" >items_2_tax</td>\n",
       "      <td id=\"T_5e81f_row30_col1\" class=\"data row30 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row30_col2\" class=\"data row30 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row30_col3\" class=\"data row30 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row30_col4\" class=\"data row30 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row31\" class=\"row_heading level0 row31\" >31</th>\n",
       "      <td id=\"T_5e81f_row31_col0\" class=\"data row31 col0\" >items_2_tax_rate</td>\n",
       "      <td id=\"T_5e81f_row31_col1\" class=\"data row31 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row31_col2\" class=\"data row31 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row31_col3\" class=\"data row31 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row31_col4\" class=\"data row31 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row32\" class=\"row_heading level0 row32\" >32</th>\n",
       "      <td id=\"T_5e81f_row32_col0\" class=\"data row32 col0\" >items_2_total</td>\n",
       "      <td id=\"T_5e81f_row32_col1\" class=\"data row32 col1\" >185.520000</td>\n",
       "      <td id=\"T_5e81f_row32_col2\" class=\"data row32 col2\" >185.520000</td>\n",
       "      <td id=\"T_5e81f_row32_col3\" class=\"data row32 col3\" >99.60%</td>\n",
       "      <td id=\"T_5e81f_row32_col4\" class=\"data row32 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row33\" class=\"row_heading level0 row33\" >33</th>\n",
       "      <td id=\"T_5e81f_row33_col0\" class=\"data row33 col0\" >items_2_unit_price</td>\n",
       "      <td id=\"T_5e81f_row33_col1\" class=\"data row33 col1\" >15.460000</td>\n",
       "      <td id=\"T_5e81f_row33_col2\" class=\"data row33 col2\" >15.460000</td>\n",
       "      <td id=\"T_5e81f_row33_col3\" class=\"data row33 col3\" >99.80%</td>\n",
       "      <td id=\"T_5e81f_row33_col4\" class=\"data row33 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row34\" class=\"row_heading level0 row34\" >34</th>\n",
       "      <td id=\"T_5e81f_row34_col0\" class=\"data row34 col0\" >items_customer_signature_is_signed</td>\n",
       "      <td id=\"T_5e81f_row34_col1\" class=\"data row34 col1\" >True</td>\n",
       "      <td id=\"T_5e81f_row34_col2\" class=\"data row34 col2\" >True</td>\n",
       "      <td id=\"T_5e81f_row34_col3\" class=\"data row34 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row34_col4\" class=\"data row34 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row35\" class=\"row_heading level0 row35\" >35</th>\n",
       "      <td id=\"T_5e81f_row35_col0\" class=\"data row35 col0\" >items_customer_signature_signatory</td>\n",
       "      <td id=\"T_5e81f_row35_col1\" class=\"data row35 col1\" >Sarah H</td>\n",
       "      <td id=\"T_5e81f_row35_col2\" class=\"data row35 col2\" >Sarah H</td>\n",
       "      <td id=\"T_5e81f_row35_col3\" class=\"data row35 col3\" >54.80%</td>\n",
       "      <td id=\"T_5e81f_row35_col4\" class=\"data row35 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row36\" class=\"row_heading level0 row36\" >36</th>\n",
       "      <td id=\"T_5e81f_row36_col0\" class=\"data row36 col0\" >items_vendor_signature_is_signed</td>\n",
       "      <td id=\"T_5e81f_row36_col1\" class=\"data row36 col1\" >True</td>\n",
       "      <td id=\"T_5e81f_row36_col2\" class=\"data row36 col2\" >True</td>\n",
       "      <td id=\"T_5e81f_row36_col3\" class=\"data row36 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row36_col4\" class=\"data row36 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row37\" class=\"row_heading level0 row37\" >37</th>\n",
       "      <td id=\"T_5e81f_row37_col0\" class=\"data row37 col0\" >items_vendor_signature_signatory</td>\n",
       "      <td id=\"T_5e81f_row37_col1\" class=\"data row37 col1\" >James T</td>\n",
       "      <td id=\"T_5e81f_row37_col2\" class=\"data row37 col2\" >James T</td>\n",
       "      <td id=\"T_5e81f_row37_col3\" class=\"data row37 col3\" >86.50%</td>\n",
       "      <td id=\"T_5e81f_row37_col4\" class=\"data row37 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row38\" class=\"row_heading level0 row38\" >38</th>\n",
       "      <td id=\"T_5e81f_row38_col0\" class=\"data row38 col0\" >payable_by</td>\n",
       "      <td id=\"T_5e81f_row38_col1\" class=\"data row38 col1\" >2024-05-24</td>\n",
       "      <td id=\"T_5e81f_row38_col2\" class=\"data row38 col2\" >2024-05-24</td>\n",
       "      <td id=\"T_5e81f_row38_col3\" class=\"data row38 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row38_col4\" class=\"data row38 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row39\" class=\"row_heading level0 row39\" >39</th>\n",
       "      <td id=\"T_5e81f_row39_col0\" class=\"data row39 col0\" >payment_terms</td>\n",
       "      <td id=\"T_5e81f_row39_col1\" class=\"data row39 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row39_col2\" class=\"data row39 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row39_col3\" class=\"data row39 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row39_col4\" class=\"data row39 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row40\" class=\"row_heading level0 row40\" >40</th>\n",
       "      <td id=\"T_5e81f_row40_col0\" class=\"data row40 col0\" >purchase_order</td>\n",
       "      <td id=\"T_5e81f_row40_col1\" class=\"data row40 col1\" >15931</td>\n",
       "      <td id=\"T_5e81f_row40_col2\" class=\"data row40 col2\" >15931</td>\n",
       "      <td id=\"T_5e81f_row40_col3\" class=\"data row40 col3\" >99.50%</td>\n",
       "      <td id=\"T_5e81f_row40_col4\" class=\"data row40 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row41\" class=\"row_heading level0 row41\" >41</th>\n",
       "      <td id=\"T_5e81f_row41_col0\" class=\"data row41 col0\" >remittance_address</td>\n",
       "      <td id=\"T_5e81f_row41_col1\" class=\"data row41 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row41_col2\" class=\"data row41 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row41_col3\" class=\"data row41 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row41_col4\" class=\"data row41 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row42\" class=\"row_heading level0 row42\" >42</th>\n",
       "      <td id=\"T_5e81f_row42_col0\" class=\"data row42 col0\" >returns_0_description</td>\n",
       "      <td id=\"T_5e81f_row42_col1\" class=\"data row42 col1\" >POSTAL TUBE BROWN</td>\n",
       "      <td id=\"T_5e81f_row42_col2\" class=\"data row42 col2\" >POSTAL TUBE BROWN</td>\n",
       "      <td id=\"T_5e81f_row42_col3\" class=\"data row42 col3\" >99.20%</td>\n",
       "      <td id=\"T_5e81f_row42_col4\" class=\"data row42 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row43\" class=\"row_heading level0 row43\" >43</th>\n",
       "      <td id=\"T_5e81f_row43_col0\" class=\"data row43 col0\" >returns_0_product_code</td>\n",
       "      <td id=\"T_5e81f_row43_col1\" class=\"data row43 col1\" >MA145</td>\n",
       "      <td id=\"T_5e81f_row43_col2\" class=\"data row43 col2\" >MA145</td>\n",
       "      <td id=\"T_5e81f_row43_col3\" class=\"data row43 col3\" >99.60%</td>\n",
       "      <td id=\"T_5e81f_row43_col4\" class=\"data row43 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row44\" class=\"row_heading level0 row44\" >44</th>\n",
       "      <td id=\"T_5e81f_row44_col0\" class=\"data row44 col0\" >returns_0_quantity</td>\n",
       "      <td id=\"T_5e81f_row44_col1\" class=\"data row44 col1\" >1</td>\n",
       "      <td id=\"T_5e81f_row44_col2\" class=\"data row44 col2\" >1</td>\n",
       "      <td id=\"T_5e81f_row44_col3\" class=\"data row44 col3\" >99.40%</td>\n",
       "      <td id=\"T_5e81f_row44_col4\" class=\"data row44 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row45\" class=\"row_heading level0 row45\" >45</th>\n",
       "      <td id=\"T_5e81f_row45_col0\" class=\"data row45 col0\" >returns_0_reason</td>\n",
       "      <td id=\"T_5e81f_row45_col1\" class=\"data row45 col1\" >This item was provided in previous order as a replacement</td>\n",
       "      <td id=\"T_5e81f_row45_col2\" class=\"data row45 col2\" >This item was provided in previous order</td>\n",
       "      <td id=\"T_5e81f_row45_col3\" class=\"data row45 col3\" >66.20%</td>\n",
       "      <td id=\"T_5e81f_row45_col4\" class=\"data row45 col4\" >Mismatch</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row46\" class=\"row_heading level0 row46\" >46</th>\n",
       "      <td id=\"T_5e81f_row46_col0\" class=\"data row46 col0\" >returns_0_tax</td>\n",
       "      <td id=\"T_5e81f_row46_col1\" class=\"data row46 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row46_col2\" class=\"data row46 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row46_col3\" class=\"data row46 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row46_col4\" class=\"data row46 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row47\" class=\"row_heading level0 row47\" >47</th>\n",
       "      <td id=\"T_5e81f_row47_col0\" class=\"data row47 col0\" >returns_0_tax_rate</td>\n",
       "      <td id=\"T_5e81f_row47_col1\" class=\"data row47 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row47_col2\" class=\"data row47 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row47_col3\" class=\"data row47 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row47_col4\" class=\"data row47 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row48\" class=\"row_heading level0 row48\" >48</th>\n",
       "      <td id=\"T_5e81f_row48_col0\" class=\"data row48 col0\" >returns_0_total</td>\n",
       "      <td id=\"T_5e81f_row48_col1\" class=\"data row48 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row48_col2\" class=\"data row48 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row48_col3\" class=\"data row48 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row48_col4\" class=\"data row48 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row49\" class=\"row_heading level0 row49\" >49</th>\n",
       "      <td id=\"T_5e81f_row49_col0\" class=\"data row49 col0\" >returns_0_unit_price</td>\n",
       "      <td id=\"T_5e81f_row49_col1\" class=\"data row49 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row49_col2\" class=\"data row49 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row49_col3\" class=\"data row49 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row49_col4\" class=\"data row49 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row50\" class=\"row_heading level0 row50\" >50</th>\n",
       "      <td id=\"T_5e81f_row50_col0\" class=\"data row50 col0\" >returns_1_description</td>\n",
       "      <td id=\"T_5e81f_row50_col1\" class=\"data row50 col1\" >MAILBOX 25PK</td>\n",
       "      <td id=\"T_5e81f_row50_col2\" class=\"data row50 col2\" >MAILBOX 25PK</td>\n",
       "      <td id=\"T_5e81f_row50_col3\" class=\"data row50 col3\" >99.20%</td>\n",
       "      <td id=\"T_5e81f_row50_col4\" class=\"data row50 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row51\" class=\"row_heading level0 row51\" >51</th>\n",
       "      <td id=\"T_5e81f_row51_col0\" class=\"data row51 col0\" >returns_1_product_code</td>\n",
       "      <td id=\"T_5e81f_row51_col1\" class=\"data row51 col1\" >JF7902</td>\n",
       "      <td id=\"T_5e81f_row51_col2\" class=\"data row51 col2\" >JF7902</td>\n",
       "      <td id=\"T_5e81f_row51_col3\" class=\"data row51 col3\" >99.50%</td>\n",
       "      <td id=\"T_5e81f_row51_col4\" class=\"data row51 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row52\" class=\"row_heading level0 row52\" >52</th>\n",
       "      <td id=\"T_5e81f_row52_col0\" class=\"data row52 col0\" >returns_1_quantity</td>\n",
       "      <td id=\"T_5e81f_row52_col1\" class=\"data row52 col1\" >1</td>\n",
       "      <td id=\"T_5e81f_row52_col2\" class=\"data row52 col2\" >17</td>\n",
       "      <td id=\"T_5e81f_row52_col3\" class=\"data row52 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row52_col4\" class=\"data row52 col4\" >Mismatch</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row53\" class=\"row_heading level0 row53\" >53</th>\n",
       "      <td id=\"T_5e81f_row53_col0\" class=\"data row53 col0\" >returns_1_reason</td>\n",
       "      <td id=\"T_5e81f_row53_col1\" class=\"data row53 col1\" >Not required</td>\n",
       "      <td id=\"T_5e81f_row53_col2\" class=\"data row53 col2\" >as replacement Not required</td>\n",
       "      <td id=\"T_5e81f_row53_col3\" class=\"data row53 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row53_col4\" class=\"data row53 col4\" >Mismatch</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row54\" class=\"row_heading level0 row54\" >54</th>\n",
       "      <td id=\"T_5e81f_row54_col0\" class=\"data row54 col0\" >returns_1_tax</td>\n",
       "      <td id=\"T_5e81f_row54_col1\" class=\"data row54 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row54_col2\" class=\"data row54 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row54_col3\" class=\"data row54 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row54_col4\" class=\"data row54 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row55\" class=\"row_heading level0 row55\" >55</th>\n",
       "      <td id=\"T_5e81f_row55_col0\" class=\"data row55 col0\" >returns_1_tax_rate</td>\n",
       "      <td id=\"T_5e81f_row55_col1\" class=\"data row55 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row55_col2\" class=\"data row55 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row55_col3\" class=\"data row55 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row55_col4\" class=\"data row55 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row56\" class=\"row_heading level0 row56\" >56</th>\n",
       "      <td id=\"T_5e81f_row56_col0\" class=\"data row56 col0\" >returns_1_total</td>\n",
       "      <td id=\"T_5e81f_row56_col1\" class=\"data row56 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row56_col2\" class=\"data row56 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row56_col3\" class=\"data row56 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row56_col4\" class=\"data row56 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row57\" class=\"row_heading level0 row57\" >57</th>\n",
       "      <td id=\"T_5e81f_row57_col0\" class=\"data row57 col0\" >returns_1_unit_price</td>\n",
       "      <td id=\"T_5e81f_row57_col1\" class=\"data row57 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row57_col2\" class=\"data row57 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row57_col3\" class=\"data row57 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row57_col4\" class=\"data row57 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row58\" class=\"row_heading level0 row58\" >58</th>\n",
       "      <td id=\"T_5e81f_row58_col0\" class=\"data row58 col0\" >returns_customer_signature_is_signed</td>\n",
       "      <td id=\"T_5e81f_row58_col1\" class=\"data row58 col1\" >True</td>\n",
       "      <td id=\"T_5e81f_row58_col2\" class=\"data row58 col2\" >True</td>\n",
       "      <td id=\"T_5e81f_row58_col3\" class=\"data row58 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row58_col4\" class=\"data row58 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row59\" class=\"row_heading level0 row59\" >59</th>\n",
       "      <td id=\"T_5e81f_row59_col0\" class=\"data row59 col0\" >returns_customer_signature_signatory</td>\n",
       "      <td id=\"T_5e81f_row59_col1\" class=\"data row59 col1\" >Sarah H</td>\n",
       "      <td id=\"T_5e81f_row59_col2\" class=\"data row59 col2\" >Sarah H</td>\n",
       "      <td id=\"T_5e81f_row59_col3\" class=\"data row59 col3\" >54.80%</td>\n",
       "      <td id=\"T_5e81f_row59_col4\" class=\"data row59 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row60\" class=\"row_heading level0 row60\" >60</th>\n",
       "      <td id=\"T_5e81f_row60_col0\" class=\"data row60 col0\" >returns_vendor_signature_is_signed</td>\n",
       "      <td id=\"T_5e81f_row60_col1\" class=\"data row60 col1\" >True</td>\n",
       "      <td id=\"T_5e81f_row60_col2\" class=\"data row60 col2\" >True</td>\n",
       "      <td id=\"T_5e81f_row60_col3\" class=\"data row60 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row60_col4\" class=\"data row60 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row61\" class=\"row_heading level0 row61\" >61</th>\n",
       "      <td id=\"T_5e81f_row61_col0\" class=\"data row61 col0\" >returns_vendor_signature_signatory</td>\n",
       "      <td id=\"T_5e81f_row61_col1\" class=\"data row61 col1\" >James T</td>\n",
       "      <td id=\"T_5e81f_row61_col2\" class=\"data row61 col2\" >James T</td>\n",
       "      <td id=\"T_5e81f_row61_col3\" class=\"data row61 col3\" >86.50%</td>\n",
       "      <td id=\"T_5e81f_row61_col4\" class=\"data row61 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row62\" class=\"row_heading level0 row62\" >62</th>\n",
       "      <td id=\"T_5e81f_row62_col0\" class=\"data row62 col0\" >shipping_address</td>\n",
       "      <td id=\"T_5e81f_row62_col1\" class=\"data row62 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row62_col2\" class=\"data row62 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row62_col3\" class=\"data row62 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row62_col4\" class=\"data row62 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row63\" class=\"row_heading level0 row63\" >63</th>\n",
       "      <td id=\"T_5e81f_row63_col0\" class=\"data row63 col0\" >subtotal</td>\n",
       "      <td id=\"T_5e81f_row63_col1\" class=\"data row63 col1\" >293.520000</td>\n",
       "      <td id=\"T_5e81f_row63_col2\" class=\"data row63 col2\" >293.520000</td>\n",
       "      <td id=\"T_5e81f_row63_col3\" class=\"data row63 col3\" >99.40%</td>\n",
       "      <td id=\"T_5e81f_row63_col4\" class=\"data row63 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row64\" class=\"row_heading level0 row64\" >64</th>\n",
       "      <td id=\"T_5e81f_row64_col0\" class=\"data row64 col0\" >total_discount</td>\n",
       "      <td id=\"T_5e81f_row64_col1\" class=\"data row64 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row64_col2\" class=\"data row64 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row64_col3\" class=\"data row64 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row64_col4\" class=\"data row64 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row65\" class=\"row_heading level0 row65\" >65</th>\n",
       "      <td id=\"T_5e81f_row65_col0\" class=\"data row65 col0\" >total_item_quantity</td>\n",
       "      <td id=\"T_5e81f_row65_col1\" class=\"data row65 col1\" >27.000000</td>\n",
       "      <td id=\"T_5e81f_row65_col2\" class=\"data row65 col2\" >27.000000</td>\n",
       "      <td id=\"T_5e81f_row65_col3\" class=\"data row65 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row65_col4\" class=\"data row65 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row66\" class=\"row_heading level0 row66\" >66</th>\n",
       "      <td id=\"T_5e81f_row66_col0\" class=\"data row66 col0\" >total_return_quantity</td>\n",
       "      <td id=\"T_5e81f_row66_col1\" class=\"data row66 col1\" >2.000000</td>\n",
       "      <td id=\"T_5e81f_row66_col2\" class=\"data row66 col2\" >2.000000</td>\n",
       "      <td id=\"T_5e81f_row66_col3\" class=\"data row66 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row66_col4\" class=\"data row66 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row67\" class=\"row_heading level0 row67\" >67</th>\n",
       "      <td id=\"T_5e81f_row67_col0\" class=\"data row67 col0\" >total_tax</td>\n",
       "      <td id=\"T_5e81f_row67_col1\" class=\"data row67 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row67_col2\" class=\"data row67 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row67_col3\" class=\"data row67 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row67_col4\" class=\"data row67 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row68\" class=\"row_heading level0 row68\" >68</th>\n",
       "      <td id=\"T_5e81f_row68_col0\" class=\"data row68 col0\" >vendor_address</td>\n",
       "      <td id=\"T_5e81f_row68_col1\" class=\"data row68 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row68_col2\" class=\"data row68 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row68_col3\" class=\"data row68 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row68_col4\" class=\"data row68 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row69\" class=\"row_heading level0 row69\" >69</th>\n",
       "      <td id=\"T_5e81f_row69_col0\" class=\"data row69 col0\" >vendor_name</td>\n",
       "      <td id=\"T_5e81f_row69_col1\" class=\"data row69 col1\" >NEXGEN</td>\n",
       "      <td id=\"T_5e81f_row69_col2\" class=\"data row69 col2\" >NEXGEN</td>\n",
       "      <td id=\"T_5e81f_row69_col3\" class=\"data row69 col3\" >99.50%</td>\n",
       "      <td id=\"T_5e81f_row69_col4\" class=\"data row69 col4\" >Match</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_5e81f_level0_row70\" class=\"row_heading level0 row70\" >70</th>\n",
       "      <td id=\"T_5e81f_row70_col0\" class=\"data row70 col0\" >vendor_tax_id</td>\n",
       "      <td id=\"T_5e81f_row70_col1\" class=\"data row70 col1\" >None</td>\n",
       "      <td id=\"T_5e81f_row70_col2\" class=\"data row70 col2\" >None</td>\n",
       "      <td id=\"T_5e81f_row70_col3\" class=\"data row70 col3\" >0.00%</td>\n",
       "      <td id=\"T_5e81f_row70_col4\" class=\"data row70 col4\" >Match</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f91874c8da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Display the outputs of the data extraction process.\n",
    "df = pd.DataFrame([\n",
    "    {\n",
    "        \"Accuracy\": f\"{accuracy['overall'] * 100:.2f}%\",\n",
    "        \"Confidence\": f\"{confidence['_overall'] * 100:.2f}%\",\n",
    "        \"Execution Time\": f\"{total_elapsed:.2f} seconds\",\n",
    "        \"Document Intelligence Execution Time\": f\"{di_stopwatch.elapsed:.2f} seconds\",\n",
    "        \"Phi Execution Time\": f\"{phi_stopwatch.elapsed:.2f} seconds\",\n",
    "        \"Prompt Tokens\": prompt_tokens,\n",
    "        \"Completion Tokens\": completion_tokens\n",
    "    }\n",
    "])\n",
    "\n",
    "display(df)\n",
    "display(get_extraction_comparison(expected_dict, invoice_dict, confidence, accuracy['accuracy']))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}