L2-Document_splitting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "571cf802",
   "metadata": {},
   "source": [
    "# Document Splitting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ead2cd5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import openai\n",
    "import sys\n",
    "sys.path.append('../..')\n",
    "\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "_ = load_dotenv(find_dotenv()) # read local .env file\n",
    "\n",
    "openai.api_key  = os.environ['OPENAI_API_KEY']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "30c7fd47",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter, CharacterTextSplitter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e45edd4b",
   "metadata": {},
   "outputs": [],
   "source": [
    "chunk_size =26\n",
    "chunk_overlap = 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c99770e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "r_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=chunk_size,\n",
    "    chunk_overlap=chunk_overlap\n",
    ")\n",
    "c_splitter = CharacterTextSplitter(\n",
    "    chunk_size=chunk_size,\n",
    "    chunk_overlap=chunk_overlap\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "631c7a66",
   "metadata": {},
   "source": [
    "Why doesn't this split the string below?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a7339b40",
   "metadata": {},
   "outputs": [],
   "source": [
    "text1 = 'abcdefghijklmnopqrstuvwxyz'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f1ff93e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['abcdefghijklmnopqrstuvwxyz']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_splitter.split_text(text1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "70c56dc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "text2 = 'abcdefghijklmnopqrstuvwxyzabcdefg'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e1919c3b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['abcdefghijklmnopqrstuvwxyz', 'wxyzabcdefg']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_splitter.split_text(text2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43d37682",
   "metadata": {},
   "source": [
    "Ok, this splits the string but we have an overlap specified as 5, but it looks like 3? (try an even number)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "20f2af86",
   "metadata": {},
   "outputs": [],
   "source": [
    "text3 = \"a b c d e f g h i j k l m n o p q r s t u v w x y z\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "78cde542",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a b c d e f g h i j k l m', 'l m n o p q r s t u v w x', 'w x y z']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_splitter.split_text(text3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "a2adb949",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a b c d e f g h i j k l m n o p q r s t u v w x y z']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c_splitter.split_text(text3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "112e208b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a b c d e f g h i j k l m', 'l m n o p q r s t u v w x', 'w x y z']"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c_splitter = CharacterTextSplitter(\n",
    "    chunk_size=chunk_size,\n",
    "    chunk_overlap=chunk_overlap,\n",
    "    separator = ' '\n",
    ")\n",
    "c_splitter.split_text(text3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "763577fd",
   "metadata": {},
   "source": [
    "## Recursive splitting details\n",
    "RecursiveCharacterTextSplitter is recommended for generic text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c5483855",
   "metadata": {},
   "outputs": [],
   "source": [
    "some_text = \"\"\"When writing documents, writers will use document structure to group content. \\\n",
    "This can convey to the reader, which idea's are related. For example, closely related ideas \\\n",
    "are in sentances. Similar ideas are in paragraphs. Paragraphs form a document. \\n\\n  \\\n",
    "Paragraphs are often delimited with a carriage return or two carriage returns. \\\n",
    "Carriage returns are the \"backslash n\" you see embedded in this string. \\\n",
    "Sentences have a period at the end, but also, have a space.\\\n",
    "and words are separated by space.\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "cd274410",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "496"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(some_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "b72c2e6d",
   "metadata": {},
   "outputs": [],
   "source": [
    "c_splitter = CharacterTextSplitter(\n",
    "    chunk_size=450,\n",
    "    chunk_overlap=0,\n",
    "    separator = ' '\n",
    ")\n",
    "r_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=450,\n",
    "    chunk_overlap=0, \n",
    "    separators=[\"\\n\\n\", \"\\n\", \" \", \"\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2c57f284",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['When writing documents, writers will use document structure to group content. This can convey to the reader, which idea\\'s are related. For example, closely related ideas are in sentances. Similar ideas are in paragraphs. Paragraphs form a document. \\n\\n Paragraphs are often delimited with a carriage return or two carriage returns. Carriage returns are the \"backslash n\" you see embedded in this string. Sentences have a period at the end, but also,',\n",
       " 'have a space.and words are separated by space.']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c_splitter.split_text(some_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "53d8d6d9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"When writing documents, writers will use document structure to group content. This can convey to the reader, which idea's are related. For example, closely related ideas are in sentances. Similar ideas are in paragraphs. Paragraphs form a document.\",\n",
       " 'Paragraphs are often delimited with a carriage return or two carriage returns. Carriage returns are the \"backslash n\" you see embedded in this string. Sentences have a period at the end, but also, have a space.and words are separated by space.']"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_splitter.split_text(some_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "019f0890",
   "metadata": {},
   "source": [
    "Let's reduce the chunk size a bit and add a period to our separators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0fe3543d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"When writing documents, writers will use document structure to group content. This can convey to the reader, which idea's are related\",\n",
       " '. For example, closely related ideas are in sentances. Similar ideas are in paragraphs. Paragraphs form a document.',\n",
       " 'Paragraphs are often delimited with a carriage return or two carriage returns',\n",
       " '. Carriage returns are the \"backslash n\" you see embedded in this string',\n",
       " '. Sentences have a period at the end, but also, have a space.and words are separated by space.']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=150,\n",
    "    chunk_overlap=0,\n",
    "    separators=[\"\\n\\n\", \"\\n\", \"\\. \", \" \", \"\"]\n",
    ")\n",
    "r_splitter.split_text(some_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "de1737bd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"When writing documents, writers will use document structure to group content. This can convey to the reader, which idea's are related.\",\n",
       " 'For example, closely related ideas are in sentances. Similar ideas are in paragraphs. Paragraphs form a document.',\n",
       " 'Paragraphs are often delimited with a carriage return or two carriage returns.',\n",
       " 'Carriage returns are the \"backslash n\" you see embedded in this string.',\n",
       " 'Sentences have a period at the end, but also, have a space.and words are separated by space.']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=150,\n",
    "    chunk_overlap=0,\n",
    "    separators=[\"\\n\\n\", \"\\n\", \"(?<=\\. )\", \" \", \"\"]\n",
    ")\n",
    "r_splitter.split_text(some_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "b8cd92d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import PyPDFLoader\n",
    "loader = PyPDFLoader(\"docs/cs229_lectures/MachineLearning-Lecture01.pdf\")\n",
    "pages = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "c286fa1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "text_splitter = CharacterTextSplitter(\n",
    "    separator=\"\\n\",\n",
    "    chunk_size=1000,\n",
    "    chunk_overlap=150,\n",
    "    length_function=len\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "2097d477",
   "metadata": {},
   "outputs": [],
   "source": [
    "docs = text_splitter.split_documents(pages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "cb0eb13b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "77"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(docs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "8cdaaf29",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "22"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(pages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "618ebb0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import NotionDirectoryLoader\n",
    "loader = NotionDirectoryLoader(\"docs/Notion_DB\")\n",
    "notion_db = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "25c93697",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(notion_db)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "ac709181",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1557"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "233d004f",
   "metadata": {},
   "source": [
    "## Token splitting\n",
    "We can also split on token count explicity, if we want.\n",
    "\n",
    "This can be useful because LLMs often have context windows designated in tokens.\n",
    "\n",
    "Tokens are often ~4 characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "b2ac9766",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import TokenTextSplitter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "557001e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = TokenTextSplitter(chunk_size=1, chunk_overlap=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "36fcf5ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "text1 = \"foo bar bazzyfoo\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "bbd53700",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['foo', ' bar', ' b', 'az', 'zy', 'foo']"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_splitter.split_text(text1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "7329eec4",
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = TokenTextSplitter(chunk_size=10, chunk_overlap=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "bb51f9d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "docs = text_splitter.split_documents(pages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "1b9ffb32",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='MachineLearning-Lecture01  \\n', metadata={'source': 'docs/cs229_lectures/MachineLearning-Lecture01.pdf', 'page': 0})"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "82a84448",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'source': 'docs/cs229_lectures/MachineLearning-Lecture01.pdf', 'page': 0}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pages[0].metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "648df746",
   "metadata": {},
   "source": [
    "## Context aware splitting\n",
    "Chunking aims to keep text with common context together.\n",
    "\n",
    "A text splitting often uses sentences or other delimiters to keep related text together but many documents (such as Markdown) have structure (headers) that can be explicitly used in splitting.\n",
    "\n",
    "We can use MarkdownHeaderTextSplitter to preserve header metadata in our chunks, as show below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "d0b4fa64",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import NotionDirectoryLoader\n",
    "from langchain.text_splitter import MarkdownHeaderTextSplitter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "fdb6db95",
   "metadata": {},
   "outputs": [],
   "source": [
    "markdown_document = \"\"\"# Title\\n\\n \\\n",
    "## Chapter 1\\n\\n \\\n",
    "Hi this is Jim\\n\\n Hi this is Joe\\n\\n \\\n",
    "### Section \\n\\n \\\n",
    "Hi this is Lance \\n\\n \n",
    "## Chapter 2\\n\\n \\\n",
    "Hi this is Molly\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "4d1bc544",
   "metadata": {},
   "outputs": [],
   "source": [
    "headers_to_split_on = [\n",
    "    (\"#\", \"Header 1\"),\n",
    "    (\"##\", \"Header 2\"),\n",
    "    (\"###\", \"Header 3\"),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "db0ab41f",
   "metadata": {},
   "outputs": [],
   "source": [
    "markdown_splitter = MarkdownHeaderTextSplitter(\n",
    "    headers_to_split_on=headers_to_split_on\n",
    ")\n",
    "md_header_splits = markdown_splitter.split_text(markdown_document)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "e771191d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'content': 'Hi this is Jim  \\nHi this is Joe',\n",
       " 'metadata': {'Header 1': 'Title', 'Header 2': 'Chapter 1'}}"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md_header_splits[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "391ae033",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'content': 'Hi this is Lance',\n",
       " 'metadata': {'Header 1': 'Title',\n",
       "  'Header 2': 'Chapter 1',\n",
       "  'Header 3': 'Section'}}"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md_header_splits[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a2cea1e",
   "metadata": {},
   "source": [
    "Try on a real Markdown file, like a Notion database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "76de0869",
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = NotionDirectoryLoader(\"docs/Notion_DB\")\n",
    "docs = loader.load()\n",
    "txt = ' '.join([d.page_content for d in docs])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "ccf84da7",
   "metadata": {},
   "outputs": [],
   "source": [
    "headers_to_split_on = [\n",
    "    (\"#\", \"Header 1\"),\n",
    "    (\"##\", \"Header 2\"),\n",
    "]\n",
    "markdown_splitter = MarkdownHeaderTextSplitter(\n",
    "    headers_to_split_on=headers_to_split_on\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "c89d678c",
   "metadata": {},
   "outputs": [],
   "source": [
    "md_header_splits = markdown_splitter.split_text(txt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "33c7c438",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'content': \"We kick off with the practical side of things and then dig into the idea behind it.  \\n- **How *time off* works at Blendle**\\n- Time off is about the time you **need,** not about a **quota.**\\n- At Blendle, **HR doesn't keep track** of your holidays **and we don't 'pay out' at the end of the ride.** When in doubt: 4-6 weeks is a good bandwidth. Less than that is not enough, more than that can happen, just check with your lead if you're in doubt if it's reasonable.\\n- **We stick to the commonly used national holidays**, which comes down to ~8 days per year. We are a startup and there are teams who have work to be done 24/7. We don't like being told whether we are off or not on Eid al-Fitr (Suikerfeest, ending of Ramadan) by some rules someone made up, so this is a guideline we use: feel free to work or be off when you want.\\n- Make sure to **take enough moments of rest when you have periods of working hard**. For example: worked a few nights to finish a project? Go home at 12:00 the next Monday to have lunch with a friend and go to the gym (or have a New Girl-marathon). It's about balance and flexibility.\\n- You agree on your time off with your team(lead).\\n- You put your time off in the agenda: add it to your own and all@ agenda.\\n- A life event occurs: communicate what you need and check with your team(lead) when in doubt. 'Hey I just had a baby, won't be at work this week, see you next week. His name is Jan by the way!'.\\n- Enjoy: don't worry if you go to the gym or get a haircut during classic 'work hours'. We trust you to make the best of your days for Blendle.\\n- **This only works when**\\n- Flexibility goes both ways.\\n- You make sure to take enough time off: since you don't get paid at the end of the ride.\\n- You communicate in a crystal clear manner: agreeing the dates with others and putting it in the agenda.\\n- **Backdrop**  \\nThe law and additional ruling tell us what we can and can not do according to time off. We believe that is a very limited way of thinking about time off. We want you to think about what you actually need. Starting with two important pillars:  \\n- We work hard, we need time to reload and rest. Sometimes we work harder, so we need more. Some times we work less, so we need less.\\n- We have something called 'life' happening which means we can't work because we have more important stuff to do.  \\nWe want you to take enough rest to reload and recharge so you can be at your prime when working. Taking enough rest is as important as giving it your all when working. The well known on and off switch.  \\nThe concept of work isn't always caught in time (40 hours) and place. Most of the people at Blendle don't keep track of time because time is such a limited variable to grasp the concept of work. There is a balance to strike here and just looking at hours or a holiday quota limits this discussion.  \\n**That is why we don't keep track of holidays.**  \\nOur people don't have a holiday quota, but a moral obligation to their loved ones and Blendle to work hard and take enough time to rest. Alright, sounds heavy. How does that work?  \\n'Time off' can be divided into two buckets:  \\n1. **Holidays and national holidays**:  \\nYour contract states you have 4 weeks of paid time off per year, by law.  \\nOn average we have 8 national holidays per year. We have no CAO (collective employment agreement), which means you are not entitled to paid time off on every national holiday. We decided to follow the most common situation in an average company, which in practice means most people are off on the big national holidays. But that's your own choice, it's not a right or an obligation, so feel free to improvise. Some teams even have to (editorial and support, for example).  \\nThese rules state that we are off on 'Bevrijdingsdag' once every 5 years. But we aren't allowed time off on Eid al-Fitr (Suikerfeest, ending of the Ramadan). That makes no sense to us.  \\nSo here is how we approach holidays and national holidays in practice.  \\nYou 'have' 5 weeks holiday and ~8 national holidays per year. We want you to use this time off to reload but you decide when and how to consume this. You must also feel free to improvise. You work hard, maybe even a bit too hard, so you might need some extra time off to stay happy and healthy. This also works the other way around: maybe you had a chill quarter and feel no need to take a holiday: feel free to do so. We want you to think about what your body and mind needs instead of your quota.  \\n2. **Life events:**  \\nLife happens. Your cat dies, your car broke, you move to another city, you get married or have kids. Life events which all need your attention. We decided that we are not going to decide for you how much time you **get** for certain events. The question is: how much time do you actually **need.**  \\nMost companies don't give you time off for funerals that are not for immediate relatives. We think we don't need a rule to tell us if that funeral is important for you. So take the time you need.  \\n- Edge cases\\n- Sabbatical: check with HR if you for example have plans to take a very long trip (longer then ~4 weeks).\",\n",
       " 'metadata': {'Header 1': 'Time off: holidays and national holidays'}}"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md_header_splits[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e4ca2850",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}