blink_detection_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Blink Detection using MediaPipe and OpenCV\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we will detect human eye blinks in real-time using a metric called the Eye Aspect Ratio (EAR). A blink is identified when the EAR falls below a specified threshold for a certain number of consecutive frames. \n",
    "\n",
    "We will explore how to fine-tune the EAR threshold and the frame count parameter to optimize blink detection accuracy. Additionally, we will demonstrate the impact of these parameters on the system's performance and plot the EAR in real-time while counting blinks. This visualization will provide valuable insights into the blink detection process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Content\n",
    "\n",
    "1. [Imports](#1-importing-necessary-libraries)\n",
    "2. [Eye Landmark Detection](#2-eye-landmark-detection)\n",
    "3. [Drawing specific EAR Landmarks](#3-drawing-the-specific-ear-landmarks)\n",
    "4. [EAR Analysis](#4-analyzing-the-eye-aspect-ratioear)\n",
    "    - [Blink Detection with Multiple Thresholds](#41-blink-detection-with-multiple-ear-thresholds)\n",
    "    - [Blink Detection with varying Consecutive Frames](#42-blink-detection-with-varying-consecutive-frames)\n",
    "5. [Blink Detection and Counting](#5-blink-detection-and-counting)\n",
    "6. [Real-Time Blink Detection with EAR plotting](#6-real-time-eye-blink-detection-and-visualization-with-ear-plotting)\n",
    "7. [Applications](#7-applications)\n",
    "8. [References](#8-references)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Importing necessary libraries\n",
    "\n",
    "Let's start off by importing some libraries that will be required during the blink detection process.\n",
    "\n",
    "> - **MediaPipe** : face mesh detection and landmark extraction\n",
    "> - **OpenCv**: image and video processing\n",
    "> - **Matplotlib** : for plotting and visualizing data\n",
    "> - **FaceMeshModule** : for generating face mesh and landmarks\n",
    "> - **Plotly**: for interactive and dynamic plotting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mediapipe as mp\n",
    "import cv2 as cv\n",
    "import numpy as np\n",
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas\n",
    "from FaceMeshModule import FaceMeshGenerator\n",
    "from utils import DrawingUtils\n",
    "import plotly.graph_objects as go\n",
    "from plotly.subplots import make_subplots\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Eye Landmark Detection\n",
    "To enable blink detection, the first crucial step is to identify the landmarks of the eyes in the input image. This class, `DetectEyeLandmarks`, leverages MediaPipe's FaceMesh to extract facial landmarks and specifically highlights the eye regions.\n",
    "\n",
    "Key Features:\n",
    "- Detects eye landmarks for both right and left eyes.\n",
    "- Visualizes landmarks by overlaying colored circles on the input image.\n",
    "- Saves the processed image if desired.\n",
    "\n",
    "Parameters:\n",
    "- **image_path**: Path to the input image.\n",
    "- **save_img**: Set to `True` if you want to save the annotated image.\n",
    "- **filename**: Name of the output file for saving.\n",
    "\n",
    "Below is the implementation of the `DetectEyeLandmarks` class.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DetectEyeLandmarks:\n",
    "    # Eye landmark indices\n",
    "    RIGHT_EYE = [33, 7, 163, 144, 145, 153, 154, 155, 133, 173, 157, 158, 159, 160, 161, 246]\n",
    "    LEFT_EYE = [362, 382, 381, 380, 374, 373, 390, 249, 263, 466, 388, 387, 386, 385, 384, 398]\n",
    "    \n",
    "    # Colors\n",
    "    GREEN_COLOR = (0, 255, 0)  # Right eye color\n",
    "    RED_COLOR = (0, 0, 255)   # Left eye color\n",
    "    \n",
    "    def __init__(self, image_path, save_img=False, filename=None):\n",
    "        \"\"\"\n",
    "        Initialize the eye landmark detector.\n",
    "        \n",
    "        Args:\n",
    "            image_path (str): Path to input image\n",
    "            save_img (bool): Whether to save the processed image\n",
    "            filename (str): Output filename if saving image\n",
    "        \"\"\"\n",
    "        self.image_path = image_path\n",
    "        self.filename = filename\n",
    "        self.save_img = save_img\n",
    "        self.generator = FaceMeshGenerator()\n",
    "        \n",
    "        # Setup save directory if needed\n",
    "        if self.save_img:\n",
    "            self._setup_save_directory()\n",
    "            \n",
    "        # Process image\n",
    "        self.image = self._load_image()\n",
    "        if self.image is not None:\n",
    "            self._process_image()\n",
    "    \n",
    "    def _setup_save_directory(self):\n",
    "        \"\"\"Create directory for saving images if it doesn't exist.\"\"\"\n",
    "        if not self.filename:\n",
    "            raise ValueError(\"Filename must be provided when save_img is True\")\n",
    "            \n",
    "        save_dir = \"DATA/IMAGES/BLINK_DETECTION\"\n",
    "        os.makedirs(save_dir, exist_ok=True)\n",
    "        self.filename = os.path.join(save_dir, self.filename)\n",
    "\n",
    "    def _load_image(self):\n",
    "        \"\"\"Load and validate input image.\"\"\"\n",
    "        try:\n",
    "            img = cv.imread(self.image_path)\n",
    "            if img is None:\n",
    "                raise FileNotFoundError(f\"Could not load image from {self.image_path}\")\n",
    "            return img\n",
    "        except Exception as e:\n",
    "            print(f\"Error loading image: {str(e)}\")\n",
    "            return None\n",
    "\n",
    "    def _process_image(self):\n",
    "        \"\"\"Process image to detect and draw landmarks.\"\"\"\n",
    "        try:\n",
    "            # Detect face landmarks\n",
    "            self.image, face_landmarks = self.generator.create_face_mesh(self.image, draw=False)\n",
    "            \n",
    "            if not face_landmarks:\n",
    "                raise ValueError(\"No face detected in the image\")\n",
    "                \n",
    "            # Draw landmarks for both eyes\n",
    "            self._draw_eye_landmarks(face_landmarks, self.RIGHT_EYE, self.GREEN_COLOR)\n",
    "            self._draw_eye_landmarks(face_landmarks, self.LEFT_EYE, self.RED_COLOR)\n",
    "            \n",
    "            # Display and save results\n",
    "            self.plot_eye_landmarks()\n",
    "            if self.save_img:\n",
    "                self._save_image()\n",
    "                \n",
    "        except Exception as e:\n",
    "            print(f\"Error processing image: {str(e)}\")\n",
    "\n",
    "    def _draw_eye_landmarks(self, landmarks, eye_landmarks, color):\n",
    "        \"\"\"\n",
    "        Draw circles on the eye landmarks and display their indices.\n",
    "        \n",
    "        Args:\n",
    "            landmarks: Detected face landmarks\n",
    "            eye_landmarks: List of landmark indices for an eye\n",
    "            color: Color to draw the landmarks\n",
    "        \"\"\"\n",
    "        for idx in eye_landmarks:\n",
    "            # Draw the circle for the landmark\n",
    "            cv.circle(self.image, (landmarks[idx]), 6, color, cv.FILLED)\n",
    "            \n",
    "            # Add the landmark index as text above the circle\n",
    "            text_pos = (landmarks[idx][0] - 10, landmarks[idx][1] - 10)  # Position text above circle\n",
    "            cv.putText(self.image, str(idx), text_pos, \n",
    "                      cv.FONT_HERSHEY_SIMPLEX, 0.6, (255, 255, 255), 2)\n",
    "\n",
    "    def plot_eye_landmarks(self):\n",
    "        \"\"\"Display the image with eye landmarks.\"\"\"\n",
    "        plt.figure(figsize=(10, 10))\n",
    "        img_rgb = cv.cvtColor(self.image, cv.COLOR_BGR2RGB)\n",
    "        plt.imshow(img_rgb)\n",
    "        plt.axis('off')\n",
    "        plt.show()\n",
    "        \n",
    "    def _save_image(self):\n",
    "        \"\"\"Save the processed image.\"\"\"\n",
    "        try:\n",
    "            cv.imwrite(self.filename, self.image)\n",
    "            print(f\"Image saved successfully to {self.filename}\")\n",
    "        except Exception as e:\n",
    "            print(f\"Error saving image: {str(e)}\")\n",
    "\n",
    "# Example usage\n",
    "if __name__ == \"__main__\":\n",
    "    try:\n",
    "        image_path = \"DATA/IMAGES/DOWNLOADED_IMAGES/eyes_2.png\"\n",
    "        eye_landmarks = DetectEyeLandmarks(\n",
    "            image_path, \n",
    "            save_img=True, filename=\"eye_landmarks_3.jpg\"  \n",
    "    )\n",
    "    except Exception as e:\n",
    "        print(f\"Failed to process image: {str(e)}\")\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Notes\n",
    "1. **Right Eye vs. Left Eye**:\n",
    "   - The code uses predefined indices for landmarks specific to the right and left eyes based on MediaPipe's FaceMesh model.\n",
    "\n",
    "2. **Visualization**:\n",
    "   - Green circles represent the right eye landmarks.\n",
    "   - Red circles represent the left eye landmarks.\n",
    "   - Landmark indices are displayed as text near each point.\n",
    "\n",
    "3. **Error Handling**:\n",
    "   - Handles errors for image loading and face detection, ensuring smooth execution.\n",
    "\n",
    "4. **Next Step**:\n",
    "   - After successfully detecting the eye landmarks, the next step is to compute the Eye Aspect Ratio (EAR). We fist highlighted the specific eye landmarks in both eyes which are taken into account for the calculation of EAR, after that we have calculated the EAR based on the formula mentioned in the next section\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Drawing the specific EAR landmarks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Eye Aspect Ratio (EAR) is a crucial metric in computer vision, used to measure the distance between key eye landmarks. It helps distinguish between regular blinks and prolonged eye closures, which can indicate drowsiness or fatigue. This metric is widely used in applications such as facial movement analysis, signal processing, and driver monitoring systems.\n",
    "\n",
    "**Calculation**\n",
    "\n",
    "The EAR is calculated by measuring the distances between the upper and lower eyelids and the corners of the eye. The formula, derived from the work by Soukupová and Čech in their 2016 paper, [*Real-Time Eye Blink Detection using Facial Landmarks*](https://vision.fe.uni-lj.si/cvww2016/proceedings/papers/05.pdf), is as follows:\n",
    "\n",
    "$$ \\text{EAR} = \\frac{(p_2 - p_6) + (p_3 - p_5)}{2 \\times (p_1 - p_4)} $$\n",
    "\n",
    "where:\n",
    "\n",
    "- $ p_1, p_2, p_3, p_4, p_5 $ and $p_6$ are the coordinates of 2D facial landmarks, such as the eyebrow, eyelid, and eye corner.\n",
    "- The numerator calculates the vertical distance between the lower and upper eyelids.\n",
    "- The denominator calculates the horizontal distance between the eye corners, weighted by 2, as there are two sets of horizontal eye landmarks but only one set of vertical eye landmarks.\n",
    "\n",
    "The EAR remains relatively constant when the eyes are open but drops significantly when the eyes are closed, indicating a blink. However, a low EAR value doesn't always mean a blink. It could result from eye irritation, intentional eye closure, or facial expressions like smiling or yawning. Therefore, a larger temporal window is necessary to accurately declare a blink."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">The corresponding EAR points for left and right landmarks are\n",
    "> - RIGHT_EAR : [33, 159, 158, 133, 153, 145]\n",
    "> - LEFT_EAR : [362, 380, 374, 263, 386, 385]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code defines a class `DrawEARLandmarks` responsible for visualizing the specific eye landmarks involved in EAR calculation and displaying the EAR value at the top-left corner of the frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DrawEARLandmarks:\n",
    "    # Eye landmark indices\n",
    "    LEFT_EYE_EAR = [362, 380, 374, 263, 386, 385]\n",
    "    RIGHT_EYE_EAR = [33, 159, 158, 133, 153, 145]\n",
    "    \n",
    "    # Colors\n",
    "    GREEN_COLOR = (0, 255, 0)  # Right eye color\n",
    "    RED_COLOR = (0, 0, 255)   # Left eye color\n",
    "    \n",
    "    def __init__(self, image_path, save_img=False, filename=None):\n",
    "        \"\"\"\n",
    "        Initialize the eye landmark detector.\n",
    "        \n",
    "        Args:\n",
    "            image_path (str): Path to input image\n",
    "            save_img (bool): Whether to save the processed image\n",
    "            filename (str): Output filename if saving image\n",
    "        \"\"\"\n",
    "        self.image_path = image_path\n",
    "        self.filename = filename\n",
    "        self.save_img = save_img\n",
    "        self.generator = FaceMeshGenerator()\n",
    "        \n",
    "        # Setup save directory if needed\n",
    "        if self.save_img:\n",
    "            self._setup_save_directory()\n",
    "            \n",
    "        # Process image\n",
    "        self.image = self._load_image()\n",
    "        if self.image is not None:\n",
    "            self._process_image()\n",
    "    \n",
    "    def _setup_save_directory(self):\n",
    "        \"\"\"Create directory for saving images if it doesn't exist.\"\"\"\n",
    "        if not self.filename:\n",
    "            raise ValueError(\"Filename must be provided when save_img is True\")\n",
    "            \n",
    "        save_dir = \"DATA/IMAGES/BLINK_DETECTION\"\n",
    "        os.makedirs(save_dir, exist_ok=True)\n",
    "        self.filename = os.path.join(save_dir, self.filename)\n",
    "\n",
    "    def _load_image(self):\n",
    "        \"\"\"Load and validate input image.\"\"\"\n",
    "        try:\n",
    "            img = cv.imread(self.image_path)\n",
    "            if img is None:\n",
    "                raise FileNotFoundError(f\"Could not load image from {self.image_path}\")\n",
    "            return img\n",
    "        except Exception as e:\n",
    "            print(f\"Error loading image: {str(e)}\")\n",
    "            return None\n",
    "\n",
    "    def _eye_aspect_ratio(self, landmarks, eye_landmarks):\n",
    "        \"\"\"\n",
    "        Calculate the Eye Aspect Ratio (EAR) for a given eye.\n",
    "        \n",
    "        Args:\n",
    "            landmarks (list): Detected face landmarks\n",
    "            eye_landmarks (list): List of landmark indices for an eye\n",
    "        \n",
    "        Returns:\n",
    "            float: The calculated EAR value\n",
    "        \"\"\"\n",
    "        A = np.linalg.norm(np.array(landmarks[eye_landmarks[1]]) - np.array(landmarks[eye_landmarks[5]]))\n",
    "        B = np.linalg.norm(np.array(landmarks[eye_landmarks[2]]) - np.array(landmarks[eye_landmarks[4]]))\n",
    "        C = np.linalg.norm(np.array(landmarks[eye_landmarks[0]]) - np.array(landmarks[eye_landmarks[3]]))\n",
    "        ear = (A + B) / (2.0 * C)\n",
    "        return ear\n",
    "    \n",
    "    def _draw_eye_landmarks(self, landmarks, eye_landmarks, color):\n",
    "        \"\"\"\n",
    "        Draw circles on the eye landmarks and display their indices.\n",
    "        \n",
    "        Args:\n",
    "            landmarks: Detected face landmarks\n",
    "            eye_landmarks: List of landmark indices for an eye\n",
    "            color: Color to draw the landmarks\n",
    "        \"\"\"\n",
    "        for idx in eye_landmarks:\n",
    "            # Draw the circle for the landmark\n",
    "            cv.circle(self.image, (landmarks[idx]), 8, color, cv.FILLED)\n",
    "\n",
    "        cv.line(self.image, landmarks[eye_landmarks[1]], landmarks[eye_landmarks[5]], (255, 255, 255), 4)\n",
    "        cv.line(self.image, landmarks[eye_landmarks[2]], landmarks[eye_landmarks[4]], (255, 255, 255), 4)\n",
    "        cv.line(self.image, landmarks[eye_landmarks[0]], landmarks[eye_landmarks[3]], (255, 255, 255), 4)\n",
    "\n",
    "        left_ear = self._eye_aspect_ratio(landmarks, self.LEFT_EYE_EAR)\n",
    "        right_ear = self._eye_aspect_ratio(landmarks, self.RIGHT_EYE_EAR)\n",
    "        ear = (left_ear + right_ear) / 2.0\n",
    "\n",
    "        DrawingUtils.draw_text_with_bg(self.image,  f\"EAR : {ear:.2f}\", (0, 80), font_scale=2, thickness=3)\n",
    "\n",
    "    def _plot_eye_landmarks(self):\n",
    "        \"\"\"Display the image with eye landmarks.\"\"\"\n",
    "        plt.figure(figsize=(10, 10))\n",
    "        img_rgb = cv.cvtColor(self.image, cv.COLOR_BGR2RGB)\n",
    "        plt.imshow(img_rgb)\n",
    "        plt.axis('off')\n",
    "        plt.show()\n",
    "        \n",
    "    def _save_image(self):\n",
    "        \"\"\"Save the processed image.\"\"\"\n",
    "        try:\n",
    "            cv.imwrite(self.filename, self.image)\n",
    "            print(f\"Image saved successfully to {self.filename}\")\n",
    "        except Exception as e:\n",
    "            print(f\"Error saving image: {str(e)}\")\n",
    "\n",
    "    def _process_image(self):\n",
    "        \"\"\"Process image to detect and draw landmarks.\"\"\"\n",
    "        try:\n",
    "            # Detect face landmarks\n",
    "            self.image, face_landmarks = self.generator.create_face_mesh(self.image, draw=False)\n",
    "            \n",
    "            if not face_landmarks:\n",
    "                raise ValueError(\"No face detected in the image\")\n",
    "                \n",
    "            # Draw landmarks for both eyes\n",
    "            self._draw_eye_landmarks(face_landmarks, self.RIGHT_EYE_EAR, self.GREEN_COLOR)\n",
    "            self._draw_eye_landmarks(face_landmarks, self.LEFT_EYE_EAR, self.RED_COLOR)\n",
    "            \n",
    "            # Display and save results\n",
    "            self._plot_eye_landmarks()\n",
    "            if self.save_img:\n",
    "                self._save_image()\n",
    "                \n",
    "        except Exception as e:\n",
    "            print(f\"Error processing image: {str(e)}\")\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    image_path = \"DATA/IMAGES/DOWNLOADED_IMAGES/eyes_3.png\"\n",
    "    DrawEARLandmarks(image_path, \n",
    "                    save_img=True, \n",
    "                    filename=\"eye_landmarks_3.jpg\"  \n",
    "                )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DrawEARLandmarks(\"DATA/IMAGES/DOWNLOADED_IMAGES/eyes_1.jpg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Analyzing the Eye Aspect Ratio(EAR) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following code we processes a video to detect blinks by calculating the Eye Aspect Ratio (EAR) for both eyes in each frame. It uses a threshold-based approach to identify blinks and provides a real-time blink counter along with an EAR plot. \n",
    "\n",
    "Key Features:\n",
    "1. **EAR Calculation**:\n",
    "   - Computes EAR using vertical and horizontal eye distances.\n",
    "   - EAR below a threshold indicates a possible blink.\n",
    "\n",
    "2. **Blink Counter**:\n",
    "   - Tracks consecutive frames with EAR below the threshold to count blinks.\n",
    "\n",
    "3. **Visualization**:\n",
    "   - Highlights detected eye landmarks in real-time.\n",
    "   - Displays the EAR value and total blinks on the video.\n",
    "   - Once the video processing is completed it will plot the EAR value as measured in each frame\n",
    "\n",
    "4. **Output Options**:\n",
    "   - Saves the processed video with blink counter annotations.\n",
    "   - Generates a plot of EAR values over time.\n",
    "\n",
    "Below is the implementation of the `PlotEAR` class, which handles video processing, blink detection, and visualization.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PlotEAR:\n",
    "   \n",
    "    def __init__(self, video_path, save_video=False, video_output=None, \n",
    "                 save_plot=False, plot_output=None):\n",
    "        self.generator = FaceMeshGenerator() \n",
    "        self.video_path = video_path\n",
    "        \n",
    "        # Video saving parameters\n",
    "        self.save_video = save_video\n",
    "        self.video_output = video_output\n",
    "        \n",
    "        # Plot saving parameters\n",
    "        self.save_plot = save_plot\n",
    "        self.plot_output = plot_output\n",
    "        \n",
    "        # Eye landmarks\n",
    "        self.RIGHT_EYE = [33, 7, 163, 144, 145, 153, 154, 155, 133, 173, 157, 158, 159, 160, 161, 246] \n",
    "        self.LEFT_EYE = [362, 382, 381, 380, 374, 373, 390, 249, 263, 466, 388, 387, 386, 385, 384, 398]\n",
    "        self.RIGHT_EYE_EAR = [33, 159, 158, 133, 153, 145]\n",
    "        self.LEFT_EYE_EAR = [362, 380, 374, 263, 386, 385]\n",
    "        \n",
    "        # Blink detection parameters\n",
    "        self.EAR_THRESHOLD = 0.275\n",
    "        self.CONSEC_FRAMES = 4\n",
    "        self.blink_counter = 0\n",
    "        self.frame_counter = 0\n",
    "        \n",
    "        # Store EAR values for plotting\n",
    "        self.ear_values = []\n",
    "        self.frame_numbers = []\n",
    "        \n",
    "        # Colors (BGR format for OpenCV)\n",
    "        self.GREEN_COLOR = (86, 241, 13)    # Bright green\n",
    "        self.RED_COLOR = (30, 46, 209)      # Bright red\n",
    "\n",
    "        # Define the output video file\n",
    "        if self.save_video and self.video_output:\n",
    "            save_dir = \"DATA/VIDEOS/OUTPUTS\"\n",
    "            os.makedirs(save_dir, exist_ok=True)\n",
    "            self.video_output = os.path.join(save_dir, self.video_output)\n",
    "\n",
    "    def eye_aspect_ratio(self, eye_landmarks, landmarks):\n",
    "        A = np.linalg.norm(np.array(landmarks[eye_landmarks[1]]) - np.array(landmarks[eye_landmarks[5]]))\n",
    "        B = np.linalg.norm(np.array(landmarks[eye_landmarks[2]]) - np.array(landmarks[eye_landmarks[4]]))\n",
    "        C = np.linalg.norm(np.array(landmarks[eye_landmarks[0]]) - np.array(landmarks[eye_landmarks[3]]))\n",
    "        ear = (A + B) / (2.0 * C)\n",
    "        return ear\n",
    "\n",
    "    def set_colors(self, ear):\n",
    "        return self.RED_COLOR if ear < self.EAR_THRESHOLD else self.GREEN_COLOR\n",
    "\n",
    "    def draw_eye_landmarks(self, frame, landmarks, eye_landmarks, color):\n",
    "        for loc in eye_landmarks:\n",
    "            cv.circle(frame, (landmarks[loc]), 4, color, cv.FILLED)\n",
    "\n",
    "    def plot_ear_values(self):\n",
    "            \n",
    "        # Set dark theme\n",
    "        plt.style.use('dark_background')\n",
    "        \n",
    "        # Create figure and axis with dark background\n",
    "        fig, ax = plt.subplots(figsize=(12, 6))\n",
    "        fig.patch.set_facecolor('#000000')\n",
    "        ax.set_facecolor('#000000')\n",
    "        \n",
    "        # Plot EAR values\n",
    "        ax.plot(self.frame_numbers, self.ear_values, color='#00FF00', linewidth=1.5, \n",
    "                label=f'EAR (Total Blinks: {self.blink_counter})')\n",
    "        \n",
    "        # Add threshold line\n",
    "        ax.axhline(y=self.EAR_THRESHOLD, color='#FF0000', linestyle='--', \n",
    "                  label=f'EAR_THRESHOLD : ({self.EAR_THRESHOLD})')\n",
    "        \n",
    "        # Customize grid\n",
    "        ax.grid(True, linestyle='--', alpha=0.3)\n",
    "        \n",
    "        # Customize labels and title\n",
    "        ax.set_xlabel('Frame Number', color='white', fontsize=12)\n",
    "        ax.set_ylabel('Eye Aspect Ratio (EAR)', color='white', fontsize=12)\n",
    "        ax.set_title('Eye Aspect Ratio Over Time', color='white', fontsize=18, pad=20, \n",
    "                     fontweight='bold')\n",
    "        \n",
    "        # Customize ticks\n",
    "        ax.tick_params(colors='white')\n",
    "        \n",
    "        # Customize legend\n",
    "        ax.legend(facecolor='#000000', edgecolor='white', labelcolor='white')\n",
    "        \n",
    "        # Adjust layout\n",
    "        plt.tight_layout()\n",
    "        \n",
    "        # Determine plot filename\n",
    "        if self.save_plot : \n",
    "            if self.plot_output :\n",
    "                plot_save_dir = 'DATA/IMAGES/BLINK_DETECTION'\n",
    "                os.makedirs(plot_save_dir, exist_ok=True)\n",
    "                plot_filename = os.path.join(plot_save_dir, self.plot_output)\n",
    "            else :\n",
    "                # Generate default filename based on video path\n",
    "                base_name = os.path.splitext(os.path.basename(self.video_path))[0]\n",
    "                plot_filename = os.path.join(plot_save_dir, f\"{base_name}_ear_plot.png\")\n",
    "            \n",
    "            # Save plot\n",
    "            fig.savefig(plot_filename, facecolor='#000000', edgecolor='none', bbox_inches='tight')\n",
    "        plt.show()\n",
    "\n",
    "    def process_video(self):\n",
    "        try:\n",
    "            cap = cv.VideoCapture(self.video_path)\n",
    "            if not cap.isOpened():\n",
    "                print(f\"Failed to open video: {self.video_path}\")\n",
    "                raise IOError(\"Error: couldn't open the video!\")\n",
    "\n",
    "            w, h, fps = (int(cap.get(x)) for x in (\n",
    "                cv.CAP_PROP_FRAME_WIDTH, \n",
    "                cv.CAP_PROP_FRAME_HEIGHT, \n",
    "                cv.CAP_PROP_FPS\n",
    "            ))\n",
    "\n",
    "            if self.save_video:\n",
    "                self.out = cv.VideoWriter(\n",
    "                    self.video_output, \n",
    "                    cv.VideoWriter_fourcc(*\"mp4v\"), \n",
    "                    fps, \n",
    "                    (w, h)\n",
    "                )\n",
    "\n",
    "            frame_num = 0\n",
    "            while cap.isOpened():\n",
    "                ret, frame = cap.read()\n",
    "                if not ret:\n",
    "                    break\n",
    "\n",
    "                frame, face_landmarks = self.generator.create_face_mesh(frame, draw=False)\n",
    "\n",
    "                if len(face_landmarks) > 0:\n",
    "                    right_ear = self.eye_aspect_ratio(self.RIGHT_EYE_EAR, face_landmarks)\n",
    "                    left_ear = self.eye_aspect_ratio(self.LEFT_EYE_EAR, face_landmarks)\n",
    "                    ear = (right_ear + left_ear) / 2.0\n",
    "                    \n",
    "                    # Store EAR value and frame number\n",
    "                    self.ear_values.append(ear)\n",
    "                    self.frame_numbers.append(frame_num)\n",
    "\n",
    "                    color = self.set_colors(ear)\n",
    "\n",
    "                    if ear < self.EAR_THRESHOLD:\n",
    "                        self.frame_counter += 1\n",
    "                    else:\n",
    "                        if self.frame_counter >= self.CONSEC_FRAMES:\n",
    "                            self.blink_counter += 1\n",
    "                        self.frame_counter = 0\n",
    "\n",
    "                    self.draw_eye_landmarks(frame, face_landmarks, self.RIGHT_EYE, color)\n",
    "                    self.draw_eye_landmarks(frame, face_landmarks, self.LEFT_EYE, color)\n",
    "\n",
    "                    DrawingUtils.draw_text_with_bg(frame, f\"Blinks: {self.blink_counter}\", (0, 60), \n",
    "                                    font_scale=2, thickness=3,\n",
    "                                    bg_color=color, text_color=(0, 0, 0))\n",
    "\n",
    "                    if self.save_video:\n",
    "                        self.out.write(frame)\n",
    "\n",
    "                    resized_frame = cv.resize(frame, (1280, 720))\n",
    "                    cv.imshow(\"Blink Counter\", resized_frame)\n",
    "\n",
    "                frame_num += 1\n",
    "                if cv.waitKey(1) & 0xFF == ord('p'):\n",
    "                    break\n",
    "\n",
    "            cap.release()\n",
    "            if self.save_video:\n",
    "                self.out.release()\n",
    "            cv.destroyAllWindows()\n",
    "            \n",
    "            # Generate plot after video processing\n",
    "            if self.ear_values:\n",
    "                self.plot_ear_values()\n",
    "\n",
    "        except Exception as e:\n",
    "            print(f\"An error occurred: {e}\")\n",
    "\n",
    "\n",
    "# Example usage\n",
    "if __name__ == \"__main__\":\n",
    "    input_video_path = \"DATA/VIDEOS/INPUTS/blinking_0.mp4\"\n",
    "    blink_counter = PlotEAR(\n",
    "        video_path=input_video_path,\n",
    "        # save_video=True,\n",
    "        # video_output=\"blink_counter_4.mp4\",\n",
    "        save_plot=True,\n",
    "        plot_output=\"blinking_0_ear_plot.png\"\n",
    "    )\n",
    "    blink_counter.process_video()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Blink Detection with Multiple EAR Thresholds\n",
    "\n",
    "This implementation extends blink detection by evaluating Eye Aspect Ratio (EAR) against multiple thresholds. By varying threshold values, this approach helps determine the optimal threshold or threshold range for accurate blink detection. \n",
    "\n",
    "#### Key Highlights:\n",
    "1. **Threshold Variation**:\n",
    "   - Multiple EAR thresholds are tested (e.g., `0.250`, `0.275`, `0.300`, and `0.325`).\n",
    "   - Each threshold tracks its own blink count, enabling comparative analysis.\n",
    "\n",
    "2. **Dynamic EAR Plotting**:\n",
    "   - EAR values are plotted alongside threshold lines.\n",
    "   - Each threshold line is color-coded for easy distinction.\n",
    "\n",
    "3. **Output Options**:\n",
    "   - Generates static and interactive plots to analyze EAR behavior with multiple thresholds.\n",
    "   - Saves plots as `.png` and `.html` for future reference.\n",
    "\n",
    "4. **Applications**:\n",
    "   - Helps fine-tune the EAR threshold for specific use cases like drowsiness detection or fatigue monitoring.\n",
    "\n",
    "Below is the implementation of the `BlinkCounterMultiThreshold` class, which processes the video, computes EAR, and evaluates blink detection across varying thresholds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class BlinkCounterMultiThreshold:\n",
    "    \"\"\"\n",
    "    A class to detect and count eye blinks in a video using multiple EAR (Eye Aspect Ratio) thresholds.\n",
    "    \n",
    "    This class processes a video file, calculates the Eye Aspect Ratio for each frame,\n",
    "    and generates a plot showing the EAR values over time with multiple threshold lines\n",
    "    and an average EAR line.\n",
    "    \n",
    "    Attributes:\n",
    "        video_path (str): Path to the input video file\n",
    "        consec_frames (int): Number of consecutive frames below threshold to count as a blink\n",
    "        EAR_thresholds (list): List of float values representing different EAR thresholds\n",
    "        save_plot (bool): Whether to save the generated plot\n",
    "        plot_output (str): Output filename for the saved plot\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self, video_path, consec_frames, EAR_thresholds, save_plot=False, plot_output=None):\n",
    "        \"\"\"\n",
    "        Initialize the BlinkCounterMultiThreshold with video and analysis parameters.\n",
    "        \n",
    "        Args:\n",
    "            video_path (str): Path to the input video file\n",
    "            consec_frames (int): Number of consecutive frames below threshold to count as a blink\n",
    "            EAR_thresholds (list): List of float values representing different EAR thresholds\n",
    "            save_plot (bool, optional): Whether to save the generated plot. Defaults to False.\n",
    "            plot_output (str, optional): Output filename for the saved plot. Defaults to None.\n",
    "        \"\"\"\n",
    "        \n",
    "        self.generator = FaceMeshGenerator() \n",
    "        self.video_path = video_path\n",
    "        self.consec_frames = consec_frames\n",
    "        self.EAR_thresholds = EAR_thresholds\n",
    "       \n",
    "        # Plot saving parameters\n",
    "        self.save_plot = save_plot\n",
    "        self.plot_output = plot_output\n",
    "        \n",
    "        # Eye landmarks\n",
    "        self.RIGHT_EYE = [33, 7, 163, 144, 145, 153, 154, 155, 133, 173, 157, 158, 159, 160, 161, 246] \n",
    "        self.LEFT_EYE = [362, 382, 381, 380, 374, 373, 390, 249, 263, 466, 388, 387, 386, 385, 384, 398]\n",
    "        self.RIGHT_EYE_EAR = [33, 159, 158, 133, 153, 145]\n",
    "        self.LEFT_EYE_EAR = [362, 380, 374, 263, 386, 385]\n",
    "        \n",
    "        # Store EAR values for plotting\n",
    "        self.ear_values = []\n",
    "        self.frame_numbers = []\n",
    "       \n",
    "    def eye_aspect_ratio(self, eye_landmarks, landmarks):\n",
    "        \"\"\"\n",
    "        Calculate the Eye Aspect Ratio (EAR) for a given eye.\n",
    "        \n",
    "        The EAR is calculated using the formula:\n",
    "        EAR = (||p2-p6|| + ||p3-p5||) / (2||p1-p4||)\n",
    "        where p1, p2, p3, p4, p5, p6 are 2D landmark points.\n",
    "        \n",
    "        Args:\n",
    "            eye_landmarks (list): List of indices for the eye landmarks\n",
    "            landmarks (list): List of all facial landmarks coordinates\n",
    "            \n",
    "        Returns:\n",
    "            float: The calculated Eye Aspect Ratio\n",
    "        \"\"\"\n",
    "        A = np.linalg.norm(np.array(landmarks[eye_landmarks[1]]) - \n",
    "                           np.array(landmarks[eye_landmarks[5]]))\n",
    "        B = np.linalg.norm(np.array(landmarks[eye_landmarks[2]]) - \n",
    "                           np.array(landmarks[eye_landmarks[4]]))\n",
    "        C = np.linalg.norm(np.array(landmarks[eye_landmarks[0]]) - \n",
    "                           np.array(landmarks[eye_landmarks[3]]))\n",
    "        ear = (A + B) / (2.0 * C)\n",
    "        return ear\n",
    "\n",
    "    def count_blinks(self, ear_threshold):\n",
    "        \"\"\"\n",
    "        Count the number of blinks that occur below a given EAR threshold.\n",
    "        \n",
    "        A blink is counted when the EAR stays below the threshold for at least\n",
    "        consec_frames number of frames.\n",
    "        \n",
    "        Args:\n",
    "            ear_threshold (float): The EAR threshold below which to count blinks\n",
    "            \n",
    "        Returns:\n",
    "            int: The total number of blinks detected\n",
    "        \"\"\"\n",
    "        blink_count = 0\n",
    "        frame_counter = 0\n",
    "        \n",
    "        for ear in self.ear_values:\n",
    "            if ear < ear_threshold:\n",
    "                frame_counter += 1\n",
    "            else:\n",
    "                if frame_counter >= self.consec_frames:\n",
    "                    blink_count += 1\n",
    "                frame_counter = 0\n",
    "                \n",
    "        return blink_count\n",
    "\n",
    "    def plot_ear_values(self):\n",
    "        \"\"\"\n",
    "        Generate an interactive plot of EAR values over time.\n",
    "        \n",
    "        Creates a plot showing:\n",
    "        - EAR values over time\n",
    "        - Average EAR line\n",
    "        - Multiple threshold lines with blink counts\n",
    "        - Dark theme with customized styling\n",
    "        \n",
    "        If save_plot is True, saves both static PNG and interactive HTML versions\n",
    "        of the plot.\n",
    "        \"\"\"\n",
    "        \n",
    "        # Create figure with dark theme\n",
    "        fig = go.Figure()\n",
    "        \n",
    "        # Calculate average EAR\n",
    "        average_ear = sum(self.ear_values) / len(self.ear_values)\n",
    "        \n",
    "        # Plot EAR values\n",
    "        fig.add_trace(go.Scatter(\n",
    "            x=self.frame_numbers,\n",
    "            y=self.ear_values,\n",
    "            mode='lines',\n",
    "            name='EAR',\n",
    "            line=dict(color='#00FF00', width=2),\n",
    "            hovertemplate='Frame: %{x}<br>EAR: %{y:.3f}<extra></extra>'\n",
    "        ))\n",
    "        \n",
    "        # Add average EAR line\n",
    "        fig.add_trace(go.Scatter(\n",
    "            x=[min(self.frame_numbers), max(self.frame_numbers)],\n",
    "            y=[average_ear, average_ear],\n",
    "            mode='lines',\n",
    "            name=f'Average EAR: {average_ear:.3f}',\n",
    "            line=dict(color='#FFFFFF', width=2, dash='dot'),\n",
    "        ))\n",
    "        \n",
    "        # Add threshold lines with different colors\n",
    "        thresholds = self.EAR_thresholds\n",
    "        colors = ['#FF0000', '#FF00FF', '#00FFFF', '#FFFF00']  # Red, Magenta, Cyan, Yellow\n",
    "        \n",
    "        for threshold, color in zip(thresholds, colors):\n",
    "            blink_count = self.count_blinks(threshold)\n",
    "            fig.add_trace(go.Scatter(\n",
    "                x=[min(self.frame_numbers), max(self.frame_numbers)],\n",
    "                y=[threshold, threshold],\n",
    "                mode='lines',\n",
    "                name=f'Threshold {threshold:.3f} (Blinks: {blink_count})',\n",
    "                line=dict(color=color, width=2, dash='dash'),\n",
    "            ))\n",
    "        \n",
    "        # Update layout with dark theme and borders\n",
    "        fig.update_layout(\n",
    "            template='plotly_dark',\n",
    "            plot_bgcolor='black',\n",
    "            paper_bgcolor='black',\n",
    "            title=dict(\n",
    "                text='Eye Aspect Ratio Over Time With Multiple Thresholds',\n",
    "                font=dict(size=24, color='white'),\n",
    "                x=0.5,\n",
    "                y=0.95\n",
    "            ),\n",
    "            xaxis=dict(\n",
    "                title='Frame Number',\n",
    "                gridcolor='rgba(128, 128, 128, 0.2)',\n",
    "                title_font=dict(size=16, color='white'),\n",
    "                tickfont=dict(color='white'),\n",
    "                showline=True,\n",
    "                linewidth=1,\n",
    "                linecolor='white',\n",
    "                mirror=True  \n",
    "            ),\n",
    "            yaxis=dict(\n",
    "                title='Eye Aspect Ratio (EAR)',\n",
    "                gridcolor='rgba(128, 128, 128, 0.2)',\n",
    "                title_font=dict(size=16, color='white'),\n",
    "                tickfont=dict(color='white'),\n",
    "                showline=True,\n",
    "                linewidth=1,\n",
    "                linecolor='white',\n",
    "                mirror=True  \n",
    "            ),\n",
    "            showlegend=True,\n",
    "            legend=dict(\n",
    "                bgcolor='rgba(0,0,0,0)',\n",
    "                bordercolor='rgba(255,255,255,0.2)',\n",
    "                borderwidth=1,\n",
    "                font=dict(color='white'),\n",
    "                x=1.02,\n",
    "                y=1\n",
    "            ),\n",
    "            hovermode='x unified',\n",
    "            margin=dict(t=100, r=200),  # Adjust margins to prevent legend cutoff\n",
    "            width=1400,  # Set specific width for consistent static image output\n",
    "            height=700   # Set specific height for consistent static image output\n",
    "        )\n",
    "        \n",
    "        # Determine plot filename and save if requested\n",
    "        if self.save_plot:\n",
    "            plot_save_dir = 'DATA/IMAGES/BLINK_DETECTION'\n",
    "            os.makedirs(plot_save_dir, exist_ok=True)\n",
    "            \n",
    "            if self.plot_output:\n",
    "                # Get the base filename without extension\n",
    "                base_filename = os.path.splitext(self.plot_output)[0]\n",
    "            else:\n",
    "                base_filename = os.path.splitext(os.path.basename(self.video_path))[0]\n",
    "                base_filename += \"_multi_threshold_ear_plot\"\n",
    "            \n",
    "            # Save as static image (PNG)\n",
    "            png_filename = os.path.join(plot_save_dir, f\"{base_filename}.png\")\n",
    "            fig.write_image(png_filename, scale=2)  # scale=2 for higher resolution\n",
    "            \n",
    "            # Save as HTML for interactive version\n",
    "            html_filename = os.path.join(plot_save_dir, f\"{base_filename}.html\")\n",
    "            fig.write_html(html_filename)\n",
    "            \n",
    "            print(f\"Saved static plot as: {os.path.abspath(png_filename)}\")\n",
    "            print(f\"Saved interactive plot as: {os.path.abspath(html_filename)}\")\n",
    "        \n",
    "        # Show the interactive plot\n",
    "        fig.show()\n",
    "\n",
    "    def process_video(self):\n",
    "        \"\"\"\n",
    "        Process the input video to detect eye blinks.\n",
    "        \n",
    "        This method:\n",
    "        1. Opens the video file\n",
    "        2. Processes each frame to detect facial landmarks\n",
    "        3. Calculates EAR for both eyes in each frame\n",
    "        4. Stores the average EAR values\n",
    "        5. Generates a plot of the results\n",
    "        \n",
    "        Raises:\n",
    "            IOError: If the video file cannot be opened\n",
    "            Exception: For other errors during video processing\n",
    "        \"\"\"\n",
    "        try:\n",
    "            cap = cv.VideoCapture(self.video_path)\n",
    "            if not cap.isOpened():\n",
    "                print(f\"Failed to open video: {self.video_path}\")\n",
    "                raise IOError(\"Error: couldn't open the video!\")\n",
    "\n",
    "            frame_num = 0\n",
    "            while cap.isOpened():\n",
    "                ret, frame = cap.read()\n",
    "                if not ret:\n",
    "                    break\n",
    "\n",
    "                frame, face_landmarks = self.generator.create_face_mesh(frame, draw=False)\n",
    "\n",
    "                if len(face_landmarks) > 0:\n",
    "                    right_ear = self.eye_aspect_ratio(self.RIGHT_EYE_EAR, face_landmarks)\n",
    "                    left_ear = self.eye_aspect_ratio(self.LEFT_EYE_EAR, face_landmarks)\n",
    "                    ear = (right_ear + left_ear) / 2.0\n",
    "                    \n",
    "                    # Store EAR value and frame number\n",
    "                    self.ear_values.append(ear)\n",
    "                    self.frame_numbers.append(frame_num)\n",
    "\n",
    "                frame_num += 1\n",
    "\n",
    "            cap.release()\n",
    "            cv.destroyAllWindows()\n",
    "            \n",
    "            # Generate plot after video processing\n",
    "            if self.ear_values:\n",
    "                self.plot_ear_values()\n",
    "\n",
    "        except Exception as e:\n",
    "            print(f\"An error occurred: {e}\")\n",
    "\n",
    "\n",
    "# Example usage\n",
    "if __name__ == \"__main__\":\n",
    "    input_video_path = \"DATA/VIDEOS/INPUTS/blinking_0.mp4\"\n",
    "    blink_counter = BlinkCounterMultiThreshold(\n",
    "        video_path=input_video_path,\n",
    "        consec_frames=4,\n",
    "        EAR_thresholds=[0.250, 0.275, 0.300, 0.325],\n",
    "        save_plot=True,\n",
    "        plot_output=\"blinking_0_multi_threshold_ear_plot.png\"\n",
    "    )\n",
    "    blink_counter.process_video()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Analysis\n",
    "  From this analysis of different EAR thresholds with CONSEC_FRAMES set to 4, we can draw several important observations:\n",
    "\n",
    "1. **Threshold Performance Analysis**:\n",
    "  - *`0.250 (Red line)`: Only detected `8 blinks`*\n",
    "    * This threshold is too low, missing many genuine blinks\n",
    "    * Only captures the most extreme eye closures\n",
    "    * Underestimates the actual blink count significantly\n",
    "\n",
    "  - *` 0.275-0.300 (Purple and Blue lines)`: Both detected `23 blinks`*\n",
    "    * This range appears to be optimal for blink detection\n",
    "    * Consistently captures natural blink patterns\n",
    "    * Shows the most accurate blink count\n",
    "    * The agreement between these two thresholds suggests this is a stable detection range\n",
    "\n",
    "  - *`0.325 (Yellow line)`: Detected `26 blinks`*\n",
    "    * This threshold is too high\n",
    "    * May be detecting false positives\n",
    "    * Counting minor eye movements or partial blinks as full blinks\n",
    "    * Overestimates the actual blink count\n",
    "\n",
    "2. **Optimal Detection Range**:\n",
    "  - The sweet spot appears to be between `0.275` and `0.300`\n",
    "  - This range provides consistent and reliable blink detection\n",
    "  - Matches typical EAR patterns during natural blinks\n",
    "  - Offers good discrimination between blinks and normal eye movements\n",
    "\n",
    "3. **Detection Reliability**:\n",
    "  - The consistency between `0.275` and `0.300` thresholds (both detecting `23` blinks) suggests these are reliable settings\n",
    "  - The divergence in counts at `0.250` and `0.325` indicates these are outside the optimal detection range\n",
    "\n",
    "4. **Implementation Recommendations**:\n",
    "  - Set the EAR threshold between `0.275` and `0.300` for optimal detection\n",
    "  - Keep `CONSEC_FRAMES` at 4 as it works well with these thresholds\n",
    "  - Avoid thresholds below `0.275` or above `0.300` to prevent missed detections or false positives\n",
    "\n",
    "> This settings for `consec_frames` and `EAR_threshold` are strictly for the video `blinking_0.mp4` and may need to be adjusted for other videos.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look for the optimal EAR_Threshold for this video `blinking_1.mp4`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_video_path = \"DATA/VIDEOS/INPUTS/blinking_1.mp4\"\n",
    "blink_counter = BlinkCounterMultiThreshold(\n",
    "    video_path=input_video_path,\n",
    "    consec_frames=3, \n",
    "    EAR_thresholds=[0.265, 0.285, 0.295, 0.305],\n",
    "    save_plot=True,\n",
    "    plot_output=\"blinking_1_multi_threshold_ear_plot.png\"\n",
    ")\n",
    "blink_counter.process_video()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Blink Detection with Varying Consecutive Frames\n",
    "\n",
    "This implementation focuses on analyzing blink detection by varying the consecutive frame count (`consec_frames`) after determining an optimal Eye Aspect Ratio (EAR) threshold. This approach helps fine-tune the sensitivity of blink detection based on the required blink duration.\n",
    "\n",
    "#### Key Highlights:\n",
    "1. **Optimal Threshold Usage**:\n",
    "   - The EAR threshold is fixed at a value (e.g., `0.285`), which was determined from prior analysis.\n",
    "\n",
    "2. **Varying Frame Intervals**:\n",
    "   - Different `consec_frames` values (e.g., `2`, `4`, and `6`) are used to evaluate their impact on blink detection accuracy.\n",
    "\n",
    "3. **Plot with GridSpec Layout**:\n",
    "   - The EAR timeline is visualized in a main plot with a fixed threshold line.\n",
    "   - Subplots highlight specific frame intervals and blink counts for each `consec_frames` value.\n",
    "\n",
    "4. **Applications**:\n",
    "   - Helps determine the ideal `consec_frames` value for diverse scenarios (e.g., fast vs. slow blinking).\n",
    "   - Enables dynamic adjustments to improve detection robustness in different environments.\n",
    "\n",
    "Below is the implementation of the `EARConsecFrames` class, which processes the video, computes EAR values, and analyzes blink detection by varying `consec_frames`.\n",
    "\n",
    "\n",
    "> The values of the blink detection parameters are chosen as per the video input `blinking_0.mp4`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class EARConsecFrames:\n",
    "    def __init__(self, video_path, Ear_threshold, consec_frames, save_plot=False, plot_output=None):\n",
    "        self.generator = FaceMeshGenerator() \n",
    "        self.video_path = video_path\n",
    "        \n",
    "        # Plot saving parameters\n",
    "        self.save_plot = save_plot\n",
    "        self.plot_output = plot_output\n",
    "        \n",
    "        # Eye landmarks\n",
    "        self.RIGHT_EYE = [33, 7, 163, 144, 145, 153, 154, 155, 133, 173, 157, 158, 159, 160, 161, 246] \n",
    "        self.LEFT_EYE = [362, 382, 381, 380, 374, 373, 390, 249, 263, 466, 388, 387, 386, 385, 384, 398]\n",
    "        self.RIGHT_EYE_EAR = [33, 159, 158, 133, 153, 145]\n",
    "        self.LEFT_EYE_EAR = [362, 380, 374, 263, 386, 385]\n",
    "\n",
    "        self.threshold = Ear_threshold\n",
    "        self.consec_frames = consec_frames\n",
    "        \n",
    "        # Store EAR values for plotting\n",
    "        self.ear_values = []\n",
    "        self.frame_numbers = []\n",
    "       \n",
    "    def eye_aspect_ratio(self, eye_landmarks, landmarks):\n",
    "        A = np.linalg.norm(np.array(landmarks[eye_landmarks[1]]) - np.array(landmarks[eye_landmarks[5]]))\n",
    "        B = np.linalg.norm(np.array(landmarks[eye_landmarks[2]]) - np.array(landmarks[eye_landmarks[4]]))\n",
    "        C = np.linalg.norm(np.array(landmarks[eye_landmarks[0]]) - np.array(landmarks[eye_landmarks[3]]))\n",
    "        ear = (A + B) / (2.0 * C)\n",
    "        return ear\n",
    "\n",
    "    def count_blinks(self, ear_threshold, consec_frames):\n",
    "        blink_count = 0\n",
    "        frame_counter = 0\n",
    "        \n",
    "        for ear in self.ear_values:\n",
    "            if ear < ear_threshold:\n",
    "                frame_counter += 1\n",
    "            else:\n",
    "                if frame_counter >= consec_frames:\n",
    "                    blink_count += 1\n",
    "                frame_counter = 0\n",
    "                \n",
    "        return blink_count\n",
    "\n",
    "    def plot_ear_values(self):\n",
    "    # Set dark theme\n",
    "        plt.style.use('dark_background')\n",
    "        \n",
    "        # Create figure with GridSpec\n",
    "        fig = plt.figure(figsize=(16, 9))\n",
    "        gs = fig.add_gridspec(2, 3)\n",
    "        \n",
    "        # Set global dark background\n",
    "        fig.patch.set_facecolor('#000000')\n",
    "        \n",
    "        # Main plot spanning entire first row\n",
    "        ax1 = fig.add_subplot(gs[0, :])\n",
    "        ax1.set_facecolor('#000000')\n",
    "        \n",
    "        # Plot full EAR values\n",
    "        ax1.plot(self.frame_numbers, self.ear_values, color='#00FF00', linewidth=1.5, label='EAR')\n",
    "        \n",
    "        # Fixed threshold\n",
    "        threshold = self.threshold\n",
    "        ax1.axhline(y=threshold, color='#FF00FF', linestyle='--', label=f'Threshold {threshold:.3f}')\n",
    "        \n",
    "        # Customize main plot\n",
    "        ax1.grid(True, linestyle='--', alpha=0.5)\n",
    "        ax1.set_xlabel('Frame Number', color='white', fontsize=12)\n",
    "        ax1.set_ylabel('Eye Aspect Ratio (EAR)', color='white', fontsize=12)\n",
    "        ax1.set_title('Complete Eye Aspect Ratio Timeline', \n",
    "                    color='white', fontsize=14, pad=10, fontweight='bold')\n",
    "        ax1.tick_params(colors='white')\n",
    "        ax1.legend(facecolor='#000000', edgecolor='white', labelcolor='white')\n",
    "        \n",
    "        # Create subplots for different intervals\n",
    "        intervals = self.consec_frames\n",
    "        colors = ['#FF0000', '#00FFFF', '#FFFF00']  # Red, Cyan, Yellow\n",
    "        \n",
    "        # Filter frame range 300-400\n",
    "        start_idx, end_idx = 110, 170\n",
    "        frames_subset = self.frame_numbers[start_idx:end_idx]\n",
    "        ear_subset = self.ear_values[start_idx:end_idx]\n",
    "        \n",
    "        for idx, (interval, color) in enumerate(zip(intervals, colors)):\n",
    "            # Create subplot in second row\n",
    "            ax = fig.add_subplot(gs[1, idx])\n",
    "            ax.set_facecolor('#000000')\n",
    "            \n",
    "            # Plot EAR values\n",
    "            ax.plot(frames_subset, ear_subset, color='#00FF00', linewidth=2, label='EAR')\n",
    "            ax.axhline(y=threshold, color='#FF00FF', linestyle='--', label=f'Threshold {threshold:.3f}')\n",
    "            \n",
    "            # Highlight regions based on interval\n",
    "            for start in range(start_idx+interval, end_idx-interval, interval):\n",
    "                ax.axvspan(start, start + interval, facecolor=color, alpha=0.1)\n",
    "                ax.axvline(x=start, color=color, linestyle='--', alpha=0.8)\n",
    "            \n",
    "            # Count blinks in this range\n",
    "            blink_count = self.count_blinks(threshold, interval)\n",
    "            \n",
    "            # Customize subplot\n",
    "            ax.grid(True, linestyle='--', alpha=0.5)\n",
    "            ax.set_xlabel('Frame Number', color='white', fontsize=10)\n",
    "            ax.set_ylabel('EAR', color='white', fontsize=10)\n",
    "            ax.set_title(f'Interval {interval} Frames\\n{blink_count} blinks', \n",
    "                        color='white', fontsize=12, pad=10)\n",
    "            ax.tick_params(colors='white')\n",
    "            ax.set_xlim(start_idx, end_idx)\n",
    "        \n",
    "        # Adjust layout\n",
    "        plt.tight_layout(h_pad=1.5)\n",
    "        \n",
    "        # Save plot if requested\n",
    "        if self.save_plot:\n",
    "            plot_save_dir = 'DATA/IMAGES/BLINK_DETECTION'\n",
    "            os.makedirs(plot_save_dir, exist_ok=True)\n",
    "            \n",
    "            if self.plot_output:\n",
    "                plot_filename = os.path.join(plot_save_dir, self.plot_output)\n",
    "            else:\n",
    "                base_name = os.path.splitext(os.path.basename(self.video_path))[0]\n",
    "                plot_filename = os.path.join(plot_save_dir, f\"{base_name}_gridspec_ear_plot.png\")\n",
    "            \n",
    "            # Save plot\n",
    "            fig.savefig(plot_filename, facecolor='#000000', edgecolor='none', bbox_inches='tight', dpi=300)\n",
    "            \n",
    "        plt.show()\n",
    "\n",
    "    def process_video(self):\n",
    "        try:\n",
    "            cap = cv.VideoCapture(self.video_path)\n",
    "            if not cap.isOpened():\n",
    "                print(f\"Failed to open video: {self.video_path}\")\n",
    "                raise IOError(\"Error: couldn't open the video!\")\n",
    "\n",
    "            frame_num = 0\n",
    "            while cap.isOpened():\n",
    "                ret, frame = cap.read()\n",
    "                if not ret:\n",
    "                    break\n",
    "\n",
    "                frame, face_landmarks = self.generator.create_face_mesh(frame, draw=False)\n",
    "\n",
    "                if len(face_landmarks) > 0:\n",
    "                    right_ear = self.eye_aspect_ratio(self.RIGHT_EYE_EAR, face_landmarks)\n",
    "                    left_ear = self.eye_aspect_ratio(self.LEFT_EYE_EAR, face_landmarks)\n",
    "                    ear = (right_ear + left_ear) / 2.0\n",
    "                    \n",
    "                    # Store EAR value and frame number\n",
    "                    self.ear_values.append(ear)\n",
    "                    self.frame_numbers.append(frame_num)\n",
    "\n",
    "                frame_num += 1\n",
    "\n",
    "            cap.release()\n",
    "            cv.destroyAllWindows()\n",
    "            \n",
    "            # Generate plot after video processing\n",
    "            if self.ear_values:\n",
    "                self.plot_ear_values()\n",
    "\n",
    "        except Exception as e:\n",
    "                print(f\"An error occurred: {e}\")\n",
    "\n",
    "\n",
    "# Example usage\n",
    "if __name__ == \"__main__\":\n",
    "    input_video_path = \"DATA/VIDEOS/INPUTS/blinking_0.mp4\"\n",
    "    blink_counter = EARConsecFrames(\n",
    "        video_path=input_video_path,\n",
    "        Ear_threshold=0.285,\n",
    "        consec_frames=[2, 4, 6],\n",
    "        # save_plot=True,\n",
    "        # plot_output=\"lady_blinking_consec_frames_ear_plot_grid.png\"\n",
    "    )\n",
    "    blink_counter.process_video()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the code shell below we have used `Plolty` instead of `Matplotlib` for an interactive plot. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class EARConsecFramesPlotly:\n",
    "    \"\"\"\n",
    "    A class for analyzing and visualizing Eye Aspect Ratio (EAR) from video data using Plotly.\n",
    "    \n",
    "    The class processes video frames to detect facial landmarks and calculate EAR values,\n",
    "    then creates interactive visualizations showing EAR variations over time with different\n",
    "    consecutive frame intervals for blink detection.\n",
    "    \n",
    "    Attributes:\n",
    "        video_path (str): Path to the input video file\n",
    "        threshold (float): EAR threshold value for blink detection\n",
    "        consec_frames (list): List of consecutive frame intervals to analyze\n",
    "        save_plot (bool): Whether to save the generated plots\n",
    "        plot_output (str): Output filename for saved plots\n",
    "    \"\"\"\n",
    "    def __init__(self, video_path, EAR_threshold, consec_frames, save_plot=False, plot_output=None):\n",
    "        self.generator = FaceMeshGenerator() \n",
    "        self.video_path = video_path\n",
    "        self.threshold = EAR_threshold\n",
    "        self.consec_frames = consec_frames\n",
    "        \n",
    "        # Plot saving parameters\n",
    "        self.save_plot = save_plot\n",
    "        self.plot_output = plot_output\n",
    "        \n",
    "        # Eye landmarks\n",
    "        self.RIGHT_EYE = [33, 7, 163, 144, 145, 153, 154, 155, 133, 173, 157, 158, 159, 160, 161, 246] \n",
    "        self.LEFT_EYE = [362, 382, 381, 380, 374, 373, 390, 249, 263, 466, 388, 387, 386, 385, 384, 398]\n",
    "        self.RIGHT_EYE_EAR = [33, 159, 158, 133, 153, 145]\n",
    "        self.LEFT_EYE_EAR = [362, 380, 374, 263, 386, 385]\n",
    "        \n",
    "        # Store EAR values for plotting\n",
    "        self.ear_values = []\n",
    "        self.frame_numbers = []\n",
    "       \n",
    "    def eye_aspect_ratio(self, eye_landmarks, landmarks):\n",
    "        \"\"\"\n",
    "        Calculate the Eye Aspect Ratio (EAR) for a given eye.\n",
    "        \n",
    "        The EAR is calculated using the formula:\n",
    "        EAR = (||p2-p6|| + ||p3-p5||) / (2||p1-p4||)\n",
    "        where p1, p2, p3, p4, p5, p6 are 2D landmark points.\n",
    "        \n",
    "        Args:\n",
    "            eye_landmarks (list): List of indices for the eye landmarks\n",
    "            landmarks (list): List of all facial landmarks coordinates\n",
    "            \n",
    "        Returns:\n",
    "            float: The calculated Eye Aspect Ratio\n",
    "        \"\"\"\n",
    "        A = np.linalg.norm(np.array(landmarks[eye_landmarks[1]]) - \n",
    "                           np.array(landmarks[eye_landmarks[5]]))\n",
    "        B = np.linalg.norm(np.array(landmarks[eye_landmarks[2]]) - \n",
    "                           np.array(landmarks[eye_landmarks[4]]))\n",
    "        C = np.linalg.norm(np.array(landmarks[eye_landmarks[0]]) - \n",
    "                           np.array(landmarks[eye_landmarks[3]]))\n",
    "        ear = (A + B) / (2.0 * C)\n",
    "        return ear\n",
    "\n",
    "    def count_blinks(self, ear_threshold, consec_frames, start_idx=None, end_idx=None):\n",
    "        \"\"\"\n",
    "        Count the number of blinks based on EAR values and consecutive frames criteria.\n",
    "        \n",
    "        Args:\n",
    "            ear_threshold (float): Threshold value for considering an eye as closed\n",
    "            consec_frames (int): Number of consecutive frames required for a blink\n",
    "            start_idx (int, optional): Starting index for analysis\n",
    "            end_idx (int, optional): Ending index for analysis\n",
    "            \n",
    "        Returns:\n",
    "            int: Number of detected blinks\n",
    "        \"\"\"\n",
    "        blink_count = 0\n",
    "        frame_counter = 0\n",
    "\n",
    "        if start_idx is not None and end_idx is not None:\n",
    "            ear_values = self.ear_values[start_idx:end_idx]\n",
    "        else:\n",
    "            ear_values = self.ear_values\n",
    "        \n",
    "        for ear in ear_values:\n",
    "            if ear < ear_threshold:\n",
    "                frame_counter += 1\n",
    "            else:\n",
    "                if frame_counter >= consec_frames:\n",
    "                    blink_count += 1\n",
    "                frame_counter = 0\n",
    "                \n",
    "        return blink_count\n",
    "\n",
    "    def plot_ear_values(self):\n",
    "        \"\"\"\n",
    "        Generate interactive plots showing EAR values over time with various analysis metrics.\n",
    "        \n",
    "        Creates a multi-panel plot including:\n",
    "        - Main plot showing complete EAR timeline\n",
    "        - Three subplots showing different consecutive frame intervals\n",
    "        - EAR threshold line\n",
    "        - Average EAR line\n",
    "        - Interval highlighting\n",
    "        \"\"\"\n",
    "        # Create figure with secondary y-axis\n",
    "        fig = make_subplots(\n",
    "            rows=2, cols=3,\n",
    "            specs=[\n",
    "                [{'colspan':3}, None, None],\n",
    "                [{}, {}, {}]\n",
    "            ],\n",
    "            subplot_titles=('Complete Eye Aspect Ratio Timeline', \n",
    "                        'Interval 2 Frames', 'Interval 4 Frames', 'Interval 6 Frames'),\n",
    "            row_heights=[0.6, 0.4],\n",
    "            vertical_spacing=0.2,\n",
    "            horizontal_spacing=0.05\n",
    "        )\n",
    "        \n",
    "        # Calculate average EAR\n",
    "        avg_ear = np.mean(self.ear_values)\n",
    "        \n",
    "        # Fixed threshold\n",
    "        threshold = self.threshold\n",
    "        \n",
    "        # Main plot (full timeline)\n",
    "        fig.add_trace(\n",
    "            go.Scatter(\n",
    "                x=self.frame_numbers,\n",
    "                y=self.ear_values,\n",
    "                name='EAR',\n",
    "                line=dict(color='#00FF00', width=1.5),\n",
    "                hovertemplate='Frame: %{x}<br>EAR: %{y:.3f}<extra></extra>',\n",
    "                legendgroup='ear',\n",
    "                showlegend=True\n",
    "            ),\n",
    "            row=1, col=1\n",
    "        )\n",
    "        \n",
    "        # Add threshold line to main plot\n",
    "        fig.add_trace(\n",
    "            go.Scatter(\n",
    "                x=[min(self.frame_numbers), max(self.frame_numbers)],\n",
    "                y=[threshold, threshold],\n",
    "                name='EAR Threshold',\n",
    "                line=dict(color='#FF00FF', width=1.5, dash='dash'),\n",
    "                legendgroup='threshold',\n",
    "                showlegend=True\n",
    "            ),\n",
    "            row=1, col=1\n",
    "        )\n",
    "        \n",
    "        # Add average EAR line to main plot\n",
    "        fig.add_trace(\n",
    "            go.Scatter(\n",
    "                x=[min(self.frame_numbers), max(self.frame_numbers)],\n",
    "                y=[avg_ear, avg_ear],\n",
    "                name='Average EAR',\n",
    "                line=dict(color='#FFFFFF', width=1.5, dash='dot'),\n",
    "                legendgroup='average',\n",
    "                showlegend=True\n",
    "            ),\n",
    "            row=1, col=1\n",
    "        )\n",
    "        \n",
    "        # Filter frame range for subplots\n",
    "        start_idx, end_idx = 400, 500\n",
    "        frames_subset = self.frame_numbers[start_idx:end_idx]\n",
    "        ear_subset = self.ear_values[start_idx:end_idx]\n",
    "        \n",
    "        # Colors for interval highlighting\n",
    "        colors = ['#FF0000', '#00FFFF', '#FFFF00']  # Red, Cyan, Yellow\n",
    "        intervals = self.consec_frames\n",
    "        \n",
    "        # Create subplots for different intervals\n",
    "        for idx, (interval, color) in enumerate(zip(intervals, colors)):\n",
    "            # Add EAR trace\n",
    "            fig.add_trace(\n",
    "                go.Scatter(\n",
    "                    x=frames_subset,\n",
    "                    y=ear_subset,\n",
    "                    name=f'EAR (interval {interval})',\n",
    "                    line=dict(color='#00FF00', width=2),\n",
    "                    hovertemplate='Frame: %{x}<br>EAR: %{y:.3f}<extra></extra>',\n",
    "                    legendgroup='ear',\n",
    "                    showlegend=False,\n",
    "                ),\n",
    "                row=2, col=idx+1\n",
    "            )\n",
    "            \n",
    "            # Add threshold line\n",
    "            fig.add_trace(\n",
    "                go.Scatter(\n",
    "                    x=[min(frames_subset), max(frames_subset)],\n",
    "                    y=[threshold, threshold],\n",
    "                    name=f'EAR Threshold (interval {interval})',\n",
    "                    line=dict(color='#FF00FF', width=1.5, dash='dash'),\n",
    "                    legendgroup='threshold',\n",
    "                    showlegend=False,\n",
    "                ),\n",
    "                row=2, col=idx+1\n",
    "            )\n",
    "            \n",
    "            # Add average EAR line\n",
    "            fig.add_trace(\n",
    "                go.Scatter(\n",
    "                    x=[min(frames_subset), max(frames_subset)],\n",
    "                    y=[avg_ear, avg_ear],\n",
    "                    name=f'Average EAR (interval {interval})',\n",
    "                    line=dict(color='#FFFFFF', width=1.5, dash='dot'),\n",
    "                    legendgroup='average',\n",
    "                    showlegend=False,\n",
    "                ),\n",
    "                row=2, col=idx+1\n",
    "            )\n",
    "            \n",
    "            # Add interval highlighting\n",
    "            for start in range(start_idx+interval, end_idx-interval, interval):\n",
    "                fig.add_vrect(\n",
    "                    x0=start,\n",
    "                    x1=start + interval,\n",
    "                    fillcolor=color,\n",
    "                    opacity=0.1,\n",
    "                    line_width=0,\n",
    "                    row=2,\n",
    "                    col=idx+1\n",
    "                )\n",
    "                \n",
    "                fig.add_vline(\n",
    "                    x=start,\n",
    "                    line_color=color,\n",
    "                    line_dash=\"3px\",\n",
    "                    line_width=1,\n",
    "                    opacity=0.8,\n",
    "                    row=2,\n",
    "                    col=idx+1\n",
    "                )\n",
    "            \n",
    "            # Count blinks for both total timeline and subset\n",
    "            total_blinks = self.count_blinks(threshold, interval)\n",
    "            subset_blinks = self.count_blinks(threshold, interval, start_idx, end_idx)\n",
    "            \n",
    "            # Update subplot title to include both blink counts\n",
    "            fig.layout.annotations[idx+1].update(\n",
    "                text=f'Interval {interval} Frames<br>Total: {total_blinks} blinks<br>Subset: {subset_blinks} blinks'\n",
    "            )\n",
    "        \n",
    "        # Update layout\n",
    "        fig.update_layout(\n",
    "            template=\"plotly_dark\",\n",
    "            paper_bgcolor='black',\n",
    "            plot_bgcolor='black',\n",
    "            title=dict(\n",
    "                text=f\"Eye Aspect Ratio Analysis\",\n",
    "                font=dict(size=30),\n",
    "                y=0.97,\n",
    "                x=0.5,\n",
    "                xanchor='center',\n",
    "                yanchor='top'\n",
    "            ),\n",
    "            showlegend=True,\n",
    "            legend=dict(\n",
    "                bgcolor='rgba(0,0,0,0)',\n",
    "                bordercolor='rgba(255,255,255,0.2)',\n",
    "                borderwidth=1,\n",
    "                font=dict(color='white'),\n",
    "                orientation='h',\n",
    "                xanchor='right',\n",
    "                yanchor='bottom',\n",
    "                x=1,\n",
    "                y=1.02,\n",
    "                groupclick=\"togglegroup\"\n",
    "            ),\n",
    "            hovermode='x unified',\n",
    "            height=900,\n",
    "            width=1600,\n",
    "        )\n",
    "        \n",
    "        for i in range(1, 5):  # Loop through all subplots (1 main + 3 small)\n",
    "            if i == 1:\n",
    "                row, col = 1, 1\n",
    "                xaxis_limit = [min(self.frame_numbers), max(self.frame_numbers)]\n",
    "            else:\n",
    "                row, col = 2, i-1\n",
    "                xaxis_limit = [min(frames_subset), max(frames_subset)]\n",
    "                \n",
    "            # Update x-axis\n",
    "            fig.update_xaxes(\n",
    "                showline=True,\n",
    "                range=xaxis_limit, \n",
    "                linewidth=1,\n",
    "                linecolor='white',\n",
    "                mirror=True,\n",
    "                title='Frame Number',\n",
    "                row=row,\n",
    "                col=col\n",
    "            )\n",
    "            \n",
    "            # Update y-axis\n",
    "            fig.update_yaxes(\n",
    "                showline=True,\n",
    "                linewidth=1,\n",
    "                linecolor='white',\n",
    "                mirror=True,\n",
    "                title='Eye Aspect Ratio (EAR)' if col == 1 else '',\n",
    "                row=row,\n",
    "                col=col\n",
    "            )\n",
    "        \n",
    "        # Save plot if requested\n",
    "        if self.save_plot:\n",
    "            plot_save_dir = 'DATA/IMAGES/BLINK_DETECTION'\n",
    "            os.makedirs(plot_save_dir, exist_ok=True)\n",
    "            \n",
    "            if self.plot_output:\n",
    "                base_filename = os.path.splitext(self.plot_output)[0]\n",
    "            else:\n",
    "                base_filename = os.path.splitext(os.path.basename(self.video_path))[0]\n",
    "                base_filename += \"_consec_frames_ear_plot_plotly\"\n",
    "            \n",
    "            png_filename = os.path.join(plot_save_dir, f\"{base_filename}.png\")\n",
    "            fig.write_image(png_filename, scale=2)\n",
    "            \n",
    "            html_filename = os.path.join(plot_save_dir, f\"{base_filename}.html\")\n",
    "            fig.write_html(html_filename)\n",
    "            \n",
    "            print(f\"Saved static plot as: {os.path.abspath(png_filename)}\")\n",
    "            print(f\"Saved interactive plot as: {os.path.abspath(html_filename)}\")\n",
    "        \n",
    "        fig.show()\n",
    "\n",
    "    def process_video(self):\n",
    "        \"\"\"\n",
    "        Process the input video to extract EAR values and generate visualization.\n",
    "        \n",
    "        Reads the video frame by frame, detects facial landmarks, calculates EAR values,\n",
    "        and stores them for later analysis and visualization.\n",
    "        \n",
    "        Raises:\n",
    "            IOError: If the video file cannot be opened\n",
    "            Exception: For other processing errors\n",
    "        \"\"\"\n",
    "        try:\n",
    "            cap = cv.VideoCapture(self.video_path)\n",
    "            if not cap.isOpened():\n",
    "                print(f\"Failed to open video: {self.video_path}\")\n",
    "                raise IOError(\"Error: couldn't open the video!\")\n",
    "\n",
    "            frame_num = 0\n",
    "            while cap.isOpened():\n",
    "                ret, frame = cap.read()\n",
    "                if not ret:\n",
    "                    break\n",
    "\n",
    "                frame, face_landmarks = self.generator.create_face_mesh(frame, draw=False)\n",
    "\n",
    "                if len(face_landmarks) > 0:\n",
    "                    right_ear = self.eye_aspect_ratio(self.RIGHT_EYE_EAR, face_landmarks)\n",
    "                    left_ear = self.eye_aspect_ratio(self.LEFT_EYE_EAR, face_landmarks)\n",
    "                    ear = (right_ear + left_ear) / 2.0\n",
    "                    \n",
    "                    self.ear_values.append(ear)\n",
    "                    self.frame_numbers.append(frame_num)\n",
    "\n",
    "                frame_num += 1\n",
    "\n",
    "            cap.release()\n",
    "            cv.destroyAllWindows()\n",
    "            \n",
    "            if self.ear_values:\n",
    "                self.plot_ear_values()\n",
    "\n",
    "        except Exception as e:\n",
    "                print(f\"An error occurred: {e}\")\n",
    "\n",
    "\n",
    "# Example usage\n",
    "if __name__ == \"__main__\":\n",
    "    input_video_path = \"DATA/VIDEOS/INPUTS/blinking_0.mp4\"\n",
    "    blink_counter = EARConsecFramesPlotly(\n",
    "        video_path=input_video_path,\n",
    "        EAR_threshold=0.285,\n",
    "        consec_frames=[2, 4, 6],\n",
    "        save_plot=True,\n",
    "        plot_output=\"blinking_0_consec_frames_ear_plot_grid_plotly.png\"\n",
    "    )\n",
    "    blink_counter.process_video()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Observations from the plots:\n",
    "We initially set the EAR_threshold to 0.28 and varied the number of consecutive_frames to find out the optimal value for `CONSEC_FRAMES`\n",
    "1. **Blink Detection Across Frame Intervals:**\n",
    "   - For `consec_frames = 2`: \n",
    "     - The system detects **23 blinks**, which matches the total expected number of blinks. \n",
    "     - This shows high accuracy as the smaller interval captures the rapid changes in EAR associated with blinks.\n",
    "   - For `consec_frames = 4`:\n",
    "     - The system again detects **23 blinks**, demonstrating that this interval is still effective for blink detection without missing any events.\n",
    "   - For `consec_frames = 6`:\n",
    "     - The system detects **10 blinks**, missing many blinks compared to the expected total (23). \n",
    "     - This suggests that larger intervals result in skipped frames where EAR dips, leading to under-detection.\n",
    "\n",
    "2. **Subset Analysis:**\n",
    "   - In the selected subset of frames (around 400–500), the number of detected blinks decreases as the frame interval increases.\n",
    "     - For `consec_frames = 2` and `4`, the subset shows **3 blinks**—indicating accurate tracking.\n",
    "     - For `consec_frames = 6`, the subset shows only **1 blink**, confirming that some blink events are being overlooked.\n",
    "\n",
    "3. **Trade-off Between Frame Interval and Blink Detection:**\n",
    "   - Smaller intervals ensure better detection accuracy as they capture transient EAR changes during blinks.\n",
    "   - Larger intervals fail to record quick events, as they skip over frames where EAR dips below the threshold.\n",
    "\n",
    "When using 6 consecutive frames (`consec_frames = 6`), some blinks are missed because the system does not sample frames frequently enough to capture the transient changes in the Eye Aspect Ratio (EAR) during a blink. Here's a detailed explanation:\n",
    "\n",
    "##### 1. **Nature of Blinks and EAR Changes**\n",
    "   - A blink is a rapid action where the EAR drops sharply and returns to normal within a short time (typically a few frames in a high-frame-rate video).\n",
    "   - If the interval between sampled frames is too large (e.g., 6 frames), the system may \"skip over\" the frames where the EAR dips below the threshold (indicating a blink). As a result:\n",
    "     - The EAR might never appear below the threshold within the sampled frames.\n",
    "     - The blink goes undetected.\n",
    "\n",
    "\n",
    "##### 2. **Temporal Resolution and Sampling Frequency**\n",
    "   - Smaller `consec_frames` (e.g., 2 or 4) provide higher temporal resolution, meaning the system samples the EAR values more frequently, ensuring that even rapid changes in EAR are captured.\n",
    "   - Larger `consec_frames` reduce the sampling frequency, causing the system to lose fine-grained information about quick events like blinks.\n",
    "\n",
    "\n",
    "\n",
    "##### 3. **Comparison of Frame Intervals**\n",
    "   - Consider an EAR timeline where a blink occurs between frames 10 and 12:\n",
    "     - **For `consec_frames = 2` or `4`**: The system samples EAR values at frames 10, 12, 14, etc., ensuring it captures the dip at frame 10 or 12.\n",
    "     - **For `consec_frames = 6`**: The system samples EAR values at frames 6, 12, 18, etc. In this case:\n",
    "       - If the blink starts and ends between sampled frames (e.g., between frames 8 and 11), the system completely misses it.\n",
    "\n",
    "\n",
    "\n",
    "##### 4. **Impact of Blink Duration**\n",
    "   - A blink typically lasts only a few milliseconds (around 100–400 ms for humans). \n",
    "   - If the frame interval exceeds the duration of the blink (as it might when `consec_frames = 6`), the system skips the blink entirely.\n",
    "\n",
    "\n",
    "\n",
    "##### Visual Summary\n",
    "Imagine this as a simplified timeline for EAR changes:\n",
    "\n",
    "| Frame Number | EAR Value | Blink Detected? (Threshold = 0.28) |\n",
    "|--------------|-----------|-----------------------------------|\n",
    "| 10           | 0.35      | No                               |\n",
    "| 11           | 0.25      | Yes (Blink starts)               |\n",
    "| 12           | 0.22      | Yes (Blink continues)            |\n",
    "| 13           | 0.27      | Yes (Blink ends)                 |\n",
    "| 14           | 0.32      | No                               |\n",
    "\n",
    "- **`consec_frames = 2`:** Captures frames 10, 12, 14 → Blink detected.\n",
    "- **`consec_frames = 6`:** Captures frames 10, 16 → Blink missed, as the system skips critical frames where the EAR drops.\n",
    "\n",
    "\n",
    "##### Key Takeaway:\n",
    "A larger frame interval (e.g., `6`) reduces the likelihood of detecting short-duration events like blinks because the system skips frames where the EAR temporarily drops below the threshold. To avoid this, it's crucial to select an interval that ensures the system can capture the rapid changes in EAR during a blink.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We observed from the EAR graph that to count the blinks properly we need to keep the `EYE_THRESHOLD` in the range 0.275 to 0.3 and the `CONSEC_FRAMES` in the range 2-4 for the video input `blinking_0.mp4` as it is providing correct and stable value of total number of blinks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Blink Detection and Counting\n",
    "\n",
    "This code provides a robust implementation for detecting and counting blinks in a video by leveraging the Eye Aspect Ratio (EAR). After determining the optimal EAR threshold and consecutive frame count (`consec_frames`), this script accurately detects blinks in real-time or from recorded videos.\n",
    "\n",
    "### Key Features:\n",
    "1. **EAR-Based Blink Detection**:\n",
    "   - Uses facial landmarks around the eyes to calculate the EAR.\n",
    "   - Blinks are detected when the EAR drops below the threshold (`ear_threshold`) for a specified number of consecutive frames.\n",
    "\n",
    "2. **Customizable Parameters**:\n",
    "   - `ear_threshold`: Defines the EAR value below which eyes are considered closed.\n",
    "   - `consec_frames`: Determines how long the eyes must stay closed to count as a blink.\n",
    "\n",
    "3. **Visualization**:\n",
    "   - Landmarks around the eyes are visualized in green (open) or red (closed) based on the EAR value.\n",
    "   - A live counter displays the total number of blinks detected.\n",
    "\n",
    "4. **Video Processing**:\n",
    "   - Processes each frame to detect blinks and can optionally save the processed video with overlays.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class BlinkCounter:\n",
    "    \"\"\"\n",
    "    A class to detect and count eye blinks in a video using facial landmarks.\n",
    "    \n",
    "    This class processes video input to detect eye blinks by calculating the Eye Aspect Ratio (EAR)\n",
    "    of both eyes. It can either process a video file or save the processed output to a new video file.\n",
    "    \n",
    "    Attributes:\n",
    "        ear_threshold (float): Threshold below which eyes are considered closed\n",
    "        consec_frames (int): Number of consecutive frames eyes must be closed to count as blink\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self, video_path, ear_threshold, consec_frames, save_video=False, output_filename=None):\n",
    "        \"\"\"\n",
    "        Initialize the BlinkCounter with video source and processing parameters.\n",
    "        \n",
    "        Args:\n",
    "            video_path (str): Path to the input video file\n",
    "            ear_threshold (float): Threshold for eye aspect ratio to detect blink (default: 0.3)\n",
    "            consec_frames (int): Number of consecutive frames needed to confirm blink (default: 4)\n",
    "            save_video (bool): Whether to save the processed video\n",
    "            output_filename (str, optional): Name for the output video file\n",
    "        \"\"\"\n",
    "        # Initialize face mesh detector\n",
    "        self.generator = FaceMeshGenerator() \n",
    "        self.video_path = video_path\n",
    "        self.save_video = save_video\n",
    "        self.output_filename = output_filename\n",
    "        \n",
    "        # Define facial landmarks for eye detection\n",
    "        # Each list contains indices corresponding to points around the eyes\n",
    "        self.RIGHT_EYE = [33, 7, 163, 144, 145, 153, 154, 155, 133, 173, 157, 158, 159, 160, 161, 246]\n",
    "        self.LEFT_EYE = [362, 382, 381, 380, 374, 373, 390, 249, 263, 466, 388, 387, 386, 385, 384, 398]\n",
    "        \n",
    "        # Specific landmarks for EAR calculation\n",
    "        # These specific points are used to calculate the eye aspect ratio\n",
    "        self.RIGHT_EYE_EAR = [33, 159, 158, 133, 153, 145]\n",
    "        self.LEFT_EYE_EAR = [362, 380, 374, 263, 386, 385]\n",
    "        \n",
    "        # Blink detection parameters\n",
    "        self.ear_threshold = ear_threshold  # Eye aspect ratio threshold for blink detection\n",
    "        self.consec_frames = consec_frames  # Minimum consecutive frames for a valid blink\n",
    "        self.blink_counter = 0    # Counter for total blinks detected\n",
    "        self.frame_counter = 0    # Counter for consecutive frames below threshold\n",
    "        \n",
    "        # Define colors for visualization (in BGR format)\n",
    "        self.GREEN_COLOR = (86, 241, 13)  # Used when eyes are open\n",
    "        self.RED_COLOR = (30, 46, 209)    # Used when eyes are closed\n",
    "        \n",
    "        # Set up output video directory and path if saving is enabled\n",
    "        if self.save_video and self.output_filename:\n",
    "            save_dir = \"DATA/VIDEOS/OUTPUTS\"\n",
    "            os.makedirs(save_dir, exist_ok=True)\n",
    "            self.output_filename = os.path.join(save_dir, self.output_filename)\n",
    "\n",
    "    def update_blink_count(self, ear):\n",
    "        \"\"\"\n",
    "        Update blink counter based on current eye aspect ratio.\n",
    "        \n",
    "        This method implements the blink detection logic:\n",
    "        - If EAR is below threshold, increment frame counter\n",
    "        - If EAR returns above threshold and enough consecutive frames were counted,\n",
    "          increment blink counter\n",
    "        \n",
    "        Args:\n",
    "            ear (float): Current eye aspect ratio\n",
    "            \n",
    "        Returns:\n",
    "            bool: True if a new blink was detected, False otherwise\n",
    "        \"\"\"\n",
    "        blink_detected = False\n",
    "        \n",
    "        if ear < self.ear_threshold:\n",
    "            self.frame_counter += 1\n",
    "        else:\n",
    "            if self.frame_counter >= self.consec_frames:\n",
    "                self.blink_counter += 1\n",
    "                blink_detected = True\n",
    "            self.frame_counter = 0\n",
    "            \n",
    "        return blink_detected\n",
    "\n",
    "    def eye_aspect_ratio(self, eye_landmarks, landmarks):\n",
    "        \"\"\"\n",
    "        Calculate the eye aspect ratio (EAR) for given eye landmarks.\n",
    "        \n",
    "        The EAR is calculated using the formula:\n",
    "        EAR = (||p2-p6|| + ||p3-p5||) / (2||p1-p4||)\n",
    "        where p1-p6 are specific points around the eye.\n",
    "        \n",
    "        Args:\n",
    "            eye_landmarks (list): Indices of landmarks for one eye\n",
    "            landmarks (list): List of all facial landmarks\n",
    "        \n",
    "        Returns:\n",
    "            float: Calculated eye aspect ratio\n",
    "        \"\"\"\n",
    "        A = np.linalg.norm(np.array(landmarks[eye_landmarks[1]]) - np.array(landmarks[eye_landmarks[5]]))\n",
    "        B = np.linalg.norm(np.array(landmarks[eye_landmarks[2]]) - np.array(landmarks[eye_landmarks[4]]))\n",
    "        C = np.linalg.norm(np.array(landmarks[eye_landmarks[0]]) - np.array(landmarks[eye_landmarks[3]]))\n",
    "        return (A + B) / (2.0 * C)\n",
    "\n",
    "    def set_colors(self, ear):\n",
    "        \"\"\"\n",
    "        Determine visualization color based on eye aspect ratio.\n",
    "        \n",
    "        Args:\n",
    "            ear (float): Current eye aspect ratio\n",
    "        \n",
    "        Returns:\n",
    "            tuple: BGR color values\n",
    "        \"\"\"\n",
    "        return self.RED_COLOR if ear < self.ear_threshold else self.GREEN_COLOR\n",
    "\n",
    "    def draw_eye_landmarks(self, frame, landmarks, eye_landmarks, color):\n",
    "        \"\"\"\n",
    "        Draw landmarks around the eyes on the frame.\n",
    "        \n",
    "        Args:\n",
    "            frame (numpy.ndarray): Video frame to draw on\n",
    "            landmarks (list): List of facial landmarks\n",
    "            eye_landmarks (list): Indices of landmarks for one eye\n",
    "            color (tuple): BGR color values for drawing\n",
    "        \"\"\"\n",
    "        for loc in eye_landmarks:\n",
    "            cv.circle(frame, (landmarks[loc]), 4, color, cv.FILLED)\n",
    "\n",
    "    def process_video(self):\n",
    "        \"\"\"\n",
    "        Main method to process the video and detect blinks.\n",
    "        \n",
    "        This method:\n",
    "        1. Opens the video file\n",
    "        2. Processes each frame to detect faces and calculate EAR\n",
    "        3. Counts blinks based on EAR values\n",
    "        4. Displays and optionally saves the processed video\n",
    "        \n",
    "        Raises:\n",
    "            IOError: If video file cannot be opened\n",
    "            Exception: For other processing errors\n",
    "        \"\"\"\n",
    "        try:\n",
    "            # Open video capture\n",
    "            cap = cv.VideoCapture(self.video_path)\n",
    "            if not cap.isOpened():\n",
    "                print(f\"Failed to open video: {self.video_path}\")\n",
    "                raise IOError(\"Error: couldn't open the video!\")\n",
    "\n",
    "            # Get video properties\n",
    "            w, h, fps = (int(cap.get(x)) for x in (\n",
    "                cv.CAP_PROP_FRAME_WIDTH,\n",
    "                cv.CAP_PROP_FRAME_HEIGHT,\n",
    "                cv.CAP_PROP_FPS\n",
    "            ))\n",
    "\n",
    "            # Initialize video writer if saving is enabled\n",
    "            if self.save_video:\n",
    "                self.out = cv.VideoWriter(\n",
    "                    self.output_filename,\n",
    "                    cv.VideoWriter_fourcc(*\"mp4v\"),\n",
    "                    fps,\n",
    "                    (w, h)\n",
    "                )\n",
    "\n",
    "            # Main processing loop\n",
    "            while cap.isOpened():\n",
    "                ret, frame = cap.read()\n",
    "                if not ret:\n",
    "                    break\n",
    "\n",
    "                # Detect facial landmarks\n",
    "                frame, face_landmarks = self.generator.create_face_mesh(frame, draw=False)\n",
    "\n",
    "                if len(face_landmarks) > 0:\n",
    "                    # Calculate eye aspect ratio\n",
    "                    right_ear = self.eye_aspect_ratio(self.RIGHT_EYE_EAR, face_landmarks)\n",
    "                    left_ear = self.eye_aspect_ratio(self.LEFT_EYE_EAR, face_landmarks)\n",
    "                    ear = (right_ear + left_ear) / 2.0\n",
    "\n",
    "                    # Update blink detection\n",
    "                    self.update_blink_count(ear)\n",
    "\n",
    "                    # Determine visualization color based on EAR\n",
    "                    color = self.set_colors(ear)\n",
    "\n",
    "                    # Draw visualizations\n",
    "                    self.draw_eye_landmarks(frame, face_landmarks, self.RIGHT_EYE, color)\n",
    "                    self.draw_eye_landmarks(frame, face_landmarks, self.LEFT_EYE, color)\n",
    "                    DrawingUtils.draw_text_with_bg(frame, f\"Blinks: {self.blink_counter}\", (0, 60),\n",
    "                                    font_scale=2, thickness=3,\n",
    "                                    bg_color=color, text_color=(0, 0, 0))\n",
    "\n",
    "                    # Save frame if enabled\n",
    "                    if self.save_video:\n",
    "                        self.out.write(frame)\n",
    "\n",
    "                    # Display the frame\n",
    "                    resized_frame = cv.resize(frame, (1280, 720))\n",
    "                    cv.imshow(\"Blink Counter\", resized_frame)\n",
    "\n",
    "                # Break loop if 'p' is pressed\n",
    "                if cv.waitKey(int(1000/fps)) & 0xFF == ord('p'):\n",
    "                    break\n",
    "\n",
    "            # Cleanup\n",
    "            cap.release()\n",
    "            if self.save_video:\n",
    "                self.out.release()\n",
    "            cv.destroyAllWindows()\n",
    "\n",
    "        except Exception as e:\n",
    "            print(f\"An error occurred: {e}\")\n",
    "\n",
    "\n",
    "# Example usage\n",
    "if __name__ == \"__main__\":\n",
    "    input_video_path = \"DATA/VIDEOS/INPUTS/blinking_4.mp4\"\n",
    "    \n",
    "    # Create blink counter with custom parameters\n",
    "    blink_counter = BlinkCounter(\n",
    "        video_path=input_video_path,\n",
    "        ear_threshold=0.3,  \n",
    "        consec_frames=4,    \n",
    "        # save_video=True,\n",
    "        # output_filename=\"blink_counter_4.mp4\"\n",
    "    )\n",
    "    blink_counter.process_video()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Real-Time Eye Blink Detection and Visualization with EAR Plotting\n",
    "\n",
    "This Python script implements a **real-time blink detection system** using the Eye Aspect Ratio (EAR). The system processes video frames to detect eye blinks, count them, and visualize the EAR over time using a live plot. Additionally, the output video can be saved with the EAR plot overlay for further analysis.\n",
    "\n",
    "---\n",
    "\n",
    "### **Features**\n",
    "1. **Blink Detection**  \n",
    "   - Uses the Eye Aspect Ratio (EAR) method to detect blinks.\n",
    "   - Counts blinks when EAR falls below a specified threshold for consecutive frames.\n",
    "\n",
    "2. **Real-Time Visualization**  \n",
    "   - Plots the EAR in real-time.\n",
    "   - Displays EAR threshold and dynamically updates the plot based on frame data.\n",
    "\n",
    "3. **Video Processing**  \n",
    "   - Processes input video frames.\n",
    "   - Option to save the processed video with EAR visualization.\n",
    "\n",
    "4. **Customization**  \n",
    "   - EAR threshold and consecutive frames for blink detection can be configured.\n",
    "   - Adjustable plot aesthetics for enhanced readability.\n",
    "\n",
    "---\n",
    "\n",
    "### **Core Components**\n",
    "\n",
    "#### **1. Eye Aspect Ratio (EAR) Calculation**\n",
    "The EAR is calculated using specific points around the eyes to monitor the openness of the eyes:\n",
    "$$\n",
    "EAR = \\frac{||p_2 - p_6|| + ||p_3 - p_5||}{2 \\cdot ||p_1 - p_4||}\n",
    "$$\n",
    "Where $p_1, p_2, \\dots, p_6$ are the eye landmarks.\n",
    "\n",
    "#### **2. Real-Time Plot**\n",
    "- The EAR plot updates live during video processing.\n",
    "- Visual indicators like a threshold line make it easier to track blinks.\n",
    "\n",
    "#### **3. Blink Counting Logic**\n",
    "- If EAR falls below the threshold for a predefined number of frames (`consec_frames`), a blink is registered.\n",
    "\n",
    "#### **4. Modular Design**\n",
    "The class provides functions for:\n",
    "   - Frame-by-frame processing.\n",
    "   - EAR calculation.\n",
    "   - Visualization of EAR values on a live plot.\n",
    "   - Saving the output video with visualized data.\n",
    "\n",
    "---\n",
    "\n",
    "### **Usage Example**\n",
    "\n",
    "#### **Inputs**\n",
    "- **Video File**: The video to be processed.\n",
    "- **EAR Threshold**: The value below which a blink is detected (e.g., 0.294).\n",
    "- **Consecutive Frames**: The minimum number of consecutive frames below the threshold to count as a blink (e.g., 3).\n",
    "\n",
    "#### **Outputs**\n",
    "- Blink count displayed on the video.\n",
    "- Real-time EAR plot overlay.\n",
    "- Optionally saves the processed video with visualizations.\n",
    "\n",
    "#### **Sample Code**\n",
    "```python\n",
    "if __name__ == \"__main__\":\n",
    "    # Define input video path and parameters\n",
    "    input_video_path = \"DATA/VIDEOS/INPUTS/blinking_1.mp4\"\n",
    "    \n",
    "    # Initialize the blink counter with parameters\n",
    "    blink_counter = BlinkCounterandEARPlot(\n",
    "        video_path=input_video_path,\n",
    "        threshold=0.294,  # EAR threshold\n",
    "        consec_frames=3,  # Frames below threshold to count as a blink\n",
    "        save_video=True,  # Save processed video\n",
    "        output_filename=\"blinking_1_output.mp4\"  # Output filename\n",
    "    )\n",
    "    \n",
    "    # Process the video and count blinks\n",
    "    blink_counter.process_video()\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "### **Class Breakdown**\n",
    "\n",
    "#### **1. `BlinkCounterandEARPlot`**\n",
    "Main class encapsulating all functionality.\n",
    "\n",
    "- **Attributes**:\n",
    "  - `EAR_THRESHOLD`: Blink detection threshold.\n",
    "  - `CONSEC_FRAMES`: Frames required to detect a blink.\n",
    "  - `video_path`: Path to the input video.\n",
    "  - `save_video`: Boolean flag to save the processed video.\n",
    "  - `output_filename`: Filename for the saved video.\n",
    "\n",
    "- **Methods**:\n",
    "  - `_init_video_saving`: Configures video saving.\n",
    "  - `_init_plot`: Initializes the live EAR plot.\n",
    "  - `eye_aspect_ratio`: Calculates EAR for eye landmarks.\n",
    "  - `process_frame`: Processes a single frame to calculate EAR and detect blinks.\n",
    "  - `process_video`: Processes the entire video file frame-by-frame.\n",
    "  - `_update_plot`: Updates the EAR plot with new values.\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "### **Expected Output**\n",
    "- **Processed Video**:  \n",
    "   A video with blink counts and a real-time EAR plot overlay.\n",
    "- **Real-Time Visualization**:  \n",
    "   EAR plot updates dynamically while processing the video.\n",
    "\n",
    "---\n",
    "\n",
    "This system is ideal for applications in **health monitoring**, **fatigue detection**, and **behavior analysis**, making it a versatile tool for real-time blink analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "class BlinkCounterandEARPlot:\n",
    "    \"\"\"\n",
    "    A class to detect and count eye blinks in a video using facial landmarks.\n",
    "    \n",
    "    This class processes video frames to detect faces, track eye movements,\n",
    "    calculate Eye Aspect Ratio (EAR), plot EAR, and count blinks in real-time.\n",
    "    \"\"\"\n",
    "    \n",
    "    # Define facial landmark indices for eyes\n",
    "    RIGHT_EYE = [33, 7, 163, 144, 145, 153, 154, 155, 133, 173, 157, 158, 159, 160, 161, 246]\n",
    "    LEFT_EYE = [362, 382, 381, 380, 374, 373, 390, 249, 263, 466, 388, 387, 386, 385, 384, 398]\n",
    "    RIGHT_EYE_EAR = [33, 159, 158, 133, 153, 145]  # Points for EAR calculation\n",
    "    LEFT_EYE_EAR = [362, 380, 374, 263, 386, 385]  # Points for EAR calculation\n",
    "    \n",
    "    # Define colors for visualization\n",
    "    COLORS = {\n",
    "        'GREEN': {'hex': '#56f10d', 'bgr': (86, 241, 13)},\n",
    "        'BLUE': {'hex': '#0329fc', 'bgr': (30, 46, 209)},\n",
    "        'RED': {'hex': '#f70202', 'bgr': None}\n",
    "    }\n",
    "\n",
    "    def __init__(self, video_path, threshold, consec_frames, save_video=False, output_filename=None):\n",
    "        \"\"\"\n",
    "        Initialize the BlinkCounter with video and detection parameters.\n",
    "        \n",
    "        Args:\n",
    "            video_path (str): Path to the input video file\n",
    "            threshold (float): EAR threshold for blink detection\n",
    "            consec_frames (int): Number of consecutive frames below threshold to count as a blink\n",
    "            save_video (bool): Whether to save the processed video\n",
    "            output_filename (str): Name of the output video file if saving\n",
    "        \"\"\"\n",
    "        # Initialize core parameters\n",
    "        self.generator = FaceMeshGenerator()\n",
    "        self.video_path = video_path\n",
    "        self.EAR_THRESHOLD = threshold\n",
    "        self.CONSEC_FRAMES = consec_frames\n",
    "        \n",
    "        # Initialize video saving parameters\n",
    "        self._init_video_saving(save_video, output_filename)\n",
    "        \n",
    "        # Initialize tracking variables\n",
    "        self._init_tracking_variables()\n",
    "        \n",
    "        # Initialize plotting\n",
    "        self._init_plot()\n",
    "\n",
    "    def _init_video_saving(self, save_video, output_filename):\n",
    "        \"\"\"Initialize video saving parameters and create output directory if needed.\"\"\"\n",
    "        self.save_video = save_video\n",
    "        self.output_filename = output_filename\n",
    "        self.out = None\n",
    "        \n",
    "        if self.save_video and self.output_filename:\n",
    "            save_dir = \"DATA/VIDEOS/OUTPUTS\"\n",
    "            os.makedirs(save_dir, exist_ok=True)\n",
    "            self.output_filename = os.path.join(save_dir, self.output_filename)\n",
    "\n",
    "    def _init_tracking_variables(self):\n",
    "        \"\"\"Initialize variables used for tracking blinks and frame processing.\"\"\"\n",
    "        self.blink_counter = 0\n",
    "        self.frame_counter = 0\n",
    "        self.frame_number = 0\n",
    "        self.ear_values = []\n",
    "        self.frame_numbers = []\n",
    "        self.max_frames = 100\n",
    "        self.new_w = self.new_h = None\n",
    "        self.default_ymin = 0.18  \n",
    "        self.default_ymax = 0.44  \n",
    "\n",
    "    def _init_plot(self):\n",
    "        \"\"\"Initialize the matplotlib plot for EAR visualization.\"\"\"\n",
    "        # Set up dark theme plot\n",
    "        plt.style.use('dark_background')\n",
    "        plt.ioff()\n",
    "        self.fig, self.ax = plt.subplots(figsize=(8, 5), dpi=200)\n",
    "        self.canvas = FigureCanvas(self.fig)\n",
    "        \n",
    "        # Configure plot aesthetics\n",
    "        self._configure_plot_aesthetics()\n",
    "        \n",
    "        # Initialize plot data\n",
    "        self._init_plot_data()\n",
    "\n",
    "        self.fig.canvas.draw()\n",
    "\n",
    "    def _configure_plot_aesthetics(self):\n",
    "        \"\"\"Configure the aesthetic properties of the plot.\"\"\"\n",
    "        # Set background colors\n",
    "        self.fig.patch.set_facecolor('#000000')\n",
    "        self.ax.set_facecolor('#000000')\n",
    "        \n",
    "        # Configure axes with default limits initially\n",
    "        self.ax.set_ylim(self.default_ymin, self.default_ymax)\n",
    "        self.ax.set_xlim(0, self.max_frames)\n",
    "        \n",
    "        # Set labels and title\n",
    "        self.ax.set_xlabel(\"Frame Number\", color='white', fontsize=12)\n",
    "        self.ax.set_ylabel(\"EAR\", color='white', fontsize=12)\n",
    "        self.ax.set_title(\"Real-Time Eye Aspect Ratio (EAR)\", \n",
    "                         color='white', pad=10, fontsize=18, fontweight='bold')\n",
    "        \n",
    "        # Configure grid and spines\n",
    "        self.ax.grid(True, color='#707b7c', linestyle='--', alpha=0.7)\n",
    "        for spine in self.ax.spines.values():\n",
    "            spine.set_color('white')\n",
    "        \n",
    "        # Configure ticks and legend\n",
    "        self.ax.tick_params(colors='white', which='both')\n",
    "\n",
    "    def _init_plot_data(self):\n",
    "        \"\"\"Initialize the plot data and curves.\"\"\"\n",
    "        self.x_vals = list(range(self.max_frames))\n",
    "        self.y_vals = [0] * self.max_frames\n",
    "        self.Y_vals = [self.EAR_THRESHOLD] * self.max_frames\n",
    "        \n",
    "        # Create curves with explicit labels\n",
    "        self.EAR_curve, = self.ax.plot(\n",
    "            self.x_vals, \n",
    "            self.y_vals,\n",
    "            color=self.COLORS['GREEN']['hex'],\n",
    "            label=\"Eye Aspect Ratio\",\n",
    "            linewidth=2\n",
    "        )\n",
    "        \n",
    "        self.threshold_line, = self.ax.plot(\n",
    "            self.x_vals,\n",
    "            self.Y_vals,\n",
    "            color=self.COLORS['RED']['hex'],\n",
    "            label=\"Blink Threshold\",\n",
    "            linewidth=2,\n",
    "            linestyle='--'\n",
    "        )\n",
    "        \n",
    "        # Add legend \n",
    "        self.legend = self.ax.legend(\n",
    "            handles=[self.EAR_curve, self.threshold_line],\n",
    "            loc='upper right',\n",
    "            fontsize=10,\n",
    "            facecolor='black',\n",
    "            edgecolor='white',\n",
    "            labelcolor='white',\n",
    "            framealpha=0.8,\n",
    "            borderpad=1,\n",
    "            handlelength=2\n",
    "        )\n",
    "\n",
    "    def eye_aspect_ratio(self, eye_landmarks, landmarks):\n",
    "        \"\"\"\n",
    "        Calculate the eye aspect ratio (EAR) for given eye landmarks.\n",
    "        \n",
    "        The EAR is calculated using the formula:\n",
    "        EAR = (||p2-p6|| + ||p3-p5||) / (2||p1-p4||)\n",
    "        where p1-p6 are specific points around the eye.\n",
    "        \n",
    "        Args:\n",
    "            eye_landmarks (list): Indices of landmarks for one eye\n",
    "            landmarks (list): List of all facial landmarks\n",
    "        \n",
    "        Returns:\n",
    "            float: Calculated eye aspect ratio\n",
    "        \"\"\"\n",
    "        A = np.linalg.norm(np.array(landmarks[eye_landmarks[1]]) - \n",
    "                          np.array(landmarks[eye_landmarks[5]]))\n",
    "        B = np.linalg.norm(np.array(landmarks[eye_landmarks[2]]) - \n",
    "                          np.array(landmarks[eye_landmarks[4]]))\n",
    "        C = np.linalg.norm(np.array(landmarks[eye_landmarks[0]]) - \n",
    "                          np.array(landmarks[eye_landmarks[3]]))\n",
    "        return (A + B) / (2.0 * C)\n",
    "\n",
    "    def _update_plot(self, ear):\n",
    "        \"\"\"Update the plot with new EAR values.\"\"\"\n",
    "        if len(self.ear_values) > self.max_frames:\n",
    "            self.ear_values.pop(0)\n",
    "            self.frame_numbers.pop(0)\n",
    "            \n",
    "        color = self.COLORS['BLUE']['hex'] if ear < self.EAR_THRESHOLD else self.COLORS['GREEN']['hex']\n",
    "        \n",
    "        self.EAR_curve.set_xdata(self.frame_numbers)\n",
    "        self.EAR_curve.set_ydata(self.ear_values)\n",
    "        self.EAR_curve.set_color(color)\n",
    "        \n",
    "        self.threshold_line.set_xdata(self.frame_numbers)\n",
    "        self.threshold_line.set_ydata([self.EAR_THRESHOLD] * len(self.frame_numbers))\n",
    "        \n",
    "        \n",
    "        if len(self.frame_numbers) > 1:\n",
    "            x_min = min(self.frame_numbers)\n",
    "            x_max = max(self.frame_numbers)\n",
    "            if x_min == x_max:\n",
    "                # Add a small padding if min and max are the same\n",
    "                x_min -= 0.5\n",
    "                x_max += 0.5\n",
    "            self.ax.set_xlim(x_min, x_max)\n",
    "        else:\n",
    "            # Default limits for initialization\n",
    "            self.ax.set_xlim(0, self.max_frames)\n",
    "\n",
    "        # Ensure the legend remains visible\n",
    "        if self.legend not in self.ax.get_children():\n",
    "            self.legend = self.ax.legend(\n",
    "                handles=[self.EAR_curve, self.threshold_line],\n",
    "                loc='upper right',\n",
    "                fontsize=10,\n",
    "                facecolor='black',\n",
    "                edgecolor='white',\n",
    "                labelcolor='white',\n",
    "                framealpha=0.8,\n",
    "                borderpad=1,\n",
    "                handlelength=2\n",
    "            )\n",
    "        \n",
    "        # Redraw with better quality\n",
    "        self.ax.draw_artist(self.ax.patch)\n",
    "        self.ax.draw_artist(self.EAR_curve)\n",
    "        self.ax.draw_artist(self.threshold_line)\n",
    "        self.ax.draw_artist(self.legend)\n",
    "        self.fig.canvas.flush_events()\n",
    "\n",
    "    def process_frame(self, frame):\n",
    "        \"\"\"\n",
    "        Process a single frame to detect and analyze eyes.\n",
    "        \n",
    "        Returns:\n",
    "            tuple: Processed frame and EAR value\n",
    "        \"\"\"\n",
    "        frame, face_landmarks = self.generator.create_face_mesh(frame, draw=False)\n",
    "        \n",
    "        if not face_landmarks:\n",
    "            return frame, None\n",
    "            \n",
    "        # Calculate EAR\n",
    "        right_ear = self.eye_aspect_ratio(self.RIGHT_EYE_EAR, face_landmarks)\n",
    "        left_ear = self.eye_aspect_ratio(self.LEFT_EYE_EAR, face_landmarks)\n",
    "        ear = (right_ear + left_ear) / 2.0\n",
    "        \n",
    "        # Determine visualization color\n",
    "        color = self.COLORS['BLUE']['bgr'] if ear < self.EAR_THRESHOLD else self.COLORS['GREEN']['bgr']\n",
    "        \n",
    "        # Draw landmarks and update blink counter\n",
    "        self._draw_frame_elements(frame, face_landmarks, color)\n",
    "        \n",
    "        return frame, ear\n",
    "\n",
    "    def _draw_frame_elements(self, frame, landmarks, color):\n",
    "        \"\"\"Draw eye landmarks and blink counter on the frame.\"\"\"\n",
    "        # Draw eye landmarks\n",
    "        for eye in [self.RIGHT_EYE, self.LEFT_EYE]:\n",
    "            for loc in eye:\n",
    "                cv.circle(frame, (landmarks[loc]), 2, color, cv.FILLED)\n",
    "        \n",
    "        # Draw blink counter\n",
    "        DrawingUtils.draw_text_with_bg(\n",
    "            frame, f\"Blinks: {self.blink_counter}\", (0, 60),\n",
    "            font_scale=2, thickness=3,\n",
    "            bg_color=color, text_color=(0, 0, 0)\n",
    "        )\n",
    "\n",
    "    def process_video(self):\n",
    "        \"\"\"Process the entire video and detect blinks.\"\"\"\n",
    "        try:\n",
    "            cap = cv.VideoCapture(self.video_path)\n",
    "            if not cap.isOpened():\n",
    "                raise IOError(f\"Failed to open video: {self.video_path}\")\n",
    "\n",
    "            self._process_video_frames(cap)\n",
    "            \n",
    "        except Exception as e:\n",
    "            print(f\"An error occurred: {e}\")\n",
    "        finally:\n",
    "            cap.release()\n",
    "            if self.out:\n",
    "                self.out.release()\n",
    "            cv.destroyAllWindows()\n",
    "\n",
    "    def _process_video_frames(self, cap):\n",
    "        \"\"\"Process individual frames from the video capture.\"\"\"\n",
    "        # Get video properties\n",
    "        w = int(cap.get(cv.CAP_PROP_FRAME_WIDTH))\n",
    "        h = int(cap.get(cv.CAP_PROP_FRAME_HEIGHT))\n",
    "        fps = int(cap.get(cv.CAP_PROP_FPS))\n",
    "\n",
    "        while cap.isOpened():\n",
    "            ret, frame = cap.read()\n",
    "            if not ret:\n",
    "                break\n",
    "\n",
    "            # Process frame and get EAR\n",
    "            frame, ear = self.process_frame(frame)\n",
    "            \n",
    "            if ear is not None:\n",
    "                self._update_blink_detection(ear)\n",
    "                self._update_visualization(frame, ear, fps)\n",
    "\n",
    "            if cv.waitKey(1) & 0xFF == ord('p'):\n",
    "                break\n",
    "\n",
    "    def _update_blink_detection(self, ear):\n",
    "        \"\"\"Update blink detection based on EAR value.\"\"\"\n",
    "        self.ear_values.append(ear)\n",
    "        self.frame_numbers.append(self.frame_number)\n",
    "        \n",
    "        if ear < self.EAR_THRESHOLD:\n",
    "            self.frame_counter += 1\n",
    "        else:\n",
    "            if self.frame_counter >= self.CONSEC_FRAMES:\n",
    "                self.blink_counter += 1\n",
    "            self.frame_counter = 0\n",
    "        \n",
    "        self.frame_number += 1\n",
    "\n",
    "    def _update_visualization(self, frame, ear, fps):\n",
    "        \"\"\"Update the visualization including the plot and video output.\"\"\"\n",
    "        self._update_plot(ear)\n",
    "        \n",
    "        # Convert plot to image and resize\n",
    "        plot_img = self.plot_to_image()\n",
    "        plot_img_resized = cv.resize(\n",
    "            plot_img,\n",
    "            (frame.shape[1], int(plot_img.shape[0] * frame.shape[1] / plot_img.shape[1]))\n",
    "        )\n",
    "        \n",
    "        # Stack frames and handle video output\n",
    "        stacked_frame = cv.vconcat([frame, plot_img_resized])\n",
    "        self._handle_video_output(stacked_frame, fps)\n",
    "\n",
    "    def _handle_video_output(self, stacked_frame, fps):\n",
    "        \"\"\"Handle video output, including saving and display.\"\"\"\n",
    "        # Initialize video writer if needed\n",
    "        if self.new_w is None:\n",
    "            self.new_w = stacked_frame.shape[1]\n",
    "            self.new_h = stacked_frame.shape[0]\n",
    "            if self.save_video:\n",
    "                self.out = cv.VideoWriter(\n",
    "                    self.output_filename,\n",
    "                    cv.VideoWriter_fourcc(*\"mp4v\"),\n",
    "                    fps,\n",
    "                    (self.new_w, self.new_h)\n",
    "                )\n",
    "\n",
    "        # Save frame if requested\n",
    "        if self.save_video:\n",
    "            self.out.write(stacked_frame)\n",
    "\n",
    "        # Display frame\n",
    "        resizing_factor = 0.4\n",
    "        resized_shape = (\n",
    "            int(resizing_factor * stacked_frame.shape[1]),\n",
    "            int(resizing_factor * stacked_frame.shape[0])\n",
    "        )\n",
    "        stacked_frame_resized = cv.resize(stacked_frame, resized_shape)\n",
    "        cv.imshow(\"Video with EAR Plot\", stacked_frame_resized)\n",
    "\n",
    "    def plot_to_image(self):\n",
    "        \"\"\"Convert the matplotlib plot to an OpenCV-compatible image.\"\"\"\n",
    "        self.canvas.draw()\n",
    "        \n",
    "        buffer = self.canvas.buffer_rgba()\n",
    "        img_array = np.asarray(buffer)\n",
    "        \n",
    "        # Convert RGBA to RGB\n",
    "        img_rgb = cv.cvtColor(img_array, cv.COLOR_RGBA2RGB)\n",
    "        return img_rgb\n",
    "\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    # Example usage\n",
    "    input_video_path = \"DATA/VIDEOS/INPUTS/blinking_1.mp4\"\n",
    "    blink_counter = BlinkCounterandEARPlot(\n",
    "        video_path=input_video_path,\n",
    "        threshold=0.294,\n",
    "        consec_frames=3,\n",
    "        save_video=True,\n",
    "        output_filename=\"blinking_1_output.mp4\"\n",
    "    )\n",
    "    blink_counter.process_video()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_video_path = \"DATA/VIDEOS/INPUTS/blinking_0.mp4\"\n",
    "blink_counter = BlinkCounter(\n",
    "    video_path=input_video_path,\n",
    "    threshold=0.275,\n",
    "    consec_frames=4,\n",
    "    save_video=True,\n",
    "    output_filename=\"blinking_0_output.mp4\"\n",
    ")\n",
    "blink_counter.process_video()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Applications\n",
    "\n",
    "1. **Driver Safety Systems**: Real-time monitoring of driver blink patterns to detect drowsiness and fatigue, triggering alerts or emergency protocols to prevent accidents. These systems are increasingly being integrated into modern vehicles and fleet management solutions.\n",
    "\n",
    "2. **Medical Diagnostics**: \n",
    "   - Detection of neurological conditions through abnormal blink patterns\n",
    "   - Assessment of facial nerve function after surgery or injury\n",
    "   - Monitoring of patients with conditions like [Bell's palsy](https://www.ninds.nih.gov/health-information/disorders/bells-palsy#:~:text=Bell's%20palsy%20is%20a%20neurological,injured%20or%20stops%20working%20properly.)\n",
    "   - Early detection of conditions like [Parkinson's disease](https://pubmed.ncbi.nlm.nih.gov/2265915/#:~:text=Parkinson's%20disease%20is%20associated%20with,striatal%20and%20mesolimbic%20dopaminergic%20activity.) which can affect blink rates\n",
    "\n",
    "3. **Human-Computer Interaction**:\n",
    "   - Assistive technology for people with mobility impairments, using blinks as input signals\n",
    "   - Virtual reality systems that track blinks to reduce eye strain\n",
    "   - Gaming interfaces that incorporate blink detection for more immersive experiences\n",
    "\n",
    "4. **Security and Authentication**:\n",
    "   - Liveness detection in facial recognition systems to prevent spoofing\n",
    "   - Biometric authentication using unique blink patterns\n",
    "   - Monitoring systems for high-security environments\n",
    "\n",
    "5. **Healthcare Monitoring**:\n",
    "   - Remote patient monitoring systems\n",
    "   - Sleep studies and sleep disorder diagnosis\n",
    "   - Assessment of medication effects on neurological function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. References\n",
    "- [MediaPipe Face Mesh](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker)\n",
    "- [Eye Aspect Ratio (EAR)](https://medium.com/analytics-vidhya/eye-aspect-ratio-ear-and-drowsiness-detector-using-dlib-a0b2c292d706)\n",
    "- [Dewi, C., Chen, R., Chang, C., Wu, S., Jiang, X., & Yu, H. (2021). Eye Aspect Ratio for Real-Time Drowsiness Detection to Improve Driver Safety. Electronics, 11(19), 3183](https://doi.org/10.3390/electronics11193183).\n",
    "- [Soukupová, E., & Čech, M. (2016). Real-Time Eye Blink Detection using Facial Landmarks. *Computer Vision and Applications*, 2016(1), 1-10](https://doi.org/10.1007/s00521-016-0934-z).\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}