Introduction

Every year, drowsy driving claims thousands of lives globally. In the U.S. alone, fatigue contributes to over 100,000 crashes annually, resulting in tragic losses that ripple through families and communities. Imagine a truck driver battling exhaustion on a midnight highway or a parent driving home after a sleepless night this project seeks to be their safeguard. By merging computer vision and machine learning, we developed a real-time system that detects early signs of drowsiness, drooping eyelids, prolonged blinks, or frequent yawns and alerts drivers before catastrophe strikes. Think of it as a vigilant co-pilot, tirelessly watching over those behind the wheel.

Problem Statement

Traditional drowsiness detection methods, such as wearable sensors or steering-wheel monitors, are often intrusive, costly, or prone to false alarms. A driver might forget to wear a device or disable it due to discomfort. Our mission is to design a non-invasive, camera-based solution that works seamlessly across real-world conditions: low light, drivers wearing glasses, and varying head angles. The challenge lies in balancing accuracy with real-time performance while ensuring affordability and ease of integration into existing vehicles.

Methodology

The system employs a multi-stage workflow:

1. Face Detection: A webcam captures live video, and OpenCV’s Haar Cascade classifier identifies the driver’s face.

2. Facial Landmark Detection: The dlib library pinpoints 68 facial landmarks, isolating regions for the eyes and mouth.

3. Eye Aspect Ratio (EAR) Calculation: The EAR, derived from vertical and horizontal distances between eye landmarks, determines blink duration. A closed eye corresponds to a sharp drop in EAR.
   
   

   

   
   

4. Yawning Detection: Mouth aspect ratio (MAR) is computed similarly to identify prolonged yawning.
   
   

   

   
   

5. Alert System: If eyes remain closed or yawns exceed predefined thresholds (e.g., 50 consecutive frames), an audible alarm activates, and vehicle speed is reduced via microcontroller integration.
   
   

The implementation uses Python, OpenCV, and dlib, ensuring real-time processing on low-resource hardware.

Code Implementation

The following is a simplified version of the implementation in Python

Research Paper Version:
You need a Arduino camera to test this code.

import numpy as np
import cv2
import dlib
import pygame
from imutils import face_utils
from scipy.spatial import distance as dist
from threading import Thread
import time

pygame.mixer.init()

def play_alarm():
    pygame.mixer.music.load("alarm.wav")
    pygame.mixer.music.play()
    time.sleep(3)
    pygame.mixer.music.stop()

def calculate_EAR(eye):
    A = dist.euclidean(eye[1], eye[5])
    B = dist.euclidean(eye[2], eye[4])
    C = dist.euclidean(eye[0], eye[3])
    return (A + B) / (2.0 * C)

def calculate_MAR(mouth):
    A = dist.euclidean(mouth[3], mouth[9])
    B = dist.euclidean(mouth[0], mouth[6])
    return A / B

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")

vs = cv2.VideoCapture(0)

while True:
    ret, frame = vs.read()
    if not ret:
        break
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = detector(gray, 0)

    for face in faces:
        shape = predictor(gray, face)
        shape = face_utils.shape_to_np(shape)

        left_eye = shape[42:48]
        right_eye = shape[36:42]
        mouth = shape[48:60]

        EAR = (calculate_EAR(left_eye) + calculate_EAR(right_eye)) / 2.0
        MAR = calculate_MAR(mouth)

        if EAR < 0.27:
            play_alarm()
        if MAR > 0.6:
            play_alarm()

    cv2.imshow("Drowsiness Detection", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

vs.release()
cv2.destroyAllWindows()

Github Project Version:
You can test this code directly in your laptop and see the results.

import numpy as np
import cv2
import dlib
import imutils
from imutils import face_utils
from scipy.spatial import distance as dist
from threading import Thread
import pygame
import time

# Initialize pygame mixer for audio playback
pygame.mixer.init()

# Function to play alarm sound for 3 seconds
def play_alarm():
    pygame.mixer.music.load("/Users/saiprudvirajyerrapragada/Desktop/Drowsiness Detection/alarm.wav")
    pygame.mixer.music.play()
    time.sleep(3)  # Play sound for 3 seconds
    pygame.mixer.music.stop()  # Stop playback

# Function to calculate Eye Aspect Ratio (EAR)
def calculate_EAR(eye):
    A = dist.euclidean(eye[1], eye[5])  # Vertical distance 1
    B = dist.euclidean(eye[2], eye[4])  # Vertical distance 2
    C = dist.euclidean(eye[0], eye[3])  # Horizontal distance
    EAR = (A + B) / (2.0 * C)
    return EAR

# Function to calculate Mouth Aspect Ratio (MAR)
def calculate_MAR(mouth):
    A = dist.euclidean(mouth[3], mouth[9])  # Vertical distance
    B = dist.euclidean(mouth[0], mouth[6])  # Horizontal distance
    MAR = A / B
    return MAR

# Thresholds and counters (parameters for detecting drowsiness and yawning)
EYE_AR_THRESH = 0.27  # EAR threshold for detecting eye closure
EYE_AR_CONSEC_FRAMES = 25  # Minimum consecutive frames for eye closure detection
MAR_THRESH = 0.6  # MAR threshold for detecting yawning
MAR_CONSEC_FRAMES = 8  # Minimum consecutive frames for yawn detection
COUNTER = 0  # Counter for frames with eyes closed
YAWNING = False  # Boolean state indicating if a yawn is currently in progress
LAST_YAWN_TIME = 0  # Timestamp of the last detected yawn
YAWN_COOLDOWN_TIME = 10  # Cooldown time in seconds between yawns
EVENT_COUNT = 0  # Total number of detected events (yawns + eye closures)
BREAK_REMINDER_THRESHOLD = 5  # Total event count threshold for reminding the driver to take a break

# Load Dlib's face detector and facial landmark predictor
print("[INFO] Loading facial landmark predictor...")
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("/Users/saiprudvirajyerrapragada/Desktop/Drowsiness Detection/shape_predictor_68_face_landmarks.dat")

# Landmark indices for eyes and mouth
(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eye"]
(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eye"]
(mStart, mEnd) = face_utils.FACIAL_LANDMARKS_IDXS["mouth"]

# Start video stream
print("[INFO] Starting video stream...")
vs = cv2.VideoCapture(0)

while True:
    ret, frame = vs.read()
    if not ret:
        break

    frame = imutils.resize(frame, width=640)
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Detect faces
    faces = detector(gray, 0)

    for face in faces:
        # Get facial landmarks
        shape = predictor(gray, face)
        shape = face_utils.shape_to_np(shape)

        # Extract eyes and mouth
        left_eye = shape[lStart:lEnd]
        right_eye = shape[rStart:rEnd]
        mouth = shape[mStart:mEnd]

        # Calculate EAR and MAR
        left_EAR = calculate_EAR(left_eye)
        right_EAR = calculate_EAR(right_eye)
        EAR = (left_EAR + right_EAR) / 2.0
        MAR = calculate_MAR(mouth)

        # Display EAR and MAR values on screen
        cv2.putText(frame, f"EAR: {EAR:.2f}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 2)
        cv2.putText(frame, f"MAR: {MAR:.2f}", (10, 60), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 2)
        cv2.putText(frame, f"Events: {EVENT_COUNT}", (10, 90), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 2)

        # Check if EAR is below threshold (Eye closure detection)
        if EAR < EYE_AR_THRESH:
            COUNTER += 1
            if COUNTER >= EYE_AR_CONSEC_FRAMES:
                EVENT_COUNT += 1  # Increment combined event count
                COUNTER = 0  # Reset counter
                if not ALARM_ON:
                    ALARM_ON = True
                    t = Thread(target=play_alarm)
                    t.daemon = True
                    t.start()
        else:
            COUNTER = 0
            ALARM_ON = False

        # Check if MAR is above threshold (Yawning detection)
        if MAR > MAR_THRESH:
            current_time = time.time()
            if not YAWNING and (current_time - LAST_YAWN_TIME) >= YAWN_COOLDOWN_TIME:
                YAWNING = True
                LAST_YAWN_TIME = current_time  # Update last yawn time
                EVENT_COUNT += 1
                if not ALARM_ON:
                    ALARM_ON = True
                    t = Thread(target=play_alarm)
                    t.daemon = True
                    t.start()
        else:
            YAWNING = False  # Reset yawning state once the mouth is closed

        # Alert the driver to take a break if events exceed the threshold
        if EVENT_COUNT >= BREAK_REMINDER_THRESHOLD:
            cv2.putText(frame, "DROWSINESS DETECTED!", (100, 200),
                        cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 3)
            cv2.putText(frame, "TAKE A BREAK!", (100, 250),
                        cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 3)
            if not ALARM_ON:
                t = Thread(target=play_alarm)
                t.daemon = True
                t.start()

    # Display the frame
    cv2.imshow("Driver Drowsiness Detection", frame)

    # Exit on pressing 'q'
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

# Cleanup
cv2.destroyAllWindows()
vs.release()

Experimental Results

The system was tested using a 5-megapixel webcam connected to a laptop. The webcam continuously captured real-time video of the driver’s face, and the facial landmark detection algorithm was used to track the driver's eye movements and yawning behavior.

The following steps were performed during the testing phase:

1. Face Detection: The system first detected the driver’s face and tracked eye and mouth landmarks.

2. Eye and Mouth Aspect Ratio Calculation: The system measured the Eye Aspect Ratio (EAR) and Mouth Aspect Ratio (MAR) to determine whether the driver was experiencing drowsiness.

3. Drowsiness Detection and Alert Triggering: If the EAR remained below the threshold (0.27) for a given number of frames, or if excessive yawning was detected, the system triggered an alarm sound.

4. Recording Accuracy Levels: The system's accuracy was evaluated under different test scenarios, such as daytime, nighttime, participants with glasses, and participants with different facial structures.

Test Scenarios and Observations

The system was evaluated under multiple conditions to determine its effectiveness in detecting drowsiness.

From the test results, it was observed that the system performed exceptionally well in standard lighting conditions, achieving an accuracy of 95 percent. However, performance decreased in low-light environments, where the detection of eye closure was less reliable. The system was still able to detect drowsiness effectively when drivers wore glasses, but accuracy was slightly reduced for heavily tinted lenses.

The results also indicated that the system could not detect head movements or distractions, meaning that a driver could still fall asleep by looking downward or away from the camera without triggering the alarm.

Key Findings from Experimentation

1. High Accuracy in Standard Conditions: The system achieved an accuracy of 95 percent in well-lit environments.

2. Challenges in Low-Light Conditions: Accuracy decreased to around 80 percent in dim lighting. Infrared-based detection could enhance performance.

3. Effectiveness with Eyeglasses: The system could detect drowsiness in drivers wearing glasses, but heavily tinted lenses affected performance.

4. Blinking and Distractions: Rapid blinking sometimes triggered false alerts, and the system could not detect if the driver was looking away from the road.

Conclusion

The driver drowsiness detection system successfully identifies fatigue symptoms such as eye closure and yawning using computer vision and machine learning techniques. The project provides a cost-effective and non-intrusive solution to reducing road accidents caused by drowsy driving.

Key Achievements

• The system was able to effectively monitor driver fatigue in real time.

• It successfully detected eye closure and yawning, issuing alerts before an accident could occur.

• The solution was developed using only a webcam, making it affordable and easy to implement in vehicles without requiring additional hardware.

• The project demonstrated over 90 percent accuracy in standard conditions, making it a viable option for integration into real-world applications.

Limitations

While the system performed well, there were some limitations:

1. Low-Light Performance: Accuracy dropped in dim lighting. Future versions could integrate infrared cameras to improve detection at night.

2. Head Pose Estimation Not Included: The system could not detect if the driver looked away from the road. Future enhancements should integrate head pose tracking.

Future Enhancements

1. Infrared-Based Night Vision: Integrating infrared cameras will improve accuracy in low-light conditions.

2. Head Pose Detection: Using deep learning-based head tracking will help detect when drivers look away from the road.

3. Vehicle Integration: Future versions could be integrated with car control systems to automatically slow down the vehicle when drowsiness is detected.

4. Driver Behavior Monitoring: Machine learning models could be trained on driver behavior patterns to predict fatigue levels more accurately.

Final Thoughts

This project has successfully demonstrated that AI and computer vision can be used to detect driver fatigue in real time. By integrating this system into vehicles, many road accidents caused by drowsy driving could be prevented, improving road safety for both drivers and passengers.

The driver drowsiness detection system is open-source and available for further development. With future enhancements, it has the potential to become a fully automated safety feature in modern vehicles.

References

1. Borghini, G., et al. (2012). "Measuring neurophysiological signals in aircraft pilots and car drivers." Neuroscience & Biobehavioral Reviews.

2. Jap, B. T., et al. (2009). "Using EEG spectral components to assess algorithms for detecting fatigue." Expert Systems with Applications.

3. Liu, D., et al. (2010). "Drowsiness Detection Based on Eyelid Movement." IEEE Workshop on Education Technology and Computer Science.

4. Bradski, G., & Kaehler, A. (2008). Learning OpenCV. O’Reilly Media.

5. Ameratunga, S., et al. (2002). "Driver sleepiness and risk of serious injury to car occupants." British Medical Journal.
   
   For a more detailed understanding of the Driver Drowsiness Detection System, including the complete research methodology, implementation details, experimental results, and future enhancements, refer to the following resources:

Research Paper

The complete research paper, “Driver Drowsiness Detection Using AI”, has been published in the International Journal of All Research Education and Scientific Methods (IJARESM). It covers in-depth technical aspects, system architecture, and evaluation results of this project.

Read the Full Research Paper: IJARESM Publication

GitHub Repository

The GitHub repository contains the full source code of the project, implementation details, and setup instructions for running the system on a personal computer or laptop. You can download the code, explore the algorithms, and contribute to further improvements.

Explore the Source Code on GitHub: Driver Drowsiness Detection GitHub Repository