Driver drowsiness detection

The problem

Fatigue is behind a large share of road crashes, and most cars on the road have no driver monitoring at all. The system had to be cheap (one webcam), non-intrusive (nothing worn), and quiet: a monitor that alarms every time someone blinks or talks gets switched off.

Data

• MRL Eye Dataset: about 85,000 images of open and closed eyes, for eye-state classification.
• YawDD: 351 videos of drivers, used to separate real yawns from talking and singing.

Frames are downsampled to 320×240 before inference to keep the pipeline fast on a CPU.

Architecture

1. Face landmarksMediaPipe extracts 468 facial landmarks per frame, giving the eye and mouth aspect ratios (EAR, MAR).
2. Two CNN classifiersSeparate MobileNetV2 models for eye state and yawning, each fine-tuned in two phases of transfer learning.
3. Double-lock checkThe CNN's prediction must agree with the geometric EAR/MAR measurement before it counts.
4. Temporal decisionA 60-frame smoothing window ignores single-frame blips. Eyes closed for 7 seconds raises a critical alert; 2 or more yawns within 2 minutes raises a fatigue alert.

A FastAPI backend streams the driver's state over WebSockets to a live dashboard, and a confirmed alert emails a supervisor automatically.

Results

Accuracy and speed against targets

Measure	Target	Achieved
Eye-state accuracy, MRL test set	96%	99%
Yawn accuracy, YawDD	90%	96%
Frame rate on CPU	–	40+ FPS
End-to-end latency	–	under 50 ms

Grad-CAM heatmaps confirmed the models look at the eyes and mouth, not at background or lighting cues that happen to correlate with the labels.

Where it breaks

• Sunglasses, a cap brim or a hand over the face hide exactly the regions both models need.
• An ordinary webcam has no infrared, so night driving without cabin light is outside what it can see.
• The 7-second rule is deliberately conservative to avoid false alarms. A one- or two-second microsleep won't trigger the critical alert on its own.
• Benchmark accuracy is per image on curated datasets. It is not the same as a measured false-alarm rate over real hours of driving, which is the number that would decide whether this ships in a car.