Occlusion-aware driver monitoring system

Published on:

Paper accepted at IEEE IV 2026 on occlusion-aware driver monitoring

A paper from the CERTAIN project has been accepted for presentation at IEEE IV 2026, the IEEE Intelligent Vehicles Symposium, one of the leading international conferences in the field of intelligent and automated vehicles. The paper will be presented in Detroit, USA, from 22 to 25 June 2026.

The paper, titled “Occlusion-Aware Driver Monitoring using VLM-Enhanced Situational Understanding” (pre-print), introduces a robust driver monitoring system that explicitly accounts for face occlusions - a known challenge for reliable in-vehicle monitoring.

The proposed system combines driver identification, gaze estimation by regions, distraction detection, and face occlusion detection. A key novelty is the use of Vision–Language Models (VLMs) to not only detect occlusions, but also to understand their cause (for example, hands, sunglasses, or the driver looking away), even under challenging lighting conditions. This improves situational awareness and helps indicate when system performance may be degraded, in line with Euro NCAP recommendations.

To ensure reliability across conditions, the system uses separate algorithms trained on RGB and infrared (IR) data and is developed using the multimodal Driver Monitoring Dataset (DMD). The paper details how these components are integrated into a single pipeline and validated both on the dataset and in real-world driving scenarios. Results show strong overall performance, with RGB-based models performing particularly well, and demonstrate the added value of explicit occlusion detection in driver monitoring systems.

Reference
Cañas Rodriguez, P. N., Diez, A., Nieto, M., Rodriguez, I., Sghairi, O., & Sánchez Juanola, M. (2026). Occlusion-Aware Driver Monitoring using VLM-Enhanced Situational Understanding. Zenodo. https://doi.org/10.5281/zenodo.18599568. Pre-print, accepted at IEEE IV 2026.

Video demonstration
The following video demonstration shows the system in operation, including real-time detection and interpretation of occlusion causes.