In this study, the authors analyse the exponential growth of advanced driver assistance systems based on video processing in the past decade. Specifically, they focus on how research and innovative ideas can finally reach the market as cost‐effective solutions. They explore well‐known computer vision methods for services like lane departure warning systems, collision avoidance systems and point out potential future trends according to a review of the state‐of‐the‐art. Along this study, the authors’ own contributions are described as examples of such systems from the perspective of real‐time by design, pursuing a trade‐off between the accuracy and reliability of the designed algorithms and the restrictive computational, economical and design requisites of embedded platforms.

Tremendous advances in advanced driver assistance systems (ADAS) have been possible thanks to the emergence of deep neural networks (DNN) and Big Data (BD) technologies. Huge volumes of data can be managed and consumed as training material to create DNN models which feed functions such as lane keeping systems (LKS), automated emergency braking (AEB), lane change assistance (LCA), etc. In the ADAS/AD domain, these advances are only possible thanks to the creation and publication of large and complex datasets, which can be used by the scientific community to benchmark and leverage research and development activities. In particular, multi-modal datasets have the potential to feed DNN that fuse information from different sensors or input modalities, producing optimised models that exploit modality redundancy, correlation, complementariness and association. Creating such datasets pose a scientific and engineering challenge. The BD dimensions to cover are volume (large datasets), variety (wide range of scenarios and context), veracity (data labels are verified), visualization (data can be interpreted) and value (data is useful). In this paper, we explore the requirements and technical approach to build a multi-sensor, multi-modal dataset for video-based applications in the ADAS/AD domain. The Driver Monitoring Dataset (DMD) was created and partially released to foster research and development on driver monitoring systems (DMS), as it is a particular sub-case which receives less attention than exterior perception. Details on the preparation, construction, post-processing, labelling and publication of the dataset are presented in this paper, along with the announcement of a subsequent release of DMD material publicly available for the community.

Vehicles are sophisticated machines equipped with sensors that provide real-time data for onboard driving assistance systems. Due to the wide variety of traffic, road, and weather conditions, continuous system enhancements are essential. Connectivity allows vehicles to transmit previously unknown data, expanding datasets and accelerating the development of new data models. This enables faster identification and integration of novel data, improving system reliability and reducing time to market. Data Spaces aim to create a data-driven, interconnected, and innovative data economy, where edge and cloud infrastructures support a virtualised IoT platform that connects data sources and development servers. This paper proposes an edge-cloud data platform to connect car data producers with multiple and heterogeneous services, addressing key challenges in Data Spaces, such as data sovereignty, governance, interoperability, and privacy. The paper also evaluates the data platform's performance limits for text, image, and video data workloads, examines the impact of connectivity technologies, and assesses latencies. The results show that latencies drop to 33ms with 5G connectivity when pipelining data to consuming applications hosted at the edge, compared to around 77ms when crossing both edge and cloud processing infrastructures. The results offer guidance on the necessary processing assets to avoid bottlenecks in car data platforms.

Humans are constantly striving to improve their transportation resources. The invention of the combustion engine car is considered a milestone in the evolution of modern transportation. However, engineers are continually pushing forward the optimization of vehicles. In this context, the development of advanced driving assistance systems (ADAS) has provided vehicles with functions that improve both security and comfort. In fact, the automobile industry has embraced the task of developing increasingly autonomous vehicles, which relieve the human driver of some of the driving tasks. The evolution of vehicles requires the inclusion of driver monitoring systems (DMS). Notably, DMS is a key element in EuroNCAP evaluation and Europe safety regulations, and it is expected that most vehicles will be equipped with different types of DMS. The goal of DMS is to assess the driver's capabilities to perform driving tasks, so observing the driver is necessary. Unobtrusive methods to assess the driver's state seem to be the most suitable approaches to building DMS. Among these, vision-based systems stand out as methods for observing the driver using cameras, as they offer the ability to infer visible human state impairments. Human state physiognomy is complex and depends on the individual characteristics of each person. However, some common phenomena could be identified, which allow the classification of signs of driver state impairment. The evaluation of driver capacities is usually performed, which can be degraded by episodes of inattention, putting the car occupants safety at risk. This thesis analyzes the most relevant DMS requirements based on human physiognomy signs of inattention to enable the development of unobtrusive camera-based components for DMS. The work done aims to provide researchers and industrial practitioners with an overview of driver state characteristics that are useful in building DMS components and tools. Camera-based methods for DMS are based on computer vision (CV) technologies,which are increasingly enhanced by deep learning (DL) methods. However, these methods require large datasets. Therefore, this thesis presents a methodology for building driver monitoring (DM) datasets. To apply the presented methodology, the DMD is built and made available to the research community. The DMD is a multi-modal, multi-camera dataset aimed at covering the most relevant characteristics of fatigue and distraction states. Moreover, the DMD is currently one of the most diverse datasets related to DM. Additionally, the analysis of DMS requirements has led to the development of a real-time software framework. This thesis elaborates on the classification of the typology of algorithms and proposes a modular framework for developing complex yet optimized real-time DMS architectures. The DMSLib framework is based on the definition of abstract processing units called Analyzers, which perform specific tasks. These units can be interconnected with each other, sharing common information and results by using defined interfaces. By connecting different components, it is possible to generate complex DMS pipelines that fulfill any monitoring requirements. A key benefit of the proposed framework is the optimization of DMS pipelines reducing the time to market efforts for OEMs and Tier 1. The DMD dataset and DMSLib framework have been used to develop different DM vision-based methods. First, this thesis shows the application in DM use cases, such as eyelid aperture estimation, driver gaze estimation and driver action recognition. Second, a novel method to estimate the driver's eyelid aperture is proposed. This method is based on the capacity to adapt to the human physiognomy of eye shape, which could affect the driver's fatigue state. Finally, as proof of industrial viability, some DM functions have been integrated into real vehicle systems to validate the integration with autonomous driving (AD) functions and IoT capabilities. RESUMEN Los seres humanos están constantemente esforzándose por mejorar sus formas de transporte. La invención del automóvil es un hito en la evolución del transporte moderno. Sin embargo, los esfuerzos continúan en la mejora de los vehículos. En este contexto, el desarrollo de sistemas ADAS ha proporcionado a los vehículos funciones que mejoran tanto la seguridad como la comodidad. De hecho, la industria tiene la tarea de desarrollar vehículos cada vez más autónomos, que liberan al conductor de algunas de las tareas requeridas al conducir un automóvil. El avance de los vehículos requiere incluir sistemas de monitorización del conductor (DMS). La inclusión de DMS es un elemento clave en la evaluación de la seguridad realizada por EuroNCAP y en las regulaciones europeas. El objetivo de los DMS es evaluar el estado del conductor, por lo que es necesario observar al conductor. Los métodos no intrusivos para evaluar el estado del conductor son los más adecuados para construir DMS. Entre estos, los sistemas basados en visión destacan, ya que ofrecen la capacidad de inferir aspectos visibles en el conductor. La fisiología del estado humano es compleja y depende de las características individuales de cada persona. Las capacidades del conductor pueden verse degradadas por episodios de falta de atención. Este trabajo analiza las características de falta de atención en la fisonomía humana para permitir el desarrollo de componentes basados en cámaras y datos para monitorización del conductor. Además, el objetivo es proporcionar una visión general de las características del estado del conductor que son útiles para construir componentes de DMS. Los métodos basados en cámaras se basan en tecnologías de visión por computador (CV) y aprendizaje automático (ML). Los resultados del Deep Learning (DL) han demostrado ser muy...