Developing a driver monitoring system that can assess the driver’s state is a prerequisite and a key to improving the road safety. With the success of deep learning, such systems can achieve a high accuracy if corresponding high-quality datasets are available. In this paper, we introduce DriverMVT (Driver Monitoring dataset with Videos and Telemetry). The dataset contains information about the driver head pose, heart rate, and driver behaviour inside the cabin like drowsiness and unfastened belt. This dataset can be used to train and evaluate deep learning models to estimate the driver’s health state, mental state, concentration level, and his/her activity in the cabin. Developing such systems that can alert the driver in case of drowsiness or distraction can reduce the number of accidents and increase the safety on the road. The dataset contains 1506 videos for 9 different drivers (7 males and 2 females) with total number of frames equal 5119k and total time over 36 h. In addition, evaluated the dataset with multi-task temporal shift convolutional attention network (MTTS-CAN) algorithm. The algorithm mean average error on our dataset is 16.375 heartbeats per minute.

Driving behaviour analysis has drawn much attention in recent years due to the dramatic increase in the number of traffic accidents and casualties, and based on many studies, there is a relationship between the driving environment or behaviour and the driver’s state. To the best of our knowledge, these studies mostly investigate relationships between one vital sign and the driving circumstances either inside or outside the cabin. Hence, our paper provides an analysis of the correlation between the driver state (vital signs, eye state, and head pose) and both the vehicle maneuver actions (caused by the driver) and external events (carried out by other vehicles or pedestrians), including the proximity to other vehicles. Our methodology employs several models developed in our previous work to estimate respiratory rate, heart rate, blood pressure, oxygen saturation, head pose, eye state from in-cabin videos, and the distance to the nearest vehicle from out-cabin videos. Additionally, new models have been developed using Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) to classify the external events from out-cabin videos, as well as a Decision Tree classifier to detect the driver’s maneuver using accelerometer and gyroscope sensor data. The dataset used includes synchronized in-cabin/out-cabin videos and sensor data, allowing for the estimation of the driver state, proximity to other vehicles and detection of external events, and driver maneuvers. Therefore, the correlation matrix was calculated between all variables to be analysed. The results indicate that there is a weak correlation connecting both the maneuver action and the overtaking external event on one side and the heart rate and the blood pressure (systolic and diastolic) on the other side. In addition, the findings suggest a correlation between the yaw angle of the head and the overtaking event and a negative correlation between the systolic blood pressure and the distance to the nearest vehicle. Our findings align with our initial hypotheses, particularly concerning the impact of performing a maneuver or experiencing a cautious event, such as overtaking, on heart rate and blood pressure due to the agitation and tension resulting from such events. These results can be the key to implementing a sophisticated safety system aimed at maintaining the driver’s stable state when aggressive external events or maneuvers occur.

This paper presents a computer vision-based approach to chronic subdural hematoma segmentation that can be performed by one click. Chronic subdural hematoma is estimated to occur in 0.002–0.02% of the general population each year and the risk increases with age, with a high frequency of about 0.05–0.06% in people aged 70 years and above. In our research, we developed our own dataset, which includes 53 series of CT scans collected from 21 patients with one or two hematomas. Based on the dataset, we trained two neural network models based on U-Net architecture to automate the manual segmentation process. One of the models performed segmentation based only on the current frame, while the other additionally processed multiple adjacent images to provide context, a technique that is more similar to the behavior of a doctor. We used a 10-fold cross-validation technique to better estimate the developed models’ efficiency. We used the Dice metric for segmentation accuracy estimation, which was 0.77. Also, for testing our approach, we used scans from five additional patients who did not form part of the dataset, and created a scenario in which three medical experts carried out a hematoma segmentation before we carried out segmentation using our best model. We developed the OsiriX DICOM Viewer plugin to implement our solution into the segmentation process. We compared the segmentation time, which was more than seven times faster using the one-click approach, and the experts agreed that the segmentation quality was acceptable for clinical usage.

Heart rate is an essential vital sign to evaluate human health. Remote heart monitoring using cheaply available devices has become a necessity in the twenty-first century to prevent any unfortunate situation caused by the hectic pace of life. In this paper, we propose a new method based on the transformer architecture with a multi-skip connection biLSTM decoder to estimate heart rate remotely from videos. Our method is based on the skin color variation caused by the change in blood volume in its surface. The presented heart rate estimation framework consists of three main steps: (1) the segmentation of the facial region of interest (ROI) based on the landmarks obtained by 3DDFA; (2) the extraction of the spatial and global features; and (3) the estimation of the heart rate value from the obtained features based on the proposed method. This paper investigates which feature extractor performs better by captioning the change in skin color related to the heart rate as well as the optimal number of frames needed to achieve better accuracy. Experiments were conducted using two publicly available datasets (LGI-PPGI and Vision for Vitals) and our own in-the-wild dataset (12 videos collected by four drivers). The experiments showed that our approach achieved better results than the previously published methods, making it the new state of the art on these datasets.

In this paper, we present a two stages solution to 3D vehicle detection and segmentation. The first stage depends on the combination of EfficientNetB3 architecture with multiparallel residual blocks (inspired by CenterNet architecture) for 3D localization and poses estimation for vehicles on the scene. The second stage takes the output of the first stage as input (cropped car images) to train EfficientNet B3 for the image recognition task. Using predefined 3D Models, we substitute each vehicle on the scene with its match using the rotation matrix and translation vector from the first stage to get the 3D detection bounding boxes and segmentation masks. We trained our models on an open-source dataset (ApolloCar3D). Our method outperforms all published solutions in terms of 6 degrees of freedom error (6 DoF err).

This paper addresses the problem of constructing a real-time 2D map for driving scenes from a single monocular RGB image. We presented a method based on three neural networks (depth estimation, 3D object detection, and semantic segmentation). We proposed a depth estimation neural network architecture that is fast and accurate in comparison with the state-of-the-art models. We designed our model to work in real time on light devices (such as an NVIDIA Jetson Nano and smartphones). The model is based on an encoder–decoder architecture with complex loss functions, i.e., normal loss, VNL, gradient loss (dx, dy), and mean absolute error. Our results show competitive results in comparison with the state-of-the-art methods, as our method is 30 times faster and smaller.