19 June 2020 in IEEE Transactions on Intelligent Vehicles
Driver maneuver prediction is of great importance in designing a modern Advanced Driver Assistance System (ADAS). Such predictions can improve driving safety by alerting the driver to the danger of unsafe or risky traffic situations. In this research, we developed a model to predict driver maneuvers, including left/right lane changes, left/right turns and driving straight forward 3.6 seconds on average before they occur in real time. We propose a deep learning method based on Long Short-Term Memory (LSTM) that utilizes data on the driver's gaze and head position as well as vehicle dynamics data. We applied our approach on real data collected during drives in an urban environment in an instrumented vehicle. In comparison with previous IOHMM techniques that predicted three maneuvers including left/right turns and driving straight, our prediction model is able to anticipate two more maneuvers. In addition to this, our experimental results show that our model using identical dataset improved F1 score by 4% and increased to 84%.
10 December 2021 in IEEE Transactions on Intelligent Vehicles
Traffic object detection and recognition systems play an essential role in Advanced Driver Assistance Systems (ADAS) and Autonomous Vehicles (AV). In this research, we focus on four important classes of traffic objects: traffic signs, road vehicles, pedestrians, and traffic lights. We first review the major traditional machine learning and deep learning methods that have been used in the literature to detect and recognize these objects. We provide a vision-based framework that detects and recognizes traffic objects inside and outside the attentional visual area of drivers. This approach uses the driver 3D absolute coordinates of the gaze point obtained by the combined, cross-calibrated use of a front-view stereo imaging system and a non-contact 3D gaze tracker. A combination of multi-scale HOG-SVM and Faster R-CNN-based models are utilized in the detection stage. The recognition stage is performed with a ResNet-101 network to verify sets of generated hypotheses. We applied our approach on real data collected during drives in an urban environment with the RoadLAB instrumented vehicle. Our framework achieved 91% of correct object detections and provided promising results in the object recognition stage.
1 December 2017 in Journal of Traffic and Transportation Engineering (English Edition)
21 November 2018 in Journal of Advanced Transportation
Our objective in this contribution is to categorize driver behavior in terms of preturning maneuvers. We analyze driving behavior in an urban environment prior to turns using data obtained from the CANbus of an instrumented vehicle during a one-hour driving period for 12 different individuals. CANbus data streams such as vehicle speed, gas pedal pressure, brake pedal pressure, steering wheel angle, and acceleration are collected and analyzed for 5, 10, and 15 seconds of driving prior to each turn. We consider all turns for each driver and extract statistical features from the signals and use cluster analysis to categorize drivers into groups reflecting different driving styles. The results show that using this approach we can effectively cluster drivers into two groups. The results show consistency in the membership within a cluster throughout the different timeframes. We conclude that driver behavior classification from such data streams is possible and we hope in the near future to devise driver descriptors that include additional maneuvers.
16 April 2022 in Journal of Advanced Transportation
A driver’s actions and intent can be factors in enabling advance driver assistance systems (ADASs) to assist drivers and avoid accidents. A driver’s gaze can provide insight into the driver’s intent or awareness of situations. Knowing that a driver gazed at a traffic sign or missed a traffic could provide indications of whether the driver is alert to impending changes in the driving environment, such as curves and stop signs. For ADASs to determine the importance of a driver seeing or missing a sign, it is important to understand the driving environment and situation. A first step is to understand what signs drivers do see or miss while driving. This contribution presents the results of analyzing driving sequences to assess traffic signs that drivers may or may not have gazed upon. The results suggest that drivers may miss 20% of traffic signs though the percentage varies depending on the type of sign. The analysis uses image sequences of the driving environment and gazes data captured during driving. The methods used in our analysis included determining whether a driver’s gaze has fallen on the image of a traffic sign or not and subsequently determining signs missed during driving. The methods presented can be useful in other scenarios involving the analysis of driver gaze and have implications for the design of future ADASs and for understanding of driver gaze and awareness.