The ability to detect vehicle accidents from on-board sensor data is of the utmost importance to provide prompt assistance to prevent injuries and fatalities. In this article, we present a novel deep learning method capable of analyzing time series recorded from Inertial Measurement Units (IMU) and GPS devices to recognize the presence of an accident along with its severity. We propose a neural architecture capable of exploiting the different sensor streams (i.e., acceleration, gyroscope, and GPS speed), a multimodal contrastive self-supervised training procedure, and an ad-hoc stack of data augmentation techniques, specifically designed to counteract the extreme class imbalance and to improve the generalization capabilities of the whole pipeline. The proposed method has been validated against several state-of-the-art methods on a large and highly imbalanced dataset, composed of more than 200 thousand time series collected from US vehicles, with different vehicle sizes and traveling on different types of road. Our method achieves an average-precision score (AP) of 0.9 in the detection of crashes and 0.76 in the detection of severe crashes, significantly outperforming all the other approaches, and has small footprint and latency, so that it can easily be deployed on embedded devices.

In this paper, we propose a novel deep learning architecture to classify unsafe driving maneuvers from dashcam and IMU data. Such architecture processes the output of an object detection algorithm in combination with raw video frames and GPS/IMU data. At the core of the architecture there is a novel Spatio-Temporal Attention Selector (STAS) module, which (1) extracts features describing the evolution of each object in the scene over time and (2) leverages multi-head dot product attention to select the relevant ones, i.e., the dangerous ones or the ones in danger, to perform classification. We also introduce a simple but effective methodology to increase the benefit of fine-tuning the backbone network. Our method is shown to achieve higher performance than other approaches in the literature applying attention over single frames.

In this article we present a novel machine learning pipeline for automatic detection of stop sign violations from dashcam videos, Inertial Measurement Units (IMU) and Global Positioning System (GPS) data. We developed a two-step approach, including a detector (Stop Sign Detector) capable of identifying stop signs presence, position, and size within video frames, followed by a classifier (Stop Violation Classifier) that assesses the presence of violations along with a severity score. The Stop Sign Detector is a deep convolutional neural network (CNN) for image classification, which leverages the information contained in its deeper layer feature maps in order to extract estimates of position and size of the detected stop signs. The Stop Violation Classifier fuses the information provided by the Stop Sign Detector with IMU/GPS data to assess the presence and severity of a stop sign violation. The proposed approach has been tested on several thousands of real-world videos, recorded from US vehicles, in all kinds of weather conditions, times of the day and environments. Our method achieves an area under the precision-recall curve of 94% with a required computational time of 2.4 seconds to process a 16-second video entirely on CPU.

Hotspot detection and classification for a fleet of vehicles is usually performed based on GPS data sampled from the vehicles. In this paper, we explore how the integration of satellite images can improve GPS-based hotspot classification. We propose a system composed of a deep Convolutional Neural Network (CNN) for image classification and a Random Forest classifier that combines GPS-based features with the CNN output for hotspot classification. We introduce also a novel metric for scoring place detection and classification systems, able to account for both detection and classification errors. The new metric is used to assess experimentally the effectiveness of our system in combining the two sources of information.

Vehicle viewpoint estimation from monocular images is a crucial component for autonomous driving vehicles and for fleet management applications. In this paper, we make several contributions to advance the state-of-the-art on this problem. We show the effectiveness of applying a smoothing filter to the output neurons of a Convolutional Neural Network (CNN) when estimating vehicle viewpoint. We point out the overlooked fact that, under the same viewpoint, the appearance of a vehicle is strongly influenced by its position in the image plane, which renders viewpoint estimation from appearance an ill-posed problem. We show how, by inserting in the model a CoordConv layer to provide the coordinates of the vehicle, we are able to solve such ambiguity and greatly increase performance. Finally, we introduce a new data augmentation technique that improves viewpoint estimation on vehicles that are closer to the camera or partially occluded. All these improvements let a lightweight CNN reach optimal results while keeping inference time low. An extensive evaluation on a viewpoint estimation benchmark () and on actual vehicle camera data () shows that our method significantly outperforms the state-of-the-art in vehicle viewpoint estimation, both in terms of accuracy and memory footprint.