As deep learning models have become increasingly complex, it is critical to understand their decision-making, particularly in safety-relevant applications. In order to support a quantitative interpretation of an autonomous agent trained through Deep Reinforcement Learning (DRL) in the highway-env simulation environment, we propose a framework featuring three types of views for analyzing data: (i) episode timeline, (ii) frame by frame, and (iii) aggregated statistical analysis, also including heatmaps for a better spatial understanding. Our methodology allowed a novel, consistent description of the behavior of the agent. The main motivator for the taken action is typically the longitudinal distance from the second-closest and, to a lower extent, third-closest vehicle. In the overtakes, also the agent’s position in lanes becomes relevant. The analysis identified interesting patterns and an issue in the last frames of an episode, when the agent is unable to overtake the last two vehicles, arguably because of the lack of reference vehicles ahead. We observed a clear differentiation between attention and SHAP values (estimating the importance of each feature for each decision), reflecting the architecture of the neural network, where the first layer implements the attention mechanism, while the deeper ones make the actual decision. Attention focuses on the proximity of the ego, while the decision is taken on a wider horizon, denoting a valuable anticipation capability. To support research, the proposed framework is released as open source.

Structural health monitoring is key in civil engineering because of the importance and the aging of the infrastructure. We argue that applying leading-edge, data-driven methods of large-scale complex industrial systems may be beneficial, particularly for accuracy and responsiveness. A fundamental step concerns the identification of the best tools to extract meaningful information from the vibrational raw signals. To this end, we study the application of two convolutional neural network architectures that have emerged in literature for efficient feature extraction from time-series, namely WaveNet and MiniRocket. The test bench is the Z24 Bridge progressive damage test (PDT) classification dataset. Results show that a model based on WaveNet reaches state-of-the-art performance, also reducing model size and computational complexity. WaveNet proves perfectly suited to interpret the bridge vibration waveforms directly in the time domain, without any specific preprocessing. On the other hand, MiniRocket excels for ease of configuration (only two hyperparameters are to be tweaked), overall training efficiency, and model size, lending itself as a valuable agile alternative (e.g., for rapid prototyping). Our main advancement is thus the identification and characterization of highly effective feature extraction methods, employable in different SHM tasks. We have assessed the performance of the models on two embedded platforms, proposing a smart sensor system where a local hub collects the signals from a constellation of inertial sensors and infers damage assessment onsite, allowing the bridge to self-assess its health state without resorting to connectivity nor cloud resources.

Binarization is an extreme quantization technique that is attracting research in the Internet of Things (IoT) field, as it radically reduces the memory footprint of deep neural networks without a correspondingly significant accuracy drop. To support the effective deployment of Binarized Neural Networks (BNNs), we propose CBin-NN, a library of layer operators that allows the building of simple yet flexible convolutional neural networks (CNNs) with binary weights and activations. CBin-NN is platform-independent and is thus portable to virtually any software-programmable device. Experimental analysis on the CIFAR-10 dataset shows that our library, compared to a set of state-of-the-art inference engines, speeds up inference by 3.6 times and reduces the memory required to store model weights and activations by 7.5 times and 28 times, respectively, at the cost of slightly lower accuracy (2.5%). An ablation study stresses the importance of a Quantized Input Quantized Kernel Convolution layer to improve accuracy and reduce latency at the cost of a slight increase in model size.

Driver distraction is one of the main causes of traffic accidents. While there are different types of distraction (manual, visual, cognitive), cognitive distraction is particularly challenging, being only partially related to visual features detectable through cameras or an eye tracker system. Moreover, since cognitive distraction is not a point in time phenomenon, spotting this kind of distraction requires the processing of a certain time interval, which poses a further challenge for real-time performance. After a data collection campaign with N=42 subjects undertaking a twenty-question task (TQT) in a driving simulator, we developed a driver cognitive distraction detection system, addressing some key gaps we identified in literature. First, we assessed the effectiveness of state-of-the-art time-series-oriented deep learning models in learning features from 60 Hz raw input signals, thus implementing an end-to-end machine learning approach, without manual feature engineering. We demonstrated that such models are able to classify time-windows as small as 0.5 seconds, and are also more robust to sensor failures. Then, also with the support of AI explainability, we showed that processing vehicular data is fundamental to ensure performance, while physiological signals provide a less important, but still useful, contribution. Moreover, through a between- and within-subject design comparison, we showed that eye-tracker and, particularly, physiological signals are much more prone to inter-individual variability, thus overfitting. This is fundamental to consider for commercial deployment, as it would require fine-tuning the system with data from the actual end-user. Our analysis has quantitatively measured the effect of such variability for all types of signals, demonstrating its huge relevance, and shown that deep learning models dedicated to timeseries processing are better able to generalize across users than the more commonly employed shallow machine learning models. With a focus on in-vehicle deployability, which is of significant industrial interest, we measured also such metrics as model size, inference time, and energy consumption, showing feasibility on two embedded platforms, which is a key advancement towards on-board deployment of robust, real-time cognitive distraction detection systems.

Tiny machine learning technologies are bringing intelligence ever closer to the sensor, thus enabling the key benefits of edge computing (e.g., reduced latency, improved data security, higher energy efficiency, and lower bandwidth consumption, also without the need for constant connectivity). This promises to significantly enhance industrial applications but requires suited development tools to deal with the complexity of the edge technologies and context. We propose an agile Jupyter Python notebook as a simple, manageable tool to efficiently and effectively develop microcontroller-based intelligent imaging classification sensors. The notebook implements a methodology involving hyperparameter tuning and comparison of different shallow and deep learning models, with quantization. It exports TensorFlow Lite models, deployable on several microcontroller families, and optionally exploits the STM32Cube.AI Developer Cloud service, which allows benchmarking the developed models on a set of real-world tiny hardware target platforms. Assessment concerns various types of metrics, both for machine learning (e.g., accuracy) and embedded systems (e.g., memory footprint, latency, energy consumption). We have verified the support for development effectiveness and efficiency on four ultra-low resolution image-classification datasets, with different levels of input and task complexity. In all cases, the tool was able to build microcontroller-deployment-ready, beyond the state-of-the-art models, within one hour on Google Colab CPUs.