Standard-sized autonomous vehicles have rapidly improved thanks to the breakthroughs of deep learning. However, scaling autonomous driving to mini-vehicles poses several challenges due to their limited on-board storage and computing capabilities. Moreover, autonomous systems lack robustness when deployed in dynamic environments where the underlying distribution is different from the distribution learned during training. To address these challenges, we propose a closed-loop learning flow for autonomous driving mini-vehicles that includes the target deployment environment in-the-loop. We leverage a family of compact and high-throughput tinyCNNs to control the mini-vehicle that learn by imitating a computer vision algorithm, i.e., the expert, in the target environment. Thus, the tinyCNNs, having only access to an on-board fast-rate linear camera, gain robustness to lighting conditions and improve over time. Moreover, we introduce an online predictor that can choose between different tinyCNN models at runtime—trading accuracy and latency—which minimises the inference’s energy consumption by up to 3.2×. Finally, we leverage GAP8, a parallel ultra-low-power RISC-V-based micro-controller unit (MCU), to meet the real-time inference requirements. When running the family of tinyCNNs, our solution running on GAP8 outperforms any other implementation on the STM32L4 and NXP k64f (traditional single-core MCUs), reducing the latency by over 13× and the energy consumption by 92%.

The spread of deep learning on embedded devices has prompted the development of numerous methods to optimize the deployment of deep neural networks (DNNs). Works have mainly focused on: 1) efficient DNN architectures; 2) network optimization techniques, such as pruning and quantization; 3) optimized algorithms to speed up the execution of the most computational intensive layers; and 4) dedicated hardware to accelerate the data flow and computation. However, there is a lack of research on cross-level optimization as the space of approaches becomes too large to test and obtain a globally optimized solution. Thus, leading to suboptimal deployment in terms of latency, accuracy, and memory. In this work, we first detail and analyze the methods to improve the deployment of DNNs across the different levels of software optimization. Building on this knowledge, we present an automated exploration framework to ease the deployment of DNNs. The framework relies on a reinforcement learning search that, combined with a deep learning inference framework, automatically explores the design space and learns an optimized solution that speeds up the performance and reduces the memory on embedded CPU platforms. Thus, we present a set of results for state-of-the-art DNNs on a range of Arm Cortex-A CPU platforms achieving up to $4\times $ improvement in performance and over $2\times $ reduction in memory with negligible loss in accuracy with respect to the BLAS floating-point implementation.

The size and diversity of the training datasets directly influences the decision-making process of AI models. Therefore, there is an immense need for massive and diverse datasets to enhance the deployment process of AI applications. Crowdsourcing marketplaces provide a fast and reliable alternative to the laborious data collection process. However, the existing crowdsourcing marketplaces are either centralized or do not fully provide data sovereignty. By contrast, this work proposes a decentralized crowdsourcing platform through prototypical implementation along with active involvement of business entities, that grants the users sovereignty over their collected data, named as Vision-Sovereignty Data Marketplace (ViSDM). This work contributes to the data marketplaces landscape by introducing (i) A liquid democracy-based voting system to negotiate prices between a buyer and multiple data owners, (ii) An automated AI-Based per-sample value calculation function to evaluate the data and distribute profit among the data owners.

Deep Neural Network-based Keyword Spotting accuracy degrades in noisy environments. On-site adaptation to previously unseen noise is crucial to recover accuracy loss, and on-device learning is required in scenarios where adaptation has to happen in the field. In this work, we propose a fully on-device domain adaptation system, enabling edge devices to achieve noise-robust keyword spotting. We achieve up to 14% accuracy gains over already-robust keyword spotting models, and up to 21% increments when evaluating our methodology on keyword datasets disjoint from the offline training set. In extreme edge scenarios where Keyword Spotting is critical, using as little as 10 kB of memory and only 100 labeled utterances, we enable on-device learning and demonstrate accuracy recovery of up to 5% after adapting to complex, non-stationary speech noise. We show that domain adaptation can be achieved on ultra-low-power microcontrollers with as little as 357 mJ within 14 seconds on always-on, battery-operated devices. This work is the first to demonstrate an end-to-end on-device domain adaptation system for noise robust keyword spotting models on ultra-low-power, extreme edge platforms.

Artificial Intelligence (AI) is becoming increasingly important and pervasive in the modern world. The widespread adoption of AI algorithms is reflected in the extensive range of HW devices on which they can be deployed, from high-performance computing nodes to low-power embedded devices. Given the large set of heterogeneous resources where AI algorithms can be deployed, finding the most suitable device and its con- figuration is challenging, even for experts. We propose a data-driven approach to assist AI adopters and developers in choosing the optimal HW resource. Our approach is based on three key elements: i) fair benchmarking of target AI algorithms on a set of hetero- geneous platforms, ii) creation of ML models to learn the behaviour of these AI algorithms, and iii) support guidelines to help identify the best deployment option for a given AI algorithm. We demonstrate our approach on a specific (and relevant) use case: Deep Neural Net- work (DNN) inference on embedded devices.