The growing popularity of edgeAI requires novel solutions to support the deployment of compute-intense algorithms in embedded devices. In this article, we advocate for a holistic approach, where application-level transformations are jointly conceived with dedicated hardware platforms. We embody such a stance in a strategy that employs ensemble-based algorithmic transformations to increase robustness and accuracy in Convolutional Neural Networks (CNNs), enabling the aggressive quantization of weights and activations. Opportunities offered by algorithmic optimizations are then harnessed in domain-specific hardware solutions, such as the use of multiple ultra-low-power processing cores, the provision of shared acceleration resources, the presence of independently power-managed memory banks, and voltage scaling to ultra-low levels, greatly reducing (up to 60% in our experiments) energy requirements. Furthermore, we show that aggressive quantization schemes can be leveraged to perform efficient computations directly in memory banks, adopting in-memory computing solutions. We showcase that the combination of parallel in-memory execution and aggressive quantization leads to more than 70% energy and latency gains compared to baseline implementations.

To reduce energy consumption, it is possible to operate embedded systems at sub-nominal conditions (e.g., reduced voltage, limited eDRAM refresh rate) that can introduce bit errors in their memories. These errors can affect the stored values of convolutional neural network (CNN) weights and activations, compromising their accuracy. In this article, we introduce Embedded Ensemble CNNs (E ² CNNs), our architectural design methodology to conceive ensembles of convolutional neural networks to improve robustness against memory errors compared to a single-instance network. Ensembles of CNNs have been previously proposed to increase accuracy at the cost of replicating similar or different architectures. Unfortunately, state-of-the-art (SoA) ensembles do not suit well embedded systems, in which memory and processing constraints limit the number of deployable models. Our proposed architecture solves that limitation applying SoA compression methods to produce an ensemble with the same memory requirements of the original architecture, but with improved error robustness. Then, as part of our new E ² CNNs design methodology, we propose a heuristic method to automate the design of the voter-based ensemble architecture that maximizes accuracy for the expected memory error rate while bounding the design effort. To evaluate the robustness of E ² CNNs for different error types and densities, and their ability to achieve energy savings, we propose three error models that simulate the behavior of SRAM and eDRAM operating at sub-nominal conditions. Our results show that E ² CNNs achieves energy savings of up to 80 percent for LeNet-5, 90 percent for AlexNet, 60 percent for GoogLeNet, 60 percent for MobileNet and 60 percent for an optimized industrial CNN, while minimizing the impact on accuracy. Furthermore, the memory size can be decreased up to 54 percent by reducing the number of members in the ensemble, with a more limited impact on the original accuracy than obtained through pruning alone.

In the Internet-of-Things (IoT) era where biomedical applications require continuous monitoring of relevant data, edge computing keeps gaining importance. These new architectures for edge computing include multi-core and parallel computing capabilities that can enable prevention, diagnosis and treatment of diseases in ambulatory or home-based setups. We explore the benefits of the parallelization capabilities and computing heterogeneity of new wearable sensors in the context of a personalized online atrial fibrillation (AF) prediction method for daily monitoring. First, we optimize a single-core design to reduce energy, based on patient-specific training models. Second, we explore multi-core and memory banks configuration changes to adapt the computation and storage requirements to the characteristics of each patient. We evaluate our methodology on the public Physionet Prediction Challenge (2001) database, assessing energy consumption in single-core (ARM Cortex-M3) and new ultra-low power multi-core architectures (open-source RISC-V) for next-generation wearable platforms. Our application level exploration highlights that a parallelization approach for personalized AF in multi-core wearable sensors enables 24% energy savings with respect to single-core sensors. Moreover, the adaptation of the memory subsystem (bank size and number), in combination with deep sleep energy saving modes, can overall provide total energy savings up to 34%, depending on the specific patient.

X-HEEP (eXtendable Heterogeneous Energy-Efficient Platform) is an open-source1, configurable, and extensible single-core RISC-V microcontroller developed at the Embedded Systems Laboratory (ESL) of EPFL for edge-computing platforms. X-HEEP can be used standalone as a low-cost microcontroller, or it can be integrated into existing platforms to act like a peripheral subsystem, or it can be extended and customized with external peripherals and accelerators nimbly. The latter is particularly appealing for novel accelerators, memories, or peripherals designers who desire a simple controller to drive their IP and communicate with the external world using software functions. X-HEEP is built on top of existing, mature open-source IPs such as CPUs, peripherals, and many other building blocks from the OpenHW Group, the PULP team from ETH Zurich and the University of Bologna, and lowRISC. Its contribution includes its expandability, configurability, and agile use, targetting a large number of users to take one step further towards the democratization of open-source hardware.