The growing number of low-power smart devices in the Internet of Things is coupled with the concept of “Edge Computing”, that is moving some of the intelligence, especially machine learning, towards the edge of the network. Enabling machine learning algorithms to run on resource-constrained hardware, typically on low-power smart devices, is challenging in terms of hardware (optimized and energy-efficient integrated circuits), algorithmic and firmware implementations. This paper presents FANN-on-MCU, an open-source toolkit built upon the Fast Artificial Neural Network (FANN) library to run lightweight and energy-efficient neural networks on microcontrollers based on both the ARM Cortex-M series and the novel RISC-V-based Parallel Ultra-Low-Power (PULP) platform. The toolkit takes multi-layer perceptrons trained with FANN and generates code targeted to low-power microcontrollers. This paper also presents detailed analyses of energy efficiency across the different cores, and the optimizations to handle different network sizes. Moreover, it provides a detailed analysis of parallel speedups and degradations due to parallelization overhead and memory transfers. Further evaluations include experimental results for three different applications using a self-sustainable wearable multi-sensor bracelet. Experimental results show a measured latency in the order of only a few microseconds and power consumption of a few milliwatts while keeping the memory requirements below the limitations of the targeted microcontrollers. In particular, the parallel implementation on the octa-core RISC-V platform reaches a speedup of 22x and a 69% reduction in energy consumption with respect to a single-core implementation on Cortex-M4 for continuous real-time classification.

Energy efficiency is crucial in the design of battery-powered end devices, such as smart sensors for the Internet of Things applications. Wireless communication between these distributed smart devices consumes significant energy, and even more when data need to reach several kilometers in distance. Low-power and long-range communication technologies such as LoRaWAN are becoming popular in IoT applications. However, LoRaWAN has drawbacks in terms of (i) data latency; (ii) limited control over the end devices by the gateway; and (iii) high rate of packet collisions in a dense network. To overcome these drawbacks, we present an energy-efficient network architecture and a high-efficiency on-demand time-division multiple access (TDMA) communication protocol for IoT improving both the energy efficiency and the latency of standard LoRa networks. We combine the capabilities of short-range wake-up radios to achieve ultra-low power states and asynchronous communication together with the long-range connectivity of LoRa. The proposed approach still works with the standard LoRa protocol, but improves performance with an on-demand TDMA. Thanks to the proposed network and protocol, we achieve a packet delivery ratio of 100% by eliminating the possibility of packet collisions. The network also achieves a round-trip latency on the order of milliseconds with sensing devices dissipating less than 46 mJ when active and 1.83 $\mathsf{\mu }$ W during periods of inactivity and can last up to three years on a 1200-mAh lithium polymer battery.

Low power and long-range communications are crucial features of the Internet of Things (IoT) paradigm that is becoming essential even for industrial applications. Today, the most promising long-range communication technologies are LoRaWAN and Narrow Band IoT (NB-IoT), which are driving a large IoT ecosystem. In this paper, we evaluate the performance of LoRaWAN and NB-IoT with accurate in-field measurements using the same application context for a fair comparison in terms of energy efficiency, lifetime, quality of service, and coverage. The NB-IoT energy transmission is scarcely dependent on the payload length. Thus applications that can tolerate buffering and caching techniques on the node are favored. On the other hand, LoRaWAN consumes 10x lower energy compared to NB-IoT for occasional and latency-sensitive communications, for which it enables much end-device lifetime. Finally, this paper provides design guidelines for future industrial applications with stringent requirements of long-range and low power wireless connectivity.

With the emergence of edge-computing platforms, the applications of smart wearable devices are immense. This technology can be incorporated in consumer products such as smartwatches and activity trackers, for continuous health monitoring, as well as for medical applications as myoelectric prosthetics, to interpret the electric activity in the residual limb and achieve fast and precise control. However, wearable technologies require a lightweight, energy efficient and low latency processing system in order to extend the devices’ autonomy while maintaining a realistic user-feedback interaction. Neuromorphic processing, thanks to its event-based and asynchronous nature, presents ideal characteristics for compact brain-inspired low-power and ultra-fast computing systems that can enable a new generation of wearable devices. This paper presents two Spiking Neural Networks for event-based electromyography gesture recognition and their evaluation on Intel’s research neuromorphic chip Loihi. Specifically, the evaluation is done on the Kapoho Bay platform which embeds the Loihi processor in a USB form factor device allowing for closed-loop edge computation. With accurate experimental evaluation this paper demonstrates that the proposed method is able to discriminate 12 different hand-gesture using a 8-channels electromyography sensor and exceeds state-of-the-art results. We obtained an accuracy of 74% on the commonly used NinaPro DB5 dataset, a processing latency of 5.7ms for 300ms EMG samples while consuming only 41mW.

This paper proposes a proof-of-concept device to continuously assess the usage of hand-held power tools and detect construction working tasks (e.g. different drilling works) along with potential misusages, e.g. drops, with an energy efficient architecture design. The designed device is based on Bluetooth Low Energy (BLE) and NFC connectivity. BLE is used to exchange data with a gateway, whereas NFC has been chosen as an energy-efficient wake-up mechanism. A temperature and humidity sensor is embedded to monitor storage conditions; an accelerometer for tool usage monitoring. The ARM Cortex-M4F core embedded in the BLE module is exploited to process the information at the edge. A Tiny Machine Learning (TinyML) algorithm is proposed to process the data directly on board and achieve low latency and high energy efficiency. The TinyML algorithm has been developed embedded in the proposed device to detect 4 different usage classes (tool transportation, no-load, metal and wood drilling). A dataset containing more than 280 minutes of three-axis accelerations during different activities has been acquired with the device attached to a construction rotary hammer drill, and used to train and validate the algorithm. A Network Architecture Search (NAS) has been performed to optimize the trade-off between accuracy and complexity, achieving an accuracy of 90.6% with a model size of roughly 30kB. Experimental results showed an ultra-low power consumption in sleep mode of 550 nA and a peak power consumption of 4mA while running the TinyML on the edge. This results in a balanced combination of edge processing capabilities and low power consumption, enabling to obtain a smart Internet of Things (IoT) device in the field with a long lifetime of up to 4 years in operation and 17 years in shelf mode with a standard 250mAh coin battery. This work enables a long battery lifetime operation of on device degradation and utility analysis, further closing the gap between edge processing and fine granularity data evaluation.

The LLaMA family, a collection of foundation language models ranging from 7B to 65B parameters, has become one of the most powerful open-source large language models (LLMs) and the popular LLM backbone of multi-modal large language models (MLLMs), widely used in computer vision and natural language understanding tasks. In particular, LLaMA3 models have recently been released and have achieved impressive performance in various domains with super-large scale pre-training on over 15T tokens of data. Given the wide application of low-bit quantization for LLMs in resource-constrained scenarios, we explore LLaMA3’s capabilities when quantized to low bit-width. This exploration can potentially provide new insights and challenges for the low-bit quantization of LLaMA3 and other future LLMs, especially in addressing performance degradation issues that suffer in LLM compression. Specifically, we comprehensively evaluate the 10 existing post-training quantization and LoRA fine-tuning (LoRA-FT) methods of LLaMA3 on 1-8 bits and various datasets to reveal the low-bit quantization performance of LLaMA3. To uncover the capabilities of low-bit quantized MLLM, we assessed the performance of the LLaMA3-based LLaVA-Next-8B model under 2-4 ultra-low bits with post-training quantization methods. Our experimental results indicate that LLaMA3 still suffers from non-negligible degradation in linguistic and visual contexts, particularly under ultra-low bit widths. This highlights the significant performance gap at low bit-width that needs to be addressed in future developments. We expect that this empirical study will prove valuable in advancing future models, driving LLMs and MLLMs to achieve higher accuracy at lower bit to enhance practicality.