25 September 2023 in IEEE Micro
Metareasoning shows promise in efficiently using the computational resources of tiny edge devices while performing highly computationally intensive Reinforcement Learning (RL) algorithms. This work proposes MetaE2RL: a hardware-aware framework that incorporates low-power pre-processing solutions and metareasoning to enable the deployment of multi-goal RL on tiny autonomous devices. For this aim, a meta-level is proposed to allocate resources efficiently in real-time by switching between models with different complexities. Moreover, Squeezed Edge YOLO is proposed for energy-efficient object detection in the pre-processing phase. For the experimental results, the proposed Squeezed Edge YOLO was deployed onboard a tiny drone named Crazyflie with GAP8 processor that includes 8 parallel RISC-V cluster cores. We compared latency and power consumption of Squeezed Edge YOLO and a lighter CNN-based model while deploying them separately onboard on GAP8. Experimental results show Squeezed Edge YOLO is 8x smaller than previous work and consumes 541 mW on GAP8 with inference latency of 130 ms.
5 December 2024 in IEEE Embedded Systems Letters
Efficiently deploying deep neural networks on resource-limited embedded systems is crucial to meet real-time and power consumption requirements. Utilizing metareasoning as a higher-level controller along with tiny machine learning (TinyML) can enhance energy efficiency and reduce latency on such systems by overseeing available resources. This study introduces MetaTinyML, a comprehensive metareasoning framework for self-guided navigation on TinyML platforms. The framework adapts its decision-making process by factoring in environmental changes to select the most suitable algorithms for the current scenario. Implementation of MetaTinyML on an NVIDIA Jetson Nano 4-GB system integrated with a Jetbot ground vehicle demonstrated up to 50% power consumption enhancement. View a video demonstration of the MetaTinyML framework at: Video.
13 May 2026 in ACM Transactions on Embedded Computing Systems
Large Language Models (LLMs) have become foundational tools in natural language processing, achieving state-of-the-art performance across a variety of tasks. However, their immense size and computational requirements make them impractical for deployment in resource-constrained environments, such as edge devices and embedded systems. In this work, we introduce Magnitude and Gradient-Informed Pruning (MaGrIP) , a novel framework for task-agnostic pruning and compression of LLMs. MaGrIP employs a dual-threshold strategy combining magnitude- and gradient-based saliency measures to efficiently prune redundant neurons while retaining task performance. Our results demonstrate the effectiveness of MaGrIP in compressing state-of-the-art models. The compression reduced the total computational complexity of the FFN layers from \(\mathcal {O}(d \cdot h)\) to \(\mathcal {O}((d - q) \cdot h)\) . In terms of model size, our pruning approach significantly reduces both model parameters and storage requirements while maintaining competitive perplexity scores evaluated on WikiText-2. For the Gemma 7B model, our method reduces the total size from 28 GB to 5 GB, while for Gemma 2B, MaGrIP achieves a size reduction from 8 GB to 1.5 GB. MaGrIP furthermore exhibits robust performance across multiple benchmarks, such as BOOLQ, ARC-E, and CSQA. Specifically, the pruned Gemma 7B model at 50% pruning achieved 59.26% accuracy on ARC-E compared to 81.06% for the baseline, and 64.74% accuracy on BoolQ compared to 59.98% for the baseline. Similarly, the pruned Llama 3 8B at 50% pruning achieved 46.76% accuracy on ARC-E compared to 77.57% for the baseline, reflecting the tradeoff between compression and accuracy. LLMs compressed using MaGrIP, when deployed on the Nvidia Jetson Orin Nano, achieved a 2.16× improvement in throughput and a 2.3× improvement in performance compared to baseline LLMs.