9 December 2024 in IEEE Internet of Things Journal
The end-cloud synergy with neural networks offers significant promise for enhancing AI-IoT systems. Numerous studies have been conducted to effectively address the dynamic and resource-constrained nature of IoT; however, challenges persist in adaptability to constraints, horizontal integration between technologies, and system-level optimization. In this paper, we propose an end-cloud collaborative system (EECS) with an on-demand offloading mechanism for AI-IoT. We enhance three critical areas previously narrowly explored: model pruning to fit into the end device’s constraints, dynamic data transmission between end and cloud, and an effective offloading policy. We pay particular attention to the constraints imposed by multiply-and-add (MAC) operations and bandwidth (BW) at the end, with a focus on image classification tasks. Our contributions are threefold. First, we introduce a two-stage trainable pruning method that can automatically adjust the end model to end constraints and optimize from a system-wide perspective. Second, we propose an adaptive mechanism that accommodates fluctuating bandwidth conditions based on our trainable pruning, integrating static pruning with dynamic inference. Third, we develop a smart offloading policy that enhances decision-making, thereby elevating system-level efficiency. Finally, our EECS shows substantial improvements, achieving 2.6×~4× reductions in MAC costs and 3.8×~10.4× reductions in BW costs compared to existing approaches, based on evaluations using the CIFAR-100 and Tiny-ImageNet-200 datasets.