With the development of deep learning technologies and edge computing, the combination of them can make artificial intelligence ubiquitous. Due to the constrained computation resources of the edge device, the research in the field of on-device deep learning not only focuses on the model accuracy but also on the model efficiency, for example, inference latency. There are many attempts to optimize the existing deep learning models for the purpose of deploying them on the edge devices that meet specific application requirements while maintaining high accuracy. Such work not only requires professional knowledge but also needs a lot of experiments, which limits the customization of neural networks for varied devices and application scenarios. In order to reduce the human intervention in designing and optimizing the neural network structure, multi-objective neural architecture search methods that can automatically search for neural networks featured with high accuracy and can satisfy certain hardware performance requirements are proposed. However, the current methods commonly set accuracy and inference latency as the performance indicator during the search process, and sample numerous network structures to obtain the required neural network. Lacking regulation to the search direction with the search objectives will generate a large number of useless networks during the search process, which influences the search efficiency to a great extent. Therefore, in this paper, an efficient resource-aware search method is proposed. Firstly, the network inference consumption profiling model for any specific device is established, and it can help us directly obtain the resource consumption of each operation in the network structure and the inference latency of the entire sampled network. Next, on the basis of the Bayesian search, a resource-aware Pareto Bayesian search is proposed. Accuracy and inference latency are set as the constraints to regulate the search direction. With a clearer search direction, the overall search efficiency will be improved. Furthermore, cell-based structure and lightweight operation are applied to optimize the search space for further enhancing the search efficiency. The experimental results demonstrate that with our method, the inference latency of the searched network structure reduced 94.71% without scarifying the accuracy. At the same time, the search efficiency increased by 18.18%.

Federated Learning (FL) is widely used in mobile edge applications. However, the heterogeneity issues of mobile edge devices pose significant challenges to the generalization of the global model in FL. In this paper, we propose LegoFL to simultaneously solve multiple heterogeneity issues in response to mobile edge computing characteristics. LegoFL identifies two types of heterogeneous behaviors in FL, namely heterogeneous parameter training and communication behaviors, to address multiple heterogeneity issues. These two types of heterogeneous behaviors result in feature and feature representation range mismatches between local communication parameters. To reduce these mismatches and improve the generalization of the global model, LegoFL dynamically distinguishes the parameter feature representation of different nodes using the global model's common feature as guidance. Then, under the connection states and system communication constraints, LegoFL dynamically selects contribution parameters on each device that can guarantee the generalization and performance of the global model for communication. Finally, to avoid the overfitting problem of the global model, heterogeneous local models are aggregated at the central server with matched feature representations. Extensive experiments on various datasets show that LegoFL achieves competitive performance. The accuracy and communication efficiency are improved by up to 12.86 $\%$ and 4.09× compared to state-of-the-art approaches.

Multiple deep neural networks (DNNs) are increasingly used in real-world intelligent applications, such as intelligent robotics and autonomous vehicles to collectively complete complicated tasks running on edge devices. Because each layer of the subtasks prefers a distinct dataflow due to the heterogeneity in shape and scale of the network layers, a variable dataflow approach on the DNN accelerators is urgently required. On DNN accelerators that enable multiple dataflows, however, we detect a dimension mismatch between parallel processing under the dataflow approach and linear data memory arrangement. When multiple DNN tasks share partial features or weights, the issue is further exacerbated. During processing, this mismatch causes a sluggish data supply from both off-chip and on-chip memory. Consequently, the overall throughput, performance, and energy efficiency suffer since DNN models are sensitive to data density. In this work, we reveal the mechanism behind this data dimension mismatch and present a series of metrics that quantify the influence on system performance. On this foundation, we offer a framework that tracks the data tensor dimension conversion and employs a flexible data arrangement over multi-DNN computation to adapt to dataflow variability. An accelerator architecture named data arrangement multi-DNN accelerator (DARMA) that features a data arrangement and distribution circuit and hierarchical memory for data dimension conversion is also presented. Since the mismatch is mitigated, the suggested accelerator outperforms current accelerators in terms of bandwidth and processing unit utilization. Through tests on VR/AR, MLperf, and other multitask applications, the evaluation results show that the proposed architecture provides both energy-efficiency and throughput improvements.

Exploiting sparsity to eliminate redundant computation and memory overhead is essential for enhancing the efficiency of on-edge deep neural network (DNN) learning. The sparsity features leveraged by existing works are derived from empirical or statistical analyses and are focused on specific phases of training. As a result, the available sparsity features are restricted to the model sparsity of convolution operations. This limitation prevents the full exploitation of the advantages of sparsity, thereby constraining the potential for efficient edge learning. This paper observes that data sparsity correlation exists in the zero-value positions between different operands during training, which can be efficiently leveraged. We propose a unitwise analytic approach that investigates data sparsity correlation by exploring the zero propagation rules among different operands. Based on this, we construct a sparsity propagation model that formally describes these features and proposes a sparsity exploitation method to ensure sufficient sparsity exploitation. To our knowledge, this paper is the first work to model sparsity during training. We also present an energy-efficient sparse training accelerator (ESPACE) that enables multiple sparsity features to avoid unnecessary computations and memory footprints, alleviating the contradiction between training high-demand and resource-constrained edge devices. Extensive experiments validate that ESPACE achieves speedups of up to 3.2\(\times \), 1.69\(\times \), and 1.73\(\times \), and energy efficiencies of up to 2.8\(\times \), 1.75\(\times \), and 1.9\(\times \), compared to the dense accelerator and two other sparse training accelerators, respectively. The proposed sparsity-exploitation method can be used alongside prior dynamic pruning methods during training to achieve a 1.89\(\times \) speedup and a 1.59\(\times \) improvement in energy efficiency, even though these approaches leverage model sparsity for aggressive pruning. These results demonstrate that modeling and exploiting sparsity correlation during training can significantly outperform conventional sparsity strategies that rely solely on model sparsity, providing a more general and effective pathway for improving both computational efficiency and energy efficiency in on-edge learning systems.