Industrial wireless networks (IWNs) have attracted significant attention for providing time-critical delivery services, which can benefit from device-to-device (D2D) communication for low transmission delay. In this paper, a distributed scheduling issue is investigated for D2D-enabled IWNs, where D2D links have age-of-information (AoI) constraints for information freshness. This is formulated as a constrained optimization problem to optimize D2D packet delivery over limited spectrum resources, which is intractable since D2D users have no prior knowledge of the operating environment. To tackle it, an AoI-aware scheduling algorithm (AoIS) is proposed based on primal-dual optimization and actor-critic reinforcement learning. In specific, multiple local actors for D2D devices learn AoI-aware scheduling policies to make on-site decisions with their stochastic AoI constraints addressed in dual domain. An edge based critic estimates the performance of all actors' policies from a global view, which effectively addresses the non-stationary environment caused by concurrent learning of multiple local actors.
3 April 2024 in IEEE Transactions on Network Science and Engineering
Digital twin (DT) can bridge the physical status with the virtual space in real-time for the Industrial Internet of Things (IIoT), where the integration of federated learning (FL) with DT can enable many edge intelligence services for timely intelligent production in the era of Industry 4.0. However, the issues of heterogeneity of devices and the resource-constrained IIoT make it challenging to achieve efficient FL via DT technology. To handle this problem, we propose a DT-enabled IIoT (DTENI) framework in wireless networks, in which DTs capture the characteristics of industrial devices to enable real-time processing and intelligent decision-making. Specifically, we first analyze the necessity of adaptive wireless parameters (i.e., CPU frequency, bandwidth, and transmission power) on FL training performance and provide theoretical analysis in the DTENI IIoT. Based on the above analysis, we then formulate the minimization problem of FL model loss under a given resource budget, which is a stochastic optimization problem with strongly coupled wireless parameters variables. Benefiting from the model-free learning superiority of deep reinforcement learning (DRL) in dealing with stochastic optimization problems, we develop DTENI-assisted DRL to adaptively adjust the wireless parameters for solving this optimization problem. Lastly, simulation results demonstrate that our proposed scheme can mostly save communication costs up to 74.23%, 69.51%, and 60.94% compared to the three benchmarks.
9 August 2023 in IEEE Transactions on Network Science and Engineering
The integration of fifth-generation (5 G) and time-sensitive networking (TSN) is a key technology to promote the wireless upgrade of manufacturing industry, and some preliminary enhancements on 5 G with TSN-integration have been introduced to empower industrial internet ubiquitous connectivity and real-time forwarding. However, at present, the 5 G system is only served as a logical bridge with 5G-specific procedures hidden from TSN networks, and thus the deep integration of 5 G and TSN is still an open issue. This article focuses on the co-design problem of flow scheduling in 5 G and TSN, where 5 G grant-free configured-grants (CGs) and TSN cyclic queuing and forwarding mechanisms are adopted. Firstly, we innovatively propose to achieve seamless 5G+TSN transmission by intelligently selecting CGs in 5 G, where the starting time offsets of different CGs will influence the flow injection time in the downstream TSN network to mitigate the queue overflow issue. Then, an age-of-information (age) aware decentralized and deterministic scheduling (DecAge) scheme based on actor-critic reinforcement learning is proposed. DecAge guides time-sensitive flow hosts to autonomously make packet sampling-transmission decisions so that timeliness-based age constraints can be respected via collision-free 5G+TSN transmission. DecAge contains multiple local actors learning hierarchically-structured scheduling policies for flow hosts and a central graph attention network-based critic to estimate these policies. Through simulation, DecAge shows the robustness to environmental non-stationarity by elaborately coordinating distributed learning and yields improved performance over non-hierarchical agents on sample-efficient learning.
12 July 2022 in IEEE Transactions on Network Science and Engineering
6G is envisioned to achieve ubiquitous Artificial Intelligence (AI) in heterogeneous and massive-scale networks, where FEderated Edge Learning (FEEL) is an effective way to assist AI technology to be implemented at the network edge. FEEL enables training a machine learning model through communication among multiple edge devices based on data from a large number of users. However, the heterogeneity among distributed edge devices and the limitation of resources may degrade the training efficiency of FEEL over 6G-enabled mobile edge computing (MEC) networks. Taking this challenge into account, a novel federated learning scheme is proposed in this paper to accelerate the training process. In particular, we formulate a training efficiency maximization problem, where a novel analytical relation among the training loss, resource consumption and heterogeneity is inferred by convergence analysis. Then, we propose a searching algorithm named IFBA to obtain the optimal inexactness of local models and frequency band allocation for edge devices in order to mitigate the straggler effect caused by the heterogeneity and resource limitation of devices. As a result, the training efficiency can be improved by adapting the inexactness of local models and frequency band allocation for edge devices on-demand. Simulation results validate that AceFL has advantages in training efficiency improvement and heterogeneity adaption, e.g., accelerating the training by $5\%$ - $15\%$ under different heterogeneity.
Due to the ever-increasing time-sensitive industrial applications, critical-machine type communication (C-MTC) is a promising technique for timely delivery services in industrial wireless networks (IWNs), where vicinal devices can benefit from device-to-device (D2D) communication for low power consumption and latency. For real-time applications, age of information (AoI) is an essential metric that represents the freshness of data from the perspective of destinations. Thus, an AoI orchestration agent for link scheduling in D2D-enabled IWNs is needed. Most existing works on AoI deal with this scheduling in a centralized manner, which cannot afford timely packet delivery requirements for numerous D2D devices. Different from the existing works, a learning-based autonomous AoI and power orchestration agent, namely L-AoI, is proposed for D2D-enabled IWNs in this paper, where D2D devices adaptively compete for wireless resources in a distributed manner. As a result, the global channel state information as well as the actions of other D2D devices are unknown. Hence, D2D devices deal with this uncertainty under the guidance of L-AoI so that AoI constraints can be respected. By leveraging from the belief-based Bayesian Reinforcement Learning, L-AoI learns the scheduling action profile with strategies of other D2D devices considered so that the spectrum sharing coalitions can be intelligently formed. Both theoretical analysis and simulation are provided to validate the performance of L-AoI in terms of AoI stability and violation ratio.
2 December 2024 in IEEE Transactions on Mobile Computing
Compared with synchronous federated learning (FL), asynchronous FL (AFL) has attracted more and more attention in edge computing (EC) fields because of its strong adaptability to heterogeneous application scenarios. However, the non-independent and identically distributed (Non-IID) data across devices and the staleness-aware estimation of unreliable wireless connections and limited edge resources make it much more difficult to achieve better AFL-related applications. To handle this problem, we propose an Adaptive Staleness-aware Momentum Accelerated AFL (ASMAFL) algorithm to reduce the resources consumption of heterogeneous wireless communication EC (WCEC) scenarios, as well as decrease the negative impact of Non-IID data for model training. Specifically, we first introduce the staleness-aware parameter and a unified momentum gradient descent (GD) framework to reformulate AFL. Then, we establish global convergence properties of AFL, derive an upper bound on AFL convergence rate, and find that the bound is related to the staleness-aware parameter and Non-IIDness. Next, we formulate the bound into a minimization problem of resource consumption under given model accuracy, and the corresponding staleness-aware parameter of devices will be recomputed after each asynchronous aggregation to eliminate the differences of local models’ contribution to global model aggregation. Finally, extensive experiments are carried out to validate the superiority of ASMAFL in model accuracy, convergence rate, resources consumption, Non-IID issue, etc.