In this paper, we present a three-layer (i.e., device, field, and factory layers) deterministic federated learning (FL) framework, named DetFed, which accelerates collaborative learning process for ultra-reliable and low-latency industrial Internet of Things (IoT) via integrating 6G-oriented Time-sensitive Networks (TSN). Utilizing dispersive local data, industrial IoT devices distributively train a deep neural network (DNN) model, and the updated model parameters are aggregated at their associated field servers every round or at a centralized factory server every a few rounds. Aiming at optimizing the learning accuracy of FL without affecting the co-transmission of burst traffic (e.g., safety-critical traffic), an integrated TSN is considered to establish connections among the three layers, where a cyclic queuing and forwarding mechanism is deployed in each switch to support deterministic model parameter transmission with microsecond-level delay and near-zero packet loss requirements. To improve the FL performance, we formulate a multi-objective stochastic optimization problem to simultaneously maximize the scheduling success ratio and learning accuracy while satisfying the deterministic requirements of delay, jitter, and packet loss. Since the objective function is implicit and the available time slots of the considered TSN in each FL round are temporally correlated, the problem is difficult to solve in real time. Therefore, we transform the problem into a Markov decision process formulation and propose a dynamic resource scheduling algorithm, based on deep reinforcement learning, to make optimal resource scheduling decisions while adapting to device heterogeneity and network dynamics. Experimental results based on real-world dataset demonstrate that the proposed DetFed significantly accelerates FL convergence and improves learning accuracy as compared to state-of-the-art benchmarks.

Deterministic transmission guarantee in time-sensitive networks (TSN) relies on queue models (such as CQF, TAS, ATS) and resource scheduling algorithms. Thanks to its ease of use, the CQF queue model has been widely adopted. However, the existing resource scheduling algorithms of CQF model only focus on periodic time-triggered (TT) flows without consideration of bursting flows. Considering that the bursting flows often carry high-priority data in real systems, in this paper we investigate the mixed-flow (i.e., TT and bursting flows) scheduling problem in CQF-based TSN aiming to maximize the number of schedulable flows and system load balance while satisfying the deterministic demands of delay, jitter, and reliability for both TT and bursting flows. Unfortunately, it is challenging to schedule the mixed flows with the original CQF model because of the huge difference between TT and bursting flows. To resolve this problem, we firstly design an enhanced Multi-CQF model to satisfy the basic demands of bursting flows sent at any time without affecting the deterministic transmission of TT flows. Given the complexity of mixed-flow scheduling and the proposed queue model, it is difficult for traditional algorithms to fully utilize network resources. Thus, we further propose a time-correlated DRL resource scheduling (TimeDRS) algorithm to optimize the resource allocation. TimeDRS can be extended to other time-related resource scheduling scenarios, such as TDMA-based scheduling. Experimental results demonstrate that our proposed approaches can greatly reduce frame loss and end-to-end latency for bursting flows, and well balance runtime and schedulability compared with state-of-the-art benchmarks.

With the great evolution of human society from the information era to the smart automation era, intelligent production and maintenance have become the core orientation of Industry 4.0. Data-driven intelligent predictive maintenance (IPdM) offers potential solutions to satisfy the urgent demand of predicting impending failures and mitigating unplanned downtime of industrial machinery, and is envisioned as a critical enabler for future industrial Internet of Things. In this article, we provide a comprehensive survey of deep learning (DL)-based IPdM for the researchers and practitioners who focused on the promotion of fault diagnosis/prognosis. First of all, four mainstream algorithms of DL are reviewed around the specific IPdM applications of recent five years. Some prominent discussions and insights on the DL-based IPdM are given here in terms of data characteristics and model performance. Particularly, as an important contribution, a new industrial dataset of dual-bearings rotating machinery is published for opening research to address one of the main challenges for the development of DL-based IPdM, i.e., the shortage of high quality industrial datasets. Compared with current datasets, our dataset is multisensory, which is closer to reality and more suitable for building advanced DL-based IPdM models. We also give a DL-based case study to demonstrate how to build a predictive model for maintenance decisions based on this dataset. Finally, potential research trends and challenges are also discussed for enhancing the intelligence of IPdM in the industrial scenarios.

Incentive plays an important role in knowledge discovery, as it impels users to provide high-quality knowledge. To promise incentive schemes with transparency, blockchain technology has been widely used in incentive schemes. Currently, privacy, reliability, streamlined processing, and quality awareness are major challenges in designing blockchain-based incentive schemes. In this paper, we design a blockchain-based eFficient and pRivacy-preserving qUality-aware IncenTive scheme called FRUIT. With well-designed smart contracts, FRUIT achieves privacy, reliability, streamlined processing, and quality awareness during the whole procedure. Specifically, we design a novel lightweight encryption method by combining matrix decomposition with proxy re-encryption and a privacy-preserving task allocation based on the polynomial fitting function and hash function. Then, we leverage our proposed lightweight encryption and task allocation to build an efficient and privacy-preserving knowledge discovery protocol in order to securely calculate the data quality and truthful knowledge. To promise user reliability in the incentive scheme, we utilize the Dirichlet distribution to realize the automatic reputation prediction based on the data quality by deploying the reputation management on the blockchain. Moreover, we also deploy the payment management on the blockchain, endowing the incentive scheme to reward participants based on the data quality automatically. Through a detailed security analysis, we demonstrate that data privacy and task privacy are well preserved during the whole process. Theoretical analysis and extensive experiments on real-world datasets demonstrate that FRUIT has acceptable efficiency and affordable performance in terms of computation cost, communication overhead, and gas consumption.

Large Language Models (LLMs), such as LLaMA and GPT-4, have transformed the paradigm of natural language comprehension and generation. Despite their impressive performance, these models still face certain challenges, including the need for extensive data, high computational resources, and privacy concerns related to their data sources. Recently, Federated Learning (FL) has surfaced as a cooperative AI methodology that enables AI training across distributed computation entities while maintaining decentralized data. Integrating FL with LLMs presents an encouraging solution for privacy-preserving and collaborative LLM learning across multiple end-users, thus addressing the aforementioned challenges. In this paper, we provide an exhaustive review of federated Large Language Models, starting from an overview of the latest progress in FL and LLMs, and proceeding to a discourse on their motivation and challenges for integration. We then conduct a thorough review of the existing federated LLM research from the perspective of the entire lifespan, from pre-training to fine-tuning and practical applications. Moreover, we address the threats and issues arising from this integration, shedding light on the delicate balance between privacy and robustness, and introduce existing approaches and potential strategies for enhancing federated LLM privacy and resilience. Finally, we conclude this survey by outlining promising avenues for future research in this emerging field.

Network slicing technology and the Open Radio Access Network (O-RAN) architecture provide solutions for resource isolation and dynamic resource optimization in resource allocation models for heterogeneous services. However, the existing resource allocation models employ uniformly sized radio Resource Blocks (RBs) and fixed-length time slots, which incur low resource utilization and a high packet loss rate, especially when the packet arrival rate is high. In this paper, we propose a multi-cell, multi-dimensional resource, and multi-timescale O-RAN slicing framework, named M3O-RANS, which meets the Quality of Service (QoS) requirements of heterogeneous services and reduces resource costs for network service providers. Specifically, the M3O-RANS framework partitions multiple timescales into a large timescale and multiple small timescales, using radio RBs of different sizes for inter-slice operations at the large timescale and reusing radio RBs across the multi-cell network for intra-slice operations at the small timescale, thereby improving resource utilization. To optimally allocate radio and computing resources according to time-varying network conditions, we formulate an optimization problem with the objective of minimizing resource costs for network service providers while satisfying delay and reliability requirements. Since inter-slice and intra-slice resource constraints are mutually coupled, it is difficult to solve the problem via traditional Deep Reinforcement Learning (DRL) methods. Therefore, we decompose the original problem into two subproblems and propose a Parallel Hierarchical DRL-based Resource Allocation (PHDRA) algorithm. The proposed algorithm can make decisions regarding the size and number of radio RBs and the amount of computing resources at the upper layer while making radio RB selection and computing resource allocation decisions at the lower layer. Simulation results demonstrate that the PHDRA algorithm can improve the QoS of heterogeneous services and reduce resource costs for network service providers compared with state-of-the-art benchmarks.