The growing number of Internet of Things (IoT) devices brings enormous time-sensitive applications, which require real-time transmission to effectuate communication services. The ultra-reliable and low latency communication (URLLC) scenario in the fifth generation (5G) has played a critical role in supporting services with delay-sensitive properties. Time-sensitive networking (TSN) has been widely considered as a promising paradigm for enabling the deterministic transmission guarantees for 5G. However, TSN is a hybrid traffic system with time-sensitive traffic and best-effort traffic, which require effective routing and scheduling to provide a deterministic and bounded delay. While joint optimization of time-sensitive and non-time-sensitive traffic greatly increases the solution space and brings a significant challenge to obtain solutions. Therefore, this paper proposes a graph convolutional network-based deep reinforcement learning (GCN-based DRL) solution for the joint optimization problem in practical communication scenarios. The GCN is integrated into DRL to obtain the network’s spatial dependence and elevate the generalization performance of the proposed method. Specifically, the GCN adopts the first order Chebyshev polynomial to approximate the graph convolution kernel, which reduces the complexity of the algorithm and improves the feasibility for the joint optimization task. Furthermore, priority experience replay is employed to accelerate the convergence speed of the model training process. Numerical simulations demonstrate that the proposed GCN-based DRL algorithm has good convergence, and outperforms the benchmark methods in terms of the average end-to-end delay.

Anomaly detection plays an essential role in network security and traffic classification. Many studies have focused on anomaly detection to improve network security, including machine learning and deep learning methods. These methods often require numerous samples and must obtain the results by classifying the entire data set, thereby limiting their inflexibility. Although transfer and multitask learning have achieved some results in the model’s transferability, these methods must manually label or reprocess the test set. These problems limit the application of previous methods in network security management. To solve these problems, we propose a transferable and adaptable network intrusion detection system (TA-NIDS) based on deep reinforcement learning. The interaction process between the agent and the environment varies every time. A small-scale data set can be used to produce many interactive processes. Therefore, robustness is guaranteed when there are few samples. Then, a reasonable reward function allows the agent to learn how to first choose outliers without classifying the entire data set. This makes the TA-NIDS more adaptable to the scene when we prioritize apparent outliers. More importantly, the original features are transformed into the state of the environment, so no requirement exists for the feature dimension. Furthermore, the general rather than the specific state of one data set makes the model transferable to other data sets. The experimental results for IDS2017, IDS2018, NSL-KDD, UNSW-NB15 and CIC-IoT2023 show that the proposed framework maintains good accuracy when prioritizing outliers and transferability are prioritized simultaneously.

In developing the sixth-generation (6G) system, integrated sensing and communication technology is becoming increasingly essential, especially for applications like autonomous driving. This paper develops an architecture for integrated sensing, communication, and computation (ISCC) in the vehicular network, where vehicles perform environment sensing, sensing data computation, and transmission. To support low-latency cooperation between vehicles and extend vehicles’ sensing range, over-air-computation federated learning is employed. The optimization problem of joint beamforming design and power resource allocation in the ISCC scenario is formulated to maximize the achievable data rate while ensuring sensing and computing performance. However, solving this joint optimization problem is a great challenge due to the high coupling resource and time-varying channel environment. Therefore, a hybrid reinforcement learning scheme is proposed in this work. First, the semidefinite relaxation and Gaussian randomization techniques are leveraged to obtain the approximate solution of the aggregation beamformer. Then, the deep deterministic policy gradient algorithm is proposed to tackle the transmit beamforming design and resource allocation problem in continuous action space. Extensive simulation results validated the admirable performance of the proposed scheme in convergence and achievable sum rate compared with the benchmark schemes. In addition, the impact of variables on the optimization performance is demonstrated via numerical results.

Human activity recognition (HAR) is a popular and challenging research topic driven by various applications. Deep learning methods have been used to improve HAR models’ accuracy and efficiency. However, this kind of method has a lot of manually adjusted parameters, which cost researchers a lot of time to train and test. So, it is challenging to design a suitable model. In this paper, we propose HARNAS, an efficient approach for automatic architecture search for HAR. Inspired by the popular multi-objective evolutionary algorithm, which has a strong capability in solving problems with multiple conflicting objectives, we set weighted f1-score, flops, and the number of parameters as objects. Furthermore, we use a surrogate model to select models with a high score from the large candidate set. Moreover, the chosen models are added to the training set of the surrogate model, which makes the surrogate model update along the search process. Our method avoids manually designing the network structure, and the experiment results demonstrate that it can reduce 40% training costs on both time and computing resources on the OPPORTUNITY dataset and 75% on the UniMiB-SHAR dataset. Additionally, we also prove the portability of the trained surrogate model and HAR model by transferring them from the training dataset to a new dataset.

This paper investigates an intelligent reflecting surface (IRS)-aided integrated sensing and communication (ISAC) framework to cope with the problem of spectrum scarcity and poor wireless environment. The main goal of the proposed framework in this work is to optimize the overall performance of the system, including sensing, communication, and computational offloading. We aim to achieve the trade-off between system performance and overhead by optimizing spectrum and computing resource allocation. On the one hand, the joint design of transmit beamforming and phase shift matrices can enhance the radar sensing quality and increase the communication data rate. On the other hand, task offloading and computation resource allocation optimize energy consumption and delay. Due to the coupled and high dimension optimization variables, the optimization problem is non-convex and NP-hard. Meanwhile, given the dynamic wireless channel condition, we formulate the optimization design as a Markov decision process. To tackle this complex optimization problem, we proposed two innovative deep reinforcement learning (DRL)-based schemes. Specifically, a deep deterministic policy gradient (DDPG) method is proposed to address the continuous high-dimensional action space, and the prioritized experience replay is adopted to speed up the convergence process. Then, a twin delayed DDPG algorithm is designed based on this DRL framework. Numerical results confirm the effectiveness of proposed schemes compared with the benchmark methods.

In the face of increasing network traffic, network security issues have gained significant attention. Existing network intrusion detection models often improve the ability to distinguish network behaviors by optimizing the model structure, while ignoring the expressiveness of network traffic at the data level. Visual analysis of network behavior through representation learning can provide a new perspective for network intrusion detection. Unfortunately, representation learning based on machine learning and deep learning often suffer from scalability and interpretability limitations. In this article, we establish an interpretable multi-layer mapping model to enhance the expressiveness of network traffic data. Moreover, the unsupervised method is used to extract the internal distribution characteristics of the data before the model to enhance the data. What's more, we analyze the feasibility of the proposed flow spectrum theory on the UNSW-NB15 dataset. Experimental results demonstrate that the flow spectrum exhibits significant advantages in characterizing network behavior compared to the original network traffic features, underscoring its practical application value. Finally, we conduct an application analysis using multiple datasets(CICIDS2017 and CICIDS2018), revealing the model's strong universality and adaptability across different datasets.