Packet routing is one of the fundamental problems in computer networks in which a router determines the next-hop of each packet in the queue to get it as quickly as possible to its destination. Reinforcement learning (RL) has been introduced to design autonomous packet routing policies with local information of stochastic packet arrival and service. However, the curse of dimensionality of RL prohibits the more comprehensive representation of dynamic network states, thus limiting its potential benefit. In this article, we propose a novel packet routing framework based on multiagent deep RL (DRL) in which each router possess an independent long short term memory (LSTM) recurrent neural network (RNN) for training and decision making in a fully distributed environment. The LSTM RNN extracts routing features from rich information regarding backlogged packets and past actions, and effectively approximates the value function of Q-learning. We further allow each route to communicate periodically with direct neighbors so that a broader view of network state can be incorporated. The experimental results manifest that our multiagent DRL policy can strike the delicate balance between congestion-aware and shortest routes, and significantly reduce the packet delivery time in general network topologies compared with its counterparts.

In device-to-device (D2D) underlaid cellular networks, several D2D pairs can share one channel to improve the system throughput. Most existing works allow D2D pairs to reuse all the channels, which may incur complicated interference management. Therefore, each D2D pair is limited to reuse at most one channel to reduce overhead. We aim to maximize the throughput of D2D pairs while suppressing the interference to cellular links. However, the optimization problem is an intractable mixed integer non-linear programming (MINLP) problem. Meanwhile, as network size increases, acquiring the global channel state information (CSI) is expensive even impossible in practice. Therefore, we propose a novel local CSI based distributed channel-power allocation scheme. The base station (BS) broadcasts a control signal to indicate its received interference from D2D pairs. Upon this signal, each D2D pair executes channel selection and power update independently and iteratively. Specifically, the channel allocation problem is formulated as a many-to-one matching game with externalities. The power control problem is modeled as a Stackelberg game. We prove the existence of two-sided swapstable matching and show that the outcomes of the Stackelberg game are locally optimal under some mild conditions. Simulation results show that our scheme is efficient with low overhead.

The rapid development of cryptocurrencies has led to an increasing severity of money laundering activities. In recent years, leveraging graph neural networks for cryptocurrency fraud detection has yielded promising results. However, many existing methods predominantly focus on node classification, i.e., detecting individual illicit transactions, rather than uncovering behavioral pattern differences among money laundering groups. In this paper, we tackle the challenges presented by the organized, heterogeneous, and noisy nature of Bitcoin money laundering. We propose a novel subgraph-based contrastive learning algorithm for heterogeneous graphs, named Bit-CHetG, to perform money laundering group detection. Specifically, we employ predefined metapaths to construct the homogeneous subgraphs of wallet addresses and transaction records from the address–transaction heterogeneous graph, enhancing our ability to capture heterogeneity. Subsequently, we utilize graph neural networks to separately extract the topological embedding representations of transaction subgraphs and associated address representations of transaction nodes. Lastly, supervised contrastive learning is introduced to reduce the effect of noise, which pulls together the transaction subgraphs with the same class while pushing apart the subgraphs with different classes. By conducting experiments on two real-world datasets with homogeneous and heterogeneous graphs, the Micro F1 Score of our proposed Bit-CHetG is improved by at least 5% compared to others.

In this paper, we consider a cooperative device-todevice (D2D) communication system, where the D2D transmitters (DTs) act as relays to assist the densified cellular network users (CUs) for transmission quality of service (QoS) improvement. The proposed system achieves a win-win situation, i.e. improving the spectrum efficiency of the CUs that cannot meet their rate requirement while providing spectrum access for D2D pairs. Unlike previous works, to reduce the overhead, we design a novel two-timescale resource allocation scheme, in which the pairing between CUs and D2D pairs is decided at a long timescale and transmission time for CU and D2D pair is determined at a short timescale. Specifically, to characterize the long-term payoff of each potential CU-D2D pair, we investigate the optimal cooperation policy to decide the transmission time based on the instantaneous channel state information (CSI). We prove that the optimal policy is a threshold policy which can be achieved via binary search. Since CUs and D2D pairs are self-interested, they are paired only when they agree to cooperate mutually. Therefore, to study the cooperation behaviors of CUs and D2D pairs, we formulate the pairing problem as a matching game, based on the long-term payoff achieved by the optimal cooperation policy of each possible pairing. Furthermore, unlike most previous matching models in D2D networks, we allow transfer between CUs and D2D pairs to improve the performance. To solve the pairing problem, a distributed algorithm is proposed, which converges to an -stable matching. We show that there is a tradeoff between the optimality and the computational complexity of the algorithm. We also analyze the algorithm in terms of the robustness to the unilateral deviation of D2D pairs. Finally, the simulation results verify the efficiency of the proposed matching algorithm.

Federated learning has provided a new approach to coordinating a group of clients to train a machine learning model collaboratively, which can be easily embedded into Unmanned Aerial Vehicle (UAV) swarms. Compared with the terrestrial wireless networks, the UAV swarm faces more precarious communication conditions, rendering synchronous aggregation no longer tenable. Additionally, the data collected from UAVs tend to be heterogeneous due to different deployment regions or requirements. To overcome these restrictions, this article has proposed a novel two-stage Asynchronous Federated Learning scheme for the UAV swarm. Initially, the convergence property of both convex and non-convex models trained by the proposed scheme is analyzed. In the pre-training stage, we modeled the learning process as a cooperative game with demonstrated monotonicity and submodularity. Furthermore, the Shapley Value is imported to quantify data values of UAVs, and the upper bound of its estimation error rate is derived. In the training stage, a new concept named Network Age of Updates (AoU) is proposed to address the fairness issue, quantifying the model's generalization capability with data value consideration, and a sequential UAV selection scheduling is performed through the AoU minimization by Whittle Index method. Finally, the system performance is validated through both theoretical analysis and simulations.

We consider a Cognitive Radio based Internet of Things (CR-IoT) system consisting of multiple Primary IoT Devices (PDs) and Secondary IoT Devices (SDs). We focus on the impact of the IoT devices’ heterogeneous traffic pattern on the energy efficiency and on the age of information (AoI) performance, and aim to maximize the sum of energy efficiency of the SDs under their individual constraints of the average age of information (AoI), via optimally deciding the transmit power and channel allocation. We first derive the closed-form expressions of the energy efficiency and the average AoI, and subsequently explore their convexity and monotonicity to the transmit power. Following these characterizations, an optimal transmit power optimization algorithm (TPOA) is proposed for the SDs, and the channel allocation problem is reformulated as a matching game based on the utilities calculated from TPOA. To improve the system performance, we introduce the virtual price charged in each channel. Taking advantage of the correlation between two adjacent matchings, we propose a semi-distributed, low-complexity adaptive matching algorithm to construct an $\epsilon $ -stable matching, and the effectiveness of the proposed matching algorithm is verified through Monte Carlo simulations.