Multi-agent Deep Reinforcement Learning (MADRL) based traffic signal control lbecomes a popular research topic in recent years. To alleviate the scalability issue of completely centralized reinforcement learning (RL) techniques and the non-stationarity issue of completely decentralized RL techniques on large-scale traffic networks, some literature utilizes a regional control approach where the whole network is firstly partitioned into multiple disjoint regions, followed by applying the centralized RL approach to each region. However, the existing partitioning rules either have no constraints on the topology of regions or require the same topology for all regions. Meanwhile, no existing regional control approach explores the performance of optimal joint action in an exponentially growing regional action space when intersections are controlled by 4-phase traffic signals (EW, EWL, NS, NSL). In this paper, we propose a novel RL training framework named RegionLight to tackle the above limitations. Specifically, the topology of regions is firstly constrained to a star network which comprises one center and an arbitrary number of leaves. Next, the network partitioning problem is modeled as an optimization problem to minimize the number of regions. Then, an Adaptive Branching Dueling Q-Network (ABDQ) model is proposed to decompose the regional control task into several joint signal control sub-tasks corresponding to particular intersections. Subsequently, these sub-tasks maximize the regional benefits cooperatively. Finally, the global control strategy for the whole network is obtained by concatenating the optimal joint actions of all regions. Experimental results demonstrate the superiority of our proposed framework over all baselines under both real and synthetic scenarios in all evaluation metrics.

Unmanned aerial vehicle (UAV) is becoming a crucial aerial platform to provide emergency or enhanced communication and sensing services benefited from its unique features, including agile mobility and high probability of Line-of-Sight coverage. In this article, we investigate the novel UAV trajectory design problem where the UAV communicates with multiple users and simultaneously senses the positions of multiple targets, via the integrated communication and sensing design. To evaluate the overall system utility, we first derive the communication throughput and localization error Cramér-Rao bound (CRB) model and highlight the coupled challenge and tradeoffs on UAV’s trajectory. We thus model the three typical integrated sensing and communication scenarios: 1) communication centric; 2) sensing centric; and 3) tradeoff scenarios. We further introduce path discretization to support in-flight communication and hovering sensing, which make it possible to jointly optimize the UAV trajectory, communication throughput and target localization estimation error CRB with limited complexity. Because this optimization problem involves integer programming caused by multiuser association, we propose a novel trajectory initialization framework based on traveling salesman problem to determine the service order for sensing targets and the initial UAV trajectory. To address the nonconvexity of this optimization problem, we propose an efficient iterative algorithm using the successive convexity approximation technique to obtain the approximate optimal solution, which is dynamically reconfigured and optimized with updated information in flight. Extensive numerical results demonstrate that the proposed algorithm achieves superior performance in the tradeoff between average achievable rate and CRB in all three scenarios.

Harmonic Drives (HDs) are key components of industrial robots, and damages or failures of the HDs can lead to robot paralysis or even cause operational accidents. To prevent this from happening, it is of great importance to design proper algorithms for fault diagnosis of the HDs, especially in real HD manufacturing lines. In this paper, a fault diagnosis method was proposed to convert multiple vibration signals into symmetrized dot pattern (SDP) images, and to use a ConvNeXt model with Transformer training procedures to classify fault features of the HDs under various fault states and working conditions. The method was experimentally validated on three datasets with a diagnostic accuracy of up to 98.5% and 95.3% under constant and variable conditions, respectively, both higher over those using either state-of-the-art convolutional neural networks or Transformers. This was the first time where the SDP and the ConvNeXt were unitedly used to give superior visibility, interpretability, accuracy and efficiency for HD fault diagnosis, indicating good feasibility of using the ConvNeXt as a backbone for this task.

The rapid advancement of deep learning and time-frequency analysis techniques have brought about a revolution in fault diagnosis for mechanical systems, offering flexible and efficient solutions. Nevertheless, data imbalance issues continue to pose significant obstacles in fault diagnosis modeling. In this research, we propose the use of a domain adaptation generative adversarial network (DAGAN) that capitalizes on transfer learning to extract valuable information from the majority-class data, while concurrently generating and augmenting minority-class data to expand the training dataset. DAGAN incorporates advanced techniques, including deep domain confusion and parameter forgetting, to enhance knowledge extraction and transfer during the transfer learning process, resulting in more realistic and comprehensive generation outcomes when dealing with small sample training. Furthermore, we have developed an imbalanced fault diagnosis method based on DAGAN, which further incorporates Continuous Wavelet Transform and Deep Residual Networks. Finally, the effectiveness and superiority of proposed method are validated on bearing and gearbox datasets. The experimental results demonstrate the outstanding performance of our method in effectively addressing imbalanced fault diagnosis. Note to Practitioners —In this paper, we present a practical approach to enhance the diagnosis of imbalanced faults. Our approach begins by utilizing Time-frequency images, which offer a comprehensive representation of the temporal and spectral characteristics of mechanical systems’ behavior. These images serve as input features and form the foundation for our fault diagnosis modeling. To address the limitations imposed by imbalanced data, we introduce DAGAN and incorporate advanced techniques such as deep domain confusion and parameter forgetting. These techniques facilitate the extraction and transfer of knowledge during the transfer learning process of DAGAN. Consequently, our approach generates more realistic and comprehensive outcomes, even when confronted with limited training samples. To validate the effectiveness and superiority of our proposed approach, we conducted extensive experiments on bearing and gearbox datasets. The results of these experiments demonstrate that our practical approach, which combines wavelet transform-based time-frequency analysis and the innovative DAGAN framework, offers a reliable and comprehensive solution for overcoming challenges associated with imbalanced fault data.