Effective multi-intersection collaboration is pivotal for reinforcement-learning-based traffic signal control to alleviate congestion. Existing work mainly chooses neighboring intersections as collaborators. However, quite a lot of congestion, even some wide-range congestion, is caused by non-neighbors failing to collaborate. To address these issues, we propose to separate the collaborator selection as a second policy to be learned, concurrently being updated with the original signal-controlling policy. Specifically, the selection policy in real-time adaptively selects the best teammates according to phase- and intersection-level features. Empirical results on both synthetic and real-world datasets provide robust validation for the superiority of our approach, offering significant improvements over existing state-of-the-art methods. Code is available at https://github.com/bonaldli/CoSLight.

Public cloud GPU clusters are becoming emerging platforms for training distributed deep learning jobs. Under this training paradigm, the job scheduler is a crucial component to improve user experiences, i.e., reducing training fees and job completion time, which can also save power costs for service providers. However, the scheduling problem is known to be NP-hard. Most existing work divides it into two easier sub-tasks, i.e., ordering task and placement task, which are responsible for deciding the scheduling orders of jobs and placement orders of GPU machines, respectively. Due to the superior adaptation ability, learning-based policies can generally perform better than traditional heuristic-based methods. Nevertheless, there are still two main challenges that have not been well-solved. First, most learning-based methods only focus on ordering or placement policy independently, while ignoring their cooperation. Second, the unbalanced machine performances and resource contention impose huge overhead and uncertainty on job duration, but rarely be considered in existing work. To tackle these issues, this paper presents a dual-agent scheduler framework abstracted from the two sub-tasks to jointly learn the ordering and placement policies and make better-informed scheduling decisions. Specifically, we design an ordering agent with a scalable squeeze-and-communicate strategy for better cooperation; for the placement agent, we propose a novel Random Walk Gaussian Process to learn the performance similarities of GPU machines while being aware of the uncertain performance fluctuation. Finally, the dual-agent is jointly optimized with multi-agent reinforcement learning. Extensive experiments conducted on the real-world production cluster trace demonstrate the superiority of our model.

Existing multiagent exploration works focus on how to explore in the fully cooperative task, which is insufficient in the environment with nonstationarity induced by agent interactions. To tackle this issue, we propose When to Explore (WToE), a simple yet effective variational exploration method to learn WToE under nonstationary environments. WToE employs an interaction-oriented adaptive exploration mechanism to adapt to environmental changes. We first propose a novel graphical model that uses a latent random variable to model the step-level environmental change resulting from interaction effects. Leveraging this graphical model, we employ the supervised variational auto-encoder (VAE) framework to derive a short-term inferred policy from historical trajectories to deal with the nonstationarity. Finally, agents engage in exploration when the short-term inferred policy diverges from the current actor policy. The proposed approach theoretically guarantees the convergence of the $Q$ -value function. In our experiments, we validate our exploration mechanism in grid examples, multiagent particle environments and the battle game of MAgent environments. The results demonstrate the superiority of WToE over multiple baselines and existing exploration methods, such as MAEXQ, NoisyNets, EITI, and PR2.

Radio frequency identification (RFID) system has been extensively employed to track missing items by affixing them with RFID tags. Many practical applications require to efficiently identify missing events for a specific subset of system tags (called key tags) due to their elevated importance. Existing methods primarily aim to identify all tags, which makes it challenging to specifically identify key tags because of interference from other non-key tags (called ordinary tags). In light of this, several key tag identification methods follow a two-step scheme that filters ordinary tags first and then identifies key tags. Nevertheless, this wastes too much time on tag filtering, resulting in low time efficiency. This paper presents a novel missing key tag identification protocol with two creative designs to gain high efficiency. First, we develop a novel verification technique that can rapidly determine the presence or absence of key tags amid the scenarios with both key tags and ordinary ones. By combining the ON-OFF Keying modulation, we could verify multiple key tags in a single slot, thereby reducing the total slots required. Second, we design a new selection technique that efficiently selects the unverified key tags for further verification, while filtering out the verified key tags and irrelevant ordinary tags to avoid redundant data transmission. Additionally, we present an enhancement protocol that leverages a preselection technique to avoid collecting useless tag responses, further boosting efficiency. We carry out rigorous theoretical analysis to optimize the performance of the proposed protocols. Both simulations and practical experiments demonstrate that our method is markedly superior to state-of-the-art solutions.