Nowadays, multiagent reinforcement learning (MARL) have shared significant advances in the adaptive traffic signal control (ATSC) problems. For most of the researches, agents are all isomorphic, which disregards the situation in which isomerous intersections cooperative together in a real ATSC scenario, especially in epidemic regions where different intersections have quite different levels of importance. To this end, this paper models the ATSC problem as a networked Markov game (NMG), in which agents take into account information, including traffic conditions of it and its connected neighbors. A cooperative MARL framework named neighborhood cooperative hysteretic DQN (NC-HDQN) is proposed. Specifically, for each NC-HDQN agent in the NMG, first, the framework analyses correlation degrees with their connected neighbors and weighs observations and rewards by these correlations. Second, NC-HDQN agents independently optimize their strategies on the weighted information using hysteretic DQN (HDQN), which is designed to learn optimal joint strategies in cooperative multiagent games. Third, a rule-based NC-HDQN method and a Pearson correlation coefficient based NC-HDQN method, i.e., empirical NC-HDQN (ENC-HDQN) and Pearson NC-HDQN (PNC-HDQN), respectively, are designed. The first method maps the correlation degree between two connected agents according to vehicle numbers on roads between the two agents. In contrast, the second method uses the Pearson correlation coefficient to calculate the correlation degree adaptively. Our methods are empirically evaluated in both a synthetic scenario and two real-world traffic scenarios and give better performances in almost every standard test metric for ATSC.

The adaptive traffic signal control (ATSC) problem can be modeled as a multiagent cooperative game among urban intersections, where intersections cooperate to counter the city's traffic conditions. Recently, reinforcement learning (RL) has achieved marked successes in managing sequential decision making problems, which motivates us to apply RL in the ATSC problem. One of the largest challenges of this problem is that the observation of intersection is typically partially observable, which limits the learning performance of RL algorithms. Considering the large scale of intersections in an urban traffic environment, we use independent RL to solve ATSC problem in this study. We model ATSC problem as a partially observable weak cooperative traffic model (PO-WCTM). Different from a traditional IRL task that averages the returns of all agents in fully cooperative games, the learning goal of each intersection in PO-WCTM is to reduce the cooperative difficulty of learning, which is also consistent with the traffic environment hypothesis. To achieve the optimal cooperative strategy of PO-WCTM, we propose an IRL algorithm called Cooperative Important Lenient Double DQN (CIL-DDQN), which extends Double DQN (DDQN) algorithm using two mechanisms: the forgetful experience mechanism and the lenient weight training mechanism. The former mechanism decreases the importance of experiences stored in the experience reply buffers, while the latter mechanism increases the weight experiences with high estimation and `leniently' trains the DDQN neural network. Experiments in two real traffic scenarios and one simulated traffic scenarios show that, CIL-DDQN outperforms other methods in almost all performance indicators of ATSC.

Parallax tolerance is a fundamental problem in image stitching. To solve this problem, increasing effort has been devoted to the spatially varying and seam cutting. However, there still exist some issues that need to be adequately addressed. First, the implementation of the spatially varying warping requires a restricted premise that the overlapping region can be aligned: if this premise is not met, ghosting caused by misalignment will emerge. In addition, the spatially varying warping may cause distortion due to the issue of inconsistent homographies. Second, conventional seam cutting will lead to objects being cropped and duplicated. Therefore, in this paper, we propose a stable framework for stitching images: the framework consists of a uniform linear structure model that is able to mitigate the distortion of projection and perspective, while preserving the structures of objects in the non-overlapping region; and a stable hybrid actor-critic that estimates stable seam measurements in the overlapping region to diminish the parallax. Comparison experiments show that the proposed method is superior to some conventional methods with respect to mitigating ghosting and preserving structure.

Multichannel physiological datasets are usually nonlinear and separable in the field of emotion recognition. Many researchers have applied linear or partial nonlinear processing in feature reduction and classification, but these applications did not work well. Therefore, this paper proposed a comprehensive nonlinear method to solve this problem. On the one hand, as traditional feature reduction may cause the loss of significant amounts of feature information, Kernel Principal Component Analysis (KPCA) based on radial basis function (RBF) was introduced to map the data into a high-dimensional space, extract the nonlinear information of the features, and then reduce the dimension. This method can provide many features carrying information about the structure in the physiological dataset. On the other hand, considering its advantages of predictive power and feature selection from a large number of features, Gradient Boosting Decision Tree (GBDT) was used as a nonlinear ensemble classifier to improve the recognition accuracy. The comprehensive nonlinear processing method had a great performance on our physiological dataset. Classification accuracy of four emotions in 29 participants achieved 93.42%.

Multi-agent Reinforcement Learning (MARL) has shown considerable promise in enhancing the efficiency of adaptive traffic signal control (ATSC) systems. However, existing MARL approaches primarily focus on optimizing overall traffic flow, often overlooking the issue of fairness in vehicle waiting times. Considering that there is no need to strive for the ultimate fairness, this paper models the ATSC problem as a Constrained Partially Observable Markov Game (CPOMG), where fairness is modeled as a constraint on the maximum waiting time of vehicles on lanes of intersections instead of a reward term that pursues maximization. CPOMG aims to find a cooperative control policy with optimal traffic efficiency within the constrained solution space by multiple agents. On this basis, this paper proposes a new centralized training and decentralized execution cooperative MARL method, i.e., vehicle-level fairness multi-agent proximity policy optimization (VF-MAPPO). VF-MAPPO leverages a centralized trained global Critic Network to estimate the average vehicle traffic efficiency and vehicle maximum waiting time, and an Actor Network shared by all intersections for decentralized execution, which converts the optimization problem with constraints to an unconstrained optimization objective through the Lagrange multiplier method and adopts proximity policy optimization during training. Additionally, VF-MAPPO incorporates spatial-temporal graph attention in the Critic network to efficiently extract state representations in multi-intersection environments. We qualitatively analyzed the monotonic improvement guarantee of VF-MAPPO. Extensive experimental validation across two real-world and one synthetic scenarios substantiates that VF-MAPPO enhances vehicle-level fairness and maintains average traffic efficiency, surpassing state-of-the-art methods.