11 September 2023 in IEEE Transactions on Intelligent Transportation Systems
For autonomous driving, one of the major challenges is to predict pedestrian crossing intention in ego-view. Pedestrian intention depends not only on their intrinsic goals but also on the stimulation of surrounding traffic elements. Considering the influence of other traffic elements on pedestrian intention, recent work introduced more traffic element information into the model to successfully improve performance. However, it is still difficult to effectively capture and fully exploit the potential dynamic spatio-temporal interactions among the target pedestrian and its surrounding traffic elements for accurate reasoning. In this work, inspired by neuroscience that human drivers tend to make continuous sensory-motor driving decisions by progressive visual stimulation, we propose a model termed Progressive Interaction Transformer (PIT) for pedestrian crossing intention prediction. Local pedestrian, global environment, and ego-vehicle motion are considered simultaneously in the proposed PIT. In particular, the temporal fusion block and self-attention mechanism are introduced to jointly and progressively model the dynamic spatio-temporal interactions among the three parties, allowing it to capture richer information and make prediction in a similar way to human drivers. Experimental results demonstrate that PIT achieves higher performance compared with other state-of-the-arts and preserves real-time inference.
28 August 2023 in IEEE Transactions on Intelligent Vehicles
Collision risk assessment aims to provide a sub-jective cognitive comprehension of the risk level in driving scenarios, which is critical for the safety of autonomous driving systems. Pedestrian crossing scenarios contain intricate human-vehicle interactions. Hence, it is important to capture the rich relations between traffic entities and to assess the collision risk promptly to ensure safety. Existing studies focus on modeling the spatial relationships between the ego-vehicle and other vehicles in typical traffic scenarios, while ignoring the complex interactions between pedestrians and the ego-vehicle in critical driving scenarios. To address this issue, we propose a novel approach that involves constructing traffic scene graphs with enhanced vehicle-pedestrian interactions, along with introducing an innovative deep model built upon Transformer and GCN for pedestrian collision risk assessment. Specifically, to facilitate spatio-temporal modeling of traffic scene graph sequence, we propose a novel unified framework that integrates Multi-Relation Graph Convolution Network (MR-GCN) and Temporal Trans-former Encoder. In addition, two variants of traffic scene graph datasets termed as Interaction-Enhanced Scene Graph (IESG) and None-Interaction-Enhanced Scene Graph (Non-IESG) are created for the purpose of assessing pedestrian collision risk, utilizing the CAP-DATA and JAAD respectively. Experiments are conducted on our newly created traffic scene graph datasets of pedestrian crossing scenes. The results on the IESG dataset show that our model outperforms the baseline model with higher accuracy (94% vs. 84%), higher AUC (98% vs. 89%), and higher F1-score (93% vs. 84%). IESG and Non-IESG datasets are available at https://github.com/Pedestrian-Crossing-Collision-Risk-Assessment-Datasets .
22 September 2022 in IEEE Journal of Radio Frequency Identification
Recently, beyond object detection and segmentation, high-level understanding of autonomous driving scenarios is attracting increasing attention. And traffic scene graph has become a potential way to model and represent the cognitive semantics of driving scenarios. However, it remains a challenging problem to construct traffic scene graph due to the complexity of driving scenes and the lack of benchmark datasets with various traffic entities and their relationships. In this paper, we propose a novel driving scene understanding paradigm, which can explicitly model traffic entities and the relationships between them from the view of ego-vehicle. Meanwhile, based on parallel vision and our paradigm, we propose an ego-centric Traffic Scene Graph dataset (TSG-451), which contains 451 images, 2,266 entities, and 4,272 relationships in real and artificial traffic scenarios. Furthermore, through qualitative and quantitative analysis, we validate the advancement of our proposed paradigm and dataset. Further investigation shows that our work can more comprehensively represent traffic scenarios and provide more fine-grained detailed semantics benefit to ego-vehicle’s decision-making.
9 November 2023 in IEEE Transactions on Instrumentation and Measurement
Remote photoplethysmography (rPPG) based heart rate (HR) measurement approaches have attracted increasing attention recently. In recent deep learning-based approaches, the inherent redundancy and noise in video frame contents are urgent issues to be addressed. Some studies utilize spatial-temporal maps (STmap) to encode the video clips into efficient spatial-temporal representations. Since the subtle physiological signals are vulnerable to external factors such as head movement and unstable illumination, how to construct a better physiological representation is a significant factor affecting the methods performance. In addition, existing STmap-based methods always neglect temporal modeling between video clips, which is very important for stable video HR estimation. To address the above problems, in this paper, we propose an effective Temporal Shift Attention network with ESTmap as its input (EST-TSANet). We first propose an enhanced STmap (ESTmap), which combines both local and global physiological information to construct a better physiological representation. To establish the temporal relationship between multiple STmaps, we construct a Split Temporal Shift Block and leverage temporal shift to perform efficient temporal modeling. Besides, considering the characteristics of STmap, a novel plug-and-play module named dimension-specific attention module (DSAM) is designed to optimize the information learning specifically for each dimension. Extensive experiments on the public VIPL-HR and UBFC-rPPG datasets show that our proposed EST-TSANet outperforms the state-of-the-art methods.