3D object detection is still an open problem in autonomous driving scenes. When recognizing and localizing key objects from sparse 3D inputs, autonomous vehicles suffer from a larger continuous searching space and higher fore-background imbalance compared to image-based object detection. In this letter, we aim to solve this fore-background imbalance in 3D object detection. Inspired by the recent use of focal loss in image-based object detection, we extend this hard-mining improvement of binary cross entropy to point-cloud-based object detection and conduct experiments to show its performance based on two different 3D detectors: 3D-FCN and VoxelNet. The evaluation results show up to 11.2AP gains through the focal loss in a wide range of hyperparameters for 3D object detection.

Recent machine learning-based multi-object tracking (MOT) frameworks are becoming popular for 3-D point clouds. Most traditional tracking approaches use filters (e.g., Kalman filter or particle filter) to predict object locations in a time sequence, however, they are vulnerable to extreme motion conditions, such as sudden braking and turning. In this letter, we propose PointTrackNet, an end-to-end 3-D object detection and tracking network, to generate foreground masks, 3-D bounding boxes, and point-wise tracking association displacements for each detected object. The network merely takes as input two adjacent point-cloud frames. Experimental results on the KITTI tracking dataset show competitive results over the state-of-thearts, especially in the irregularly and rapidly changing scenarios.

Task-specific data-fusion networks have marked considerable achievements in urban scene parsing. Among these networks, our recently proposed RoadFormer successfully extracts heterogeneous features from RGB images and surface normal maps and fuses these features through attention mechanisms, demonstrating compelling efficacy in RGB-Normal road scene parsing. However, its performance significantly deteriorates when handling other types/sources of data or performing more universal, all-category scene parsing tasks. To overcome these limitations, this study introduces RoadFormer+, an efficient, robust, and adaptable model capable of effectively fusing RGB-X data, where “X” represents additional types/modalities of data such as depth, thermal, surface normal, and polarization. Specifically, we propose a novel hybrid feature decoupling encoder to extract heterogeneous features and decouple them into global and local components. These decoupled features are then fused through a dual-branch multi-scale heterogeneous feature fusion block, which employs parallel Transformer attentions and convolutional neural network modules to merge multi-scale features across different scales and receptive fields. The fused features are subsequently fed into a decoder to generate the final semantic predictions. Notably, our proposed RoadFormer+ ranks first on the KITTI Road benchmark and achieves state-of-the-art performance in mean intersection over union on the Cityscapes, MFNet, FMB, and ZJU datasets. Moreover, it reduces the number of learnable parameters by 65% compared to RoadFormer. Our source code is publicly available at mias.group/RoadFormerPlus.

Achieving safe and effective navigation in crowds is a crucial yet challenging problem. Recent work has mainly encoded the pedestrian-robot state pairs, which cannot fully capture the interactions among humans. Besides, existing work attempts to achieve “hard” collision avoidance, which may leave no feasible path to the robot in human-rich scenarios. We suppose that this can be addressed by introducing the local risk map and thus incorporate the risk map into the deep reinforcement learning architecture. The proposed map structure contains the crowd interaction states and geometric information. Meanwhile, a “soft” risk mapping of pedestrians is proposed to promote the robot to generate more humanlike motion patterns, and the riskaware dynamic window is designed to enhance the robot's obstacle avoidance ability. Experiments show that our method outperforms the baseline in terms of navigation performance and social attributes. Furthermore, we successfully validate the proposed policy through real-world environments.

Most of existing category-level object pose estimation methods devote to learning the object category information from point cloud modality. However, the scale of 3D datasets is limited due to the high cost of 3D data collection and annotation. Consequently, the category features extracted from these limited point cloud samples may not be comprehensive. This motivates us to investigate whether we can draw on knowledge of other modalities to obtain category information. Inspired by this motivation, we propose CLIPose, a novel 6D pose framework that employs the pre-trained vision-language model to develop better learning of object category information, which can fully leverage abundant semantic knowledge in image and text modalities. To make the 3D encoder learn category-specific features more efficiently, we align representations of three modalities in feature space via multi-modal contrastive learning. In addition to exploiting the pre-trained knowledge of the CLIP’s model, we also expect it to be more sensitive with pose parameters. Therefore, we introduce a prompt tuning approach to fine-tune image encoder while we incorporate rotations and translations information in the text descriptions. CLIPose achieves state-of-the-art performance on two mainstream benchmark datasets, REAL275 and CAMERA25, and runs in real-time during inference (40FPS).

This paper is concerned with the problem of membership-function-dependent controller design for a class of discrete-time T-S fuzzy systems. Based on the partition method of premise variable space, the original T-S fuzzy model is equivalently converted into a kind of piecewise-fuzzy system. Then by employing some staircase functions, the continuous membership functions are approximated by a series of discrete values, via which the information of membership functions is brought into the stability analysis to reduce the design conservatism. With piecewise Lyapunov functions, the approaches to the piecewise-fuzzy state feedback and observer-based output feedback controller design are proposed, respectively, in terms of linear matrix inequalities such that the closed-loop system is asymptotically stable with a prescribed $\mathcal{H}_{\infty}$ performance level. It is shown that the membership functions of the fuzzy model and fuzzy controllers are not necessarily the same, which allows more design flexibility. Finally, two illustrative examples are provided to show the effectiveness of the developed methods.