In this paper, we rethink our earlier work on self-attention based crack segmentation, and propose an upgraded CrackFormer network (CrackFormer-II) for pavement crack segmentation, instead of only for fine-grained crack-detection tasks. This work embeds novel Transformer encoder modules into a SegNet-like encoder-decoder structure, where the basic module is composed of novel Transformer encoder blocks with effective relative positional embedding and long range interactions to extract efficient contextual information from feature-channels. Further, fusion modules of scaling-attention are proposed to integrate the results of each respective encoder and decoder block to highlight semantic features and suppress non-semantic ones. Moreover, we update the Transformer encoder blocks enhanced by the local feed-forward layer and skip-connections, and optimize the channel configurations to compress the model parameters. Compared with the original CrackFormer, the CrackFormer-II is trained and evaluated on more general crack datasets. It achieves higher accuracy than the original CrackFormer, and the state-of-the-art (SOTA) method with $6.7$ $\times$ fewer FLOPs and $6.2$ $\times$ fewer parameters, and its practical inference speed is comparable to most classical CNN models. The experimental results show that it achieves the F-measures on Optimal Dataset Scale (ODS) of 0.912, 0.908, 0.914 and 0.869, respectively, on the four benchmarks. Codes are available at https://github.com/LouisNUST/CrackFormer-II.

Road detection is an important task in autonomous navigation systems. In this paper, we propose a road detection framework induced by the inverse depth of LiDAR's point cloud. This framework is a fusion of a 3-D LiDAR and a monocular camera, where the 3-D point cloud of LiDAR is projected onto the camera's image frame, to exploit both range and color information. For the same road detection task, we propose an inverse-depth aware fully convolutional neural network based on image information and a line scanning strategy based on an inverse-depth histogram of LiDAR's point cloud. Finally, a conditional random field fusion method integrates the two road detection results. Our method is evaluated on KITTI-Road benchmark. Experiments demonstrate that our method achieves the state-of-the-art performance in road detection among all the referable methods that have ever reported their results on the KITTI-Road benchmark.

Curb detection, a significant area of autonomous driving, plays an important role in road detection and obstacle avoidance, etc. However, curb detection is challenging due to the problems like occlusions, shadows and the small size of the target. In this paper, we propose a curb detection paradigm for road and sidewalk detection for mobile robots using stereo vision in the urban residential region. First, the flat area is estimated based on the disparity and v-disparity maps generated from stereo matching. In the estimated flat area, to distinguish curbs from the road and obstacles, we propose an efficient 16-dimensional descriptor based on the appearance, geometry and disparity characteristics of curbs. The curb points can be extracted by an SVM classifier with the obtained descriptors. Second, the curb points are exploited by a vanishing point constrained Dijkstra road model to find the road region, where two shortest paths, generated from the vanishing point to the last row of the cost map, are searched and regarded as the road borders. Following the curb and road region detection results, the sidewalk area is detected by a region-growing-like method based on the geometry characteristic of the sidewalk area and the layout of road scenes. The proposed method has been tested for road and sidewalk detection tasks on the KITTI dataset. The experimental results of the curb, road and sidewalk detection demonstrate the accuracy and robustness of this method.

In this paper, we propose a vanishing-point constrained Dijkstra road model for road detection in a stereo-vision paradigm. First, the stereo-camera is used to generate the u- and v-disparity maps of road image, from which the horizon can be extracted. With the horizon and ground region constraints, we can robustly locate the vanishing point of road region. Second, a weighted graph is constructed using all pixels of the image, and the detected vanishing point is treated as the source node of the graph. By computing a vanishing-point constrained Dijkstra minimum-cost map, where both disparity and gradient of gray image are used to calculate cost between two neighbor pixels, the problem of detecting road borders in image is transformed into that of finding two shortest paths that originate from the vanishing point to two pixels in the last row of image. The proposed approach has been implemented and tested over 2600 grayscale images of different road scenes in the KITTI data set. The experimental results demonstrate that this training-free approach can detect horizon, vanishing point, and road regions very accurately and robustly. It can achieve promising performance.

In this paper, we propose a robust vanishing-point constrained lane detection method with a stereo-rig. This method can achieve promising detection performance for both straight and curved lanes without assuming any parametric lane model. First, we propose an accurate and efficient road vanishing point detection scheme based on the v-disparity and visual odometry techniques, where the v-disparity map can significantly reduce the searching space for vanishing point, and the visual odometry can benefit the vanishing point detection of both straight and curved roads. Next, we formulate the lane detection problem as a graph-search procedure, where a vanishing-point constrained Dijkstra shortest-path lane model is proposed to obtain a minimum-cost map. The two lane borders can be detected by finding two optimal paths which originate from the vanishing point to two cost-map derived terminal points, respectively. The proposed method has been tested on the KITTI and the Oxford RobotCar data sets and it works accurately and robustly on a variety of road scenes.

In the point-cloud-based place recognition area, the existing hybrid architectures combining both convolutional networks and transformers have shown promising performance. They mainly apply the voxel-wise transformer after the sparse convolution (SPConv). However, they can induce information loss by the sparse voxelization and further result in loss propagation to the transformer, significantly degrading the performance of the network, especially in outdoor scenes with complex geometric structures and multiple small objects. To address this issue, we propose a novel Point-wise Transformer with sparse Convolution (PTC). Specifically, SPConv is applied to the sparsely voxelized point cloud to extract local features, which are then converted to the point-based representation via a feature transformation unit (FTU). As such, our PTC can apply a transformer model based on the point-wise representation rather than on the voxel-wise one. To enhance the ability to capture long-range features and reduce the computational complexity of the transformer, we propose a two-step transformer, each with different grouping strategies. Meanwhile, in both steps, the attention matrix is computed with much fewer points by grouping a single point cloud into different attention domains. The experiments show that the PTC-Net can achieve state-of-the-art (SOTA) performance, with an improvement of 3.6% on average recall@1. Furthermore, to demonstrate the effectiveness of the PTC, we introduce an extremely light-weight version, PTC-Net-L, with only one PTC layer and half initial channel dimensions, also achieving SOTA performance in terms of the average recall rate and running time with only 0.08 M parameters.