This paper proposes to utilize supervised deep convolutional neural networks to take full advantage of the long-term spatial-temporal information in order to improve the video saliency detection performance. The conventional methods, which use the temporally neighbored frames solely, could easily encounter transient failure cases when the spatial-temporal saliency clues are less-trustworthy for a long period. To tackle the aforementioned limitation, we plan to identify those beyond-scope frames with trustworthy long-term saliency clues first and then align it with the current problem domain for an improved video saliency detection.
30 March 2018 in IEEE Transactions on Image Processing
A major bottleneck of pedestrian detection lies on the sharp performance deterioration in the presence of small-size pedestrians that are relatively far from the camera. Motivated by the observation that pedestrians of disparate spatial scales exhibit distinct visual appearances, we propose in this paper an active pedestrian detector that explicitly operates over multiple-layer neuronal representations of the input still image. More specifically, convolutional neural nets, such as ResNet and faster R-CNNs, are exploited to provide a rich and discriminative hierarchy of feature representations, as well as initial pedestrian proposals. Here each pedestrian observation of distinct size could be best characterized in terms of the ResNet feature representation at a certain layer of the hierarchy. Meanwhile, initial pedestrian proposals are attained by the faster R-CNNs techniques, i.e., region proposal network and follow-up region of interesting pooling layer employed right after the specific ResNet convolutional layer of interest, to produce joint predictions on the bounding-box proposals' locations and categories (i.e., pedestrian or not). This is engaged as an input to our active detector, where for each initial pedestrian proposal, a sequence of coordinate transformation actions is carried out to determine its proper x-y 2D location and the layer of feature representation, or eventually terminated as being background. Empirically our approach is demonstrated to produce overall lower detection errors on widely used benchmarks, and it works particularly well with far-scale pedestrians. For example, compared with 60.51% log-average miss rate of the state-of-the-art MS-CNN for far-scale pedestrians (those below 80 pixels in bounding-box height) of the Caltech benchmark, the miss rate of our approach is 41.85%, with a notable reduction of 18.66%.
20 November 2024 in IEEE Signal Processing Letters
Point cloud completion 3D object detectors effectively tackle the challenge of incomplete shapes in sparse point clouds by generating pseudo points to improve detection performance. However, the absence of guidance provided by the heatmap information and the geometric shape information renders the precise recovery of object shapes an arduous task. To this end, we propose a Geometry-guided Point Generation for 3D Object Detection, named GgPG. Specifically, we first design a 3D heatmap auxiliary supervision subnetwork to enhance the quality of object proposals by capturing the actual size and position of the object within the 3D heatmap representation. Moreover, we introduce a density-aware point generation module that employs Kernel Density Estimation (KDE) to embed the point density into the grid point's feature representation, thereby enabling the completion of more precise object shapes. Our GgPG achieves progressive performance in both Waymo and KITTI benchmarks, notably GgPG outperforms PGRCNN by +1.02 $\%$ , +1.18 $\%$ , and +0.56 $\%$ on the vehicle, pedestrian, and cyclist under LEVEL $\_$ 2 mAPH classes on Waymo Open Dataset, respectively.
22 September 2025 in IEEE Transactions on Circuits and Systems for Video Technology
3D object detection has achieved significant progress in outdoor LiDAR point clouds, however, the inherent irregularity and varying sparsity distribution of point occupancy present a key challenge. Existing transformer-based 3D detectors often treat all tokens within the attention window as equally important, regardless of varying sparsity, which not only fails to address the disparities between the varying beam densities but also results in increased memory and computational costs. In this work, we propose an adaptive structure-aware cascaded transformer (ASCFormer) that dynamically captures density-insensitive multiscale structure features to model long-range dependencies via cascaded learning. Our ASCFormer detector includes an adaptive structure-aware token learning module that embeds voxel-level foreground probability and grid-level local density into the grid tokens to enhance structural perception capability. Moreover, we integrate these factors to compute significance scores, which are then utilized in inverse transform sampling to select a subset of multiscale tokens with varying receptive field sizes. To improve the training convergence of the window-based transformer in 3D voxel space, we employ cascaded learning via cross-stage attention to enhance the feature representation capability and refine the localization precision of 3D bounding boxes. This design of structure-aware reweighting effectively enhances the cascade paradigm, making to more adaptable to the varying sparsity distribution of point clouds. Extensive experiments on the KITTI and Waymo Open datasets demonstrate that the proposed ASCFormer detector achieves exceptional performance compared with state-of-the-art 3D object detection methods. The source code is publicly available at https://github.com/Xinglong-Li1/ASCFormer.
29 July 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Recently, Transformer-based few-shot classification methods are widely exploited. However, they only leverage feature information at a single scale, resulting in weak feature representations, which cannot fully capture the rich information contained in a limited number of images regarding diverse objects with different scales, even those belonging to the same category. To mitigate this issue, we propose a multi-scale feature sets matching scheme in vision Transformer for few-shot classification, and name it FSViT, which can sufficiently extract discriminative features from the few number of labeled support examples. Concretely, we establish a patch-based multi-scale feature representation based on the feature extractors of FSViT, where we introduce an attention-aware grid pooling operation to merge adjacent patches with various scales to obtain multi-scale feature sets. Moreover, we devise a multi-scale patch matching metric to aggregate the measurement of similarity over the multi-scale feature sets for few-shot classification. Extensive experiments demonstrate the effectiveness of the proposed FSViT in both 1-shot and 5-shot scenarios on standard single-domain and cross-domain few-shot classification, especially improving the state-of-the-art recognition accuracy by 1.27% and 1.33% on average on the Mini-ImageNet and CFAIR-FS datasets, respectively. The code of FSViT is available at https://github.com/codeshop715/FSViT.