3D object detection in LiDAR point clouds has been extensively used in autonomous driving, intelligent robotics, and augmented reality. Although the one-stage 3D detector has satisfactory training and inference speed, there are still some performance problems due to insufficient utilization of bird’s eye view (BEV) information. In this paper, a new backbone network is proposed to complete the cross-layer fusion of multi-scale BEV feature maps, which makes full use of various information for detection. Specifically, our proposed backbone network can be divided into a coarse branch and a fine branch. In the coarse branch, we use the pyramidal feature hierarchy (PFH) to generate multi-scale BEV feature maps, which retain the advantages of different levels and serves as the input of the fine branch. In the fine branch, our proposed pyramid splitting and aggregation (PSA) module deeply integrates different levels of multi-scale feature maps, thereby improving the expressive ability of the final features. Extensive experiments on the challenging KITTI-3D benchmark show that our method has better performance in both 3D and BEV object detection compared with some previous state-of-the-art methods. Experimental results with average precision (AP) prove the effectiveness of our network.

In this study, detection and three-dimensional (3D) imaging of lightning plasma channels are presented using radar interferometry. Experiments were carried out in Leshan, China with a 48.2 MHz VHF radar configured with an interferometric antenna array. The typical characteristics of lightning echoes are studied in the form of amplitude, phase, and doppler spectra derived from the raw in-phase/quadrature (I/Q) data. In addition, the 3D structure of lightning channels is reconstructed using the interferometry technique. The localization results of lightning are verified with the locating results of lightning detection networks operating at VLF ranges, which indicate the feasibility of using VHF radar for lightning mapping. The interpretation of the observational results is complicated by the dendric structure of lightning channel and the overlap between passive electromagnetic radiations and return echoes. Nevertheless, some parts of the characteristics of lightning are still evident. The observational result of return echoes shows good consistency with the overdense assumption of lightning channels. The transition from the overdense channel to the underdense channel in the form of amplitude and phase is clearly observed. This technique is very promising to reveal the typical characteristics of lightning return echoes and structure of lightning propagation processes.

In the field of hyperspectral and multispectral (HS-MS) image fusion, convolutional neural network (CNN) has achieved promising performance. Nevertheless, most networks can not fully extract the spatial-spectral features from 3D HS-MS images via vanilla convolution. 3D-CNN, treating the spatial and spectral dimension at the same weight maybe a solution to this problem. However, this approach ignores the distinctiveness of spatial-spectral features and suffers from high computational complexity. To meet these challenges, a novel architecture called four-stream network (FS-Net) is proposed, which integrates the spatial-spectral feature extraction, and multi-scale and multi-level feature fusion into a united framework. Firstly, the simple but effective 2D spatial feature extractor and spectral feature extractor are designed, which encode the spatial and spectral features of HS-MS images into four groups of feature maps. Next, these four groups of feature maps are fused in pairs, where the output can be seen as the comprehensive representation of input images. Finally, multi-level and multi-scale feature fusion strategies helps to recover the HS image at high-spatial-resolution, and thus simultaneously capture the detail texture and contextual information of the different and complex ground objects. Extensive experiments were conducted on three datasets with various experimental configurations to demonstrate that the proposed FS-Net achieves superior results quantitatively and visually when compared with the state-of-the-art networks.

Communications industry has remarkably changed with the development of fifth-generation cellular networks. Image, as an indispensable component of communication, has attracted wide attention. Thus, finding a suitable approach to assess image quality is important. Therefore, we propose a deep learning model for image quality assessment (IQA) based on explicit-implicit dual stream network. We use frequency domain features of kurtosis based on wavelet transform to represent explicit features and spatial features extracted by convolutional neural network (CNN) to represent implicit features. Thus, we constructed an explicit-implicit (EI) parallel deep learning model, namely, EI-IQA model. The EI-IQA model is based on the VGGNet that extracts the spatial domain features. On this basis, the number of network layers of VGGNet is reduced by adding the parallel wavelet kurtosis value frequency domain features. Thus, the training parameters and the sample requirements decline. We verified, by cross-validation of different databases, that the wavelet kurtosis feature fusion method based on deep learning has a more complete feature extraction effect and a better generalisation ability. Thus, the method can simulate the human visual perception system better, and subjective feelings become closer to the human eye. The source code about the proposed EI-IQA model is available on githubhttps://github.com/jacob6/EI-IQA.

Three-dimensional object detection from point cloud data is becoming more and more significant, especially for autonomous driving applications. However, it is difficult for lidar to obtain the complete structure of an object in a real scene due to its scanning characteristics. Although the existing methods have made great progress, most of them ignore the prior information of object structure, such as symmetry. So, in this paper, we use the symmetry of the object to complete the missing part in the point cloud and then detect it. Specifically, we propose a two-stage detection framework. In the first stage, we adopt an encoder–decoder structure to generate the symmetry points of the foreground points and make the symmetry points and the non-empty voxel centers form an enhanced point cloud. In the second stage, the enhanced point cloud is input into the baseline, which is an anchor-based region proposal network, to generate the detection results. Extensive experiments on the challenging KITTI benchmark show the effectiveness of our method, which has better performance on both 3D and BEV (bird’s eye view) object detection compared with some previous state-of-the-art methods.

Due to their characteristics of low latency, high temporal resolution, and wide dynamic range, event cameras have emerged as advanced visual sensors for robust multi-modal tracking. However, recent approaches for processing multi-modal sensor data often rely on multiple disjointed modules, such as feature extraction, fusion, and adaptive learning, resulting in high model complexity. Moreover, the feature representations of sensor data obtained through these methods are limited by the traditional attention mechanism, which overlooks the role of frequency domain knowledge in enhancing modality-specific features. This work introduces a unified tracking framework named Frequency-Aware Event Frame Tracker (FAEFTrack) to reduce model complexity and improve modality-specific features. Specifically, a voxel-based event representation is adopted to effectively capture spatial-temporal details from event data. Based on the vision transformer network, we design a single-stream sensor data fusion backbone which is able to integrate feature learning with relation modeling and simplify the tracking process. Furthermore, a frequency attention block with multi-frequency attention is proposed to enhance modality-specific features in the frequency domain. Extensive experiments on the VisEvent and COESOT datasets demonstrate that our approach outperforms state-of-the-art methods, achieving improvements of 3.6%/1.5% and 1.6%/4.1% in SR/PR scores, respectively.