In this paper we introduce a novel change detection approach called ChangeGAN for coarsely registered} point clouds in complex street-level urban environment. Our generative adversarial network (GAN) architecture compounds Siamese-style feature extraction, U-net-like use of multiscale features, and Spatial Trans-formation Network (STN) blocks for optimal transformation estimation. The input point clouds are represented by range images, which enables the use of 2D convolutional neural networks. The result is a pair of binary masks showing the change regions on each input range image, which can be backprojected to the input point clouds without loss of information. We have evaluated the proposed method on various challenging scenarios and we have shown its superiority against state-of-the-art change detection methods.

In this paper, we propose a novel, vision-transformer-based end-to-end pose estimation method, LidPose, for real-time human skeleton estimation in non-repetitive circular scanning (NRCS) lidar point clouds. Building on the ViTPose architecture, we introduce novel adaptations to address the unique properties of NRCS lidars, namely, the sparsity and unusual rosetta-like scanning pattern. The proposed method addresses a common issue of NRCS lidar-based perception, namely, the sparsity of the measurement, which needs balancing between the spatial and temporal resolution of the recorded data for efficient analysis of various phenomena. LidPose utilizes foreground and background segmentation techniques for the NRCS lidar sensor to select a region of interest (RoI), making LidPose a complete end-to-end approach to moving pedestrian detection and skeleton fitting from raw NRCS lidar measurement sequences captured by a static sensor for surveillance scenarios. To evaluate the method, we have created a novel, real-world, multi-modal dataset, containing camera images and lidar point clouds from a Livox Avia sensor, with annotated 2D and 3D human skeleton ground truth.

In this paper, we propose a point-level foreground-background separation technique for the segmentation of measurement sequences of a Non-repetitive Circular Scanning (NRCS) Lidar sensor, which is used as a 3D surveillance camera mounted in a fixed position. We show that by applying the NRCS Lidar technology, we can overcome various limitations of rotating multi-beam Lidar sensors, such as low vertical measurement resolution, which is disadvantageous in surveillance applications. As the main challenge, we need to efficiently balance between the spatial and the temporal resolution of the recorded range data. For this reason, we automatically generate and maintain a very high-resolution background model of the sensor’s Field of View, while for enabling real-time analysis of dynamic objects we use low integration time to extract the consecutive time frames. As a result, the laser reflections from foreground objects reflect sparse, but geometrically accurate samples of the silhouettes providing valuable input for higher-level shape description or event analysis steps. We demonstrate the efficiency of the new approach in different realistic NRCS Lidar measurements sequences, obtaining a 0.76 overall F1-score on the measured dataset.

In this paper we propose a novel approach for upgrading real time 3D dynamic object detection methods operating on rotating multi-beam (RMB) Lidar measurements using 3D background city maps stored in new generation geographic information systems (GIS) and previously detected dynamic objects propagated by tracking. First, we apply a state-of-the-art object detection method and distinguish the predicted dynamic object candidates and the remaining static regions of the current Lidar measurement. Next we find an optimal transformation between the static part of the RMB Lidar measurements and the background city map using a multimodal point cloud registration algorithm operating in the Hough space. After the accurate alignment, we filter false-positively detected object candidates in the RMB Lidar data based on the map. To find additional objects missed by the object detector on the current measurement, we apply a Kalman-filter based object tracking. Hereby we first predict the current state of the previously detected and tracked objects. Next, we apply a Hungarian matcher based assignment between the tracked and the current objects and update the object list according to the result. For better accuracy, we keep all predictions through a couple of frames. We evaluated our method qualitatively and quantitatively in crowded urban scenes of Budapest, Hungary, and the results showed that with background map based filtering we can achieve a 26,52% improvement detecting vehicles and 9,38% for pedestrians in precision, while via tracking, a 12,84% improvement for vehicles and 14,34% for pedestrians in recall against the state-of-the-art object detection method relying purely on a single Lidar time frame.