State-of-the-art human detection methods focus on deep network architectures to achieve higher recognition performance, at the expense of huge computation. However, computational efficiency and real-time performance are also important evaluation indicators. This paper presents a fast real-time human detection and flow estimation method using depth images captured by a top-view TOF camera. The proposed algorithm mainly consists of head detection based on local pooling and searching, classification refinement based on human morphological features, and tracking assignment filter based on dynamic multi-dimensional feature. A depth image dataset record with more than 10k entries and departure events with detailed human location annotations is established. Taking full advantage of the distance information implied in the depth image, we achieve high-accuracy human detection and people counting with accuracy of 97.73% and significantly reduce the running time. Experiments demonstrate that our algorithm can run at 23.10 ms per frame on a CPU platform. In addition, the proposed robust approach is effective in complex situations such as fast walking, occlusion, crowded scenes, etc.

Due to the imaging mechanism of time-of-flight (ToF) sensors, the captured depth images usually suffer from severe noise and degradation. Though many RGB-guided methods have been proposed for depth image enhancement in the past few years, yet the enhancement performance on real-world depth images is still largely unsatisfactory. Two main reasons are the complexity of realistic noise and degradation in depth images, and the difficulty in collecting noise-clean pairs for supervised enhancement learning. This work aims to develop a self-supervised learning method for RGB-guided depth image enhancement, which does not require any noisy-clean pairs but can significantly boost the enhancement performance on real-world noisy depth images. To this end, we exploit the dependency between RGB and depth images to self-supervise the learning of the enhancement model. It is achieved by maximizing the cross-modal dependency between RGB and depth to promote the enhanced depth having dependency with the RGB of the same scene as much as possible. Furthermore, we augment the cross-modal dependency maximization formulation based on the optimal transport theory to achieve further performance improvement. Experimental results on both synthetic and real-world data demonstrate that our method can significantly outperform existing state-of-the-art methods on depth denoising, multi-path interference suppression, and hole filling. Particularly, our method shows remarkable superiority over existing ones on real-world data in handling various realistic complex degradation. Code is available at https://github.com/wjcyt/SRDE .

Phase unwrapping is a fundamental problem in Time-of-Flight (ToF) imaging, especially when high modulation frequency is used to achieve precise measurement accuracy. This paper proposes a novel double-frequency based phase unwrapping method for ToF sensors. The new method incorporates the idea of image segmentation to solve phase unwrapping region-by-region instead of pixel-by-pixel. The depth image is segmented based on the constraint between depth measurements and wrap numbers at two modulation frequencies. To avoid misclassification of the pixels around phase jump edges in the presence of inevitable noise, we employ a modified distance function to classify the edge pixels into the corresponding connected area. Furthermore, a graph model is used to accurately model the phase jumping relation between different connected areas. On this basis, we propose an MRF framework to fuse the amplitude and depth information to unwrap the phase. Experimental results on both synthetic and real-world data demonstrate that, the proposed method is robust to noise and outperforms state-of-the-art methods, while being highly efficient enabling real-time running.

Visual features of images and video frames have become pervasive and maturely developed in extensive research fields such as computer vision and visual search. In more and more cases, the visual feature becomes necessary information which needs to be transmitted and stored at server side in cloud. Among visual features, the local feature descriptors extracted by SIFT can represent both translational and non-translational motion, such as orientation and zooming. On the other hand, only translational motion can be represented by the Motion Vector (MV) in current MV based block video coding standard. Inspired by these properties, a method that utilizes the available feature to optimize inter prediction video coding is proposed in this paper. In this method, the localization, orientation and scale parameters of matching features extracted by SIFT are delivered to inter prediction to provide non-translational motion estimation (ME) and optimized merge mode. Experimental results have shown that the proposed method can efficiently improve the coding performance according to the accurate feature-matching.

The shape of circle is one of fundamental geometric primitives of man-made engineering objects. Thus, extraction of circles from scanned point clouds is a quite important task in 3D geometry data processing. However, existing circle extraction methods either are sensitive to the quality of raw point clouds when classifying circle-boundary points, or require well-designed fitting functions when regressing circle parameters. To relieve the challenges, we propose an end-to-end Point Cloud Circle Algebraic Fitting Network (Circle-Net) based on a synergy of deep circle-boundary point feature learning and weighted algebraic fitting. First, we design a circle-boundary learning module, which considers local and global neighboring contexts of each point, to detect all potential circle-boundary points. Second, we develop a deep feature based circle parameter learning module for weighted algebraic fitting, without designing any weight metric, to avoid the influence of outliers during fitting. Unlike most of the cutting-edge circle extraction wisdoms, the proposed classification-and-fitting modules are originally co-trained with a comprehensive loss to enhance the quality of extracted circles.Comparisons on the established dataset and real-scanned point clouds exhibit clear improvements of Circle-Net over SOTAs in terms of both noise-robustness and extraction accuracy. We will release our code, model, and data for both training and evaluation on GitHub upon publication.