By the laws of quantum physics, pixel intensity does not have a true value, but should be a random variable. Contrary to the conventional assumptions, the distribution of intensity may not be an additive Gaussian. We propose to directly model the intensity difference and show its validity by an experimental comparison to the conventional additive model. As a model of the intensity difference, we present a Skellam distribution derived from the Poisson photon noise model. This modeling induces a linear relationship between intensity and Skellam parameters, while conventional variance computation methods do not yield any significant relationship between these parameters under natural illumination. The intensity-Skellam line is invariant to scene, illumination, and even most of camera parameters. We also propose practical methods to obtain the line using a color pattern and an arbitrary image under natural illumination. Because the Skellam parameters that can be obtained from this linearity determine a noise distribution for each intensity value, we can statistically determine whether any intensity difference is caused by an underlying signal difference or by noise. We demonstrate the effectiveness of this new noise model by applying it to practical applications of background subtraction and edge detection.

Accurate stereo matching for depth extraction requires a large memory space which limits its use in resource-limited systems. The problem is aggravated by the recent trend of applications requiring significantly high pixel resolution and disparity levels. To alleviate the high memory requirement we propose to represent the aggregation costs as a Gaussian mixture model (GMM) function. Only a set of GMM parameters are stored and used instead of all costs for each pixel. We also propose parameter update-based aggregation along multiple paths. To preserve the accuracy of disparity map we employ depth confidence measure and propose an update rule for slanted surface of object. Experimental results over the KITTI dataset show that the proposed method reduces the memory requirement to less than 5% of that of semi-global matching while the accuracy is preserved in the level of state-of-the-art semi-global and local methods.

RGB-Depth (RGB-D) cameras are widely used in computer vision and robotics applications such as 3D modeling and human–computer interaction. To capture 3D information of an object from different viewpoints simultaneously, we need to use multiple RGB-D cameras. To minimize costs, the cameras are often sparsely distributed without shared scene features. Due to the advantage of being visible from different viewpoints, spherical objects have been used for extrinsic calibration of widely-separated cameras. Assuming that the projected shape of the spherical object is circular, this paper presents a multi-cue-based method for detecting circular regions in a single color image. Experimental comparisons with existing methods show that our proposed method accurately detects spherical objects with cluttered backgrounds under different illumination conditions. The circle detection method is then applied to extrinsic calibration of multiple RGB-D cameras, for which we propose to use robust cost functions to reduce errors due to misdetected sphere centers. Through experiments, we show that the proposed method provides accurate calibration results in the presence of outliers and performs better than a least-squares-based method.

The goal of multi-modal learning is to use complimentary information on the relevant task provided by the multiple modalities to achieve reliable and robust performance. Recently, deep learning has led significant improvement in multi-modal learning by allowing for the information fusion in the intermediate feature levels. This paper addresses a problem of designing robust deep multi-modal learning architecture in the presence of imperfect modalities. We introduce deep fusion architecture for object detection which processes each modality using the separate convolutional neural network (CNN) and constructs the joint feature map by combining the intermediate features from the CNNs. In order to facilitate the robustness to the degraded modalities, we employ the gated information fusion (GIF) network which weights the contribution from each modality according to the input feature maps to be fused. The weights are determined through the convolutional layers followed by a sigmoid function and trained along with the information fusion network in an end-to-end fashion. Our experiments show that the proposed GIF network offers the additional architectural flexibility to achieve robust performance in handling some degraded modalities, and show a significant performance improvement based on Single Shot Detector (SSD) for KITTI dataset using the proposed fusion network and data augmentation schemes.

Object proposals for detecting moving or static video objects need to address issues such as speed, memory complexity and temporal consistency. We propose an efficient Video Object Proposal (VOP) generation method and show its efficacy in learning a better video object detector. A deep-learning based video object detector learned using the proposed VOP achieves state-of-the-art detection performance on the Youtube-Objects dataset. We further propose a clustering of VOPs which can efficiently be used for detecting objects in video in a streaming fashion. As opposed to applying per-frame convolutional neural network (CNN) based object detection, our proposed method called Objects in Video Enabler thRough LAbel Propagation (OVERLAP) needs to classify only a small fraction of all candidate proposals in every video frame through streaming clustering of object proposals and class-label propagation. Source code will be made available soon.