Reconstruction of high dynamic range image from a single low dynamic range image captured by a conventional RGB camera, which suffers from over- or under-exposure, is an ill-posed problem. In contrast, recent neuromorphic cameras like event camera and spike camera can record high dynamic range scenes in the form of intensity maps, but with much lower spatial resolution and no color information. In this paper, we propose a hybrid imaging system (denoted as NeurImg) that captures and fuses the visual information from a neuromorphic camera and ordinary images from an RGB camera to reconstruct high-quality high dynamic range images and videos. The proposed NeurImg-HDR+ network consists of specially designed modules, which bridges the domain gaps on resolution, dynamic range, and color representation between two types of sensors and images to reconstruct high-resolution, high dynamic range images and videos. We capture a test dataset of hybrid signals on various HDR scenes using the hybrid camera, and analyze the advantages of the proposed fusing strategy by comparing it to state-of-the-art inverse tone mapping methods and merging two low dynamic range images approaches. Quantitative and qualitative experiments on both synthetic data and real-world scenarios demonstrate the effectiveness of the proposed hybrid high dynamic range imaging system. Code and dataset can be found at: https://github.com/hjynwa/NeurImg-HDR

This paper presents a technique for determining an object's shape based on the similarity of radiance changes observed at points on its surface under varying illumination. To examine the similarity, we use an observation vector that represents a sequence of pixel intensities of a point on the surface under different lighting conditions. Assuming convex objects under distant illumination and orthographic projection, we show that the similarity between two observation vectors is closely related to the similarity between the surface normals of the corresponding points. This enables us to estimate the object's surface normals solely from the similarity of radiance changes under unknown distant lighting by using dimensionality reduction. Unlike most previous shape reconstruction methods, our technique neither assumes particular reflection models nor requires reference materials. This makes our method applicable to a wide variety of objects made of different materials.

This paper presents a precise, stable, and invertible reflectance model for photometric stereo. This microfacet-based model is applicable to all types of isotropic surface reflectance, covering cases from diffusion to specular reflections. We introduce a single variable to physically quantify the surface smoothness, and by monotonically sliding this variable between 0 and 1, our model enables a versatile representation that can smoothly transform between an ellipsoid of revolution and the equation for Lambertian reflectance. In the inverse domain, this model offers a compact and physically interpretable formulation, for which we introduce a fast and lightweight solver that allows accurate estimations for both surface smoothness and surface shape. Finally, extensive experiments on the appearances of synthesized and real objects evidence that this model is state-of-the-art in our off-the-shelf solution.

In this paper, we propose a novel city-scale distance sensing algorithm based on atmosphere optics. The suspended particles, especially in bad weather, would attenuate the light at almost all wavelengths. Observing this fact and starting from the light scattering mechanism, we derive a bispectral distance sensing algorithm by leveraging the difference of extinction coefficient between two specifically selected near infrared wavelengths. The extinction coefficient of the atmosphere is related to both wavelength and meteorological conditions, also known as visibility, such as the fog and haze day. To account for different bad weather conditions, we explicitly introduce visibility into our algorithm by incorporating it into the calculation of extinction coefficient, making our algorithm simple yet effective. To capture the data, we build a bispectral imaging system that is able to take a pair of images with a monochrome camera and two narrow band-pass filters. We also present a wavelength selection strategy that allows us to accurately sense distance regardless of material reflectance and texture. Specifically, this strategy determines two distinct near infrared wavelengths by maximising the extinction coefficient difference while minimizing the influence of building’s reflectance variance. The experiments empirically validate our model and its practical performance on the distance sensing for the city-scale buildings.

This paper introduces a novel depth recovery method based on light absorption in water. Water absorbs light at almost all wavelengths whose absorption coefficient is related to the wavelength. Based on the Beer-Lambert model, we introduce a bispectral depth recovery method that leverages the light absorption difference between two near-infrared wavelengths captured with a distant point source and orthographic cameras. Through extensive analysis, we show that accurate depth can be recovered irrespective of the surface texture and reflectance, and introduce algorithms to correct for nonidealities of a practical implementation including tilted light source and camera placement, nonideal bandpass filters and the perspective effect of the camera with a diverging point light source. We construct a coaxial bispectral depth imaging system using low-cost off-the-shelf hardware and demonstrate its use for recovering the shapes of complex and dynamic objects in water. We also present a trispectral variant to further improve robustness to extremely challenging surface reflectance. Experimental results validate the theory and practical implementation of this novel depth recovery paradigm, which we refer to as shape from water.

Recognizing wet surfaces and their degrees of wetness is essential for many computer vision applications. Surface wetness can inform us slippery spots on a road to autonomous vehicles, muddy areas of a trail to humanoid robots, and the freshness of groceries to us. In this paper, we show that color change, particularly in its spectral behavior, carries rich information about surface wetness. We first derive an analytical spectral appearance model of wet surfaces that expresses the characteristic spectral sharpening due to multiple scattering and absorption in the surface. We present a novel method for estimating key parameters of this spectral appearance model, which enables the recovery of the original surface color and the degree of wetness from a single multispectral image. Applied to a multispectral image, the method estimates the spatial map of wetness together with the dry spectral distribution of the surface. To our knowledge, we are the first to model and leverage the spectral characteristics of wet surfaces to revert its appearance. The results demonstrate the accuracy of our model and the effectiveness of our method for surface wetness and color estimation.