2 June 2023 in IEEE Journal of Solid-State Circuits
A dual-tap coded-exposure-pixel (CEP) image sensor is presented and validated in two computational imaging applications. The NMOS-only data-memory pixel (DMP) reduces the transistor count yielding a $7\text{-}\mu \text{m}$ pitch. One frame period can include up to 900 subexposures when operating at 30 frames/s, corresponding to 39 000 coded subexposures/s. The $320\ttimes320$ -pixel sensor features two readout modes using column-parallel analog-to-digital converters (ADCs). ADC1 is a conventional high-accuracy $\Delta\Sigma$ -modulated ADC that digitizes pixel voltage at the end of every frame period, and ADC2 is a fast energy-efficient comparator that compares the pixel voltage with a constant reference voltage during each subexposure. The outputs of the 12-bit frame-rate ADC1 and the 1-bit subexposure-rate ADC2 are adaptively combined to boost the native dynamic range of the uncoded pixel by over 57 dB, demonstrating over 101-dB dynamic range in intensity imaging. In the second demonstrated application, combined with machine-learned projected illumination patterns, the CEP camera enables single-shot structured-light 3-D imaging at the native resolution and the nominal 30 frames/s video rate.
18 June 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Optical blur is an inherent property of any lens system and is challenging to model in modern cameras because of their complex optical elements. To tackle this challenge, we introduce a high-dimensional neural representation of blur—the lens blur field—and a practical method for acquiring it. The lens blur field is a multilayer perceptron (MLP) designed to (1) accurately capture variations of the lens 2D point spread function over image plane location, focus setting and, optionally, depth and (2) represent these variations parametrically as a single, sensor-specific function. The representation models the combined effects of defocus, diffraction, aberration, and accounts for sensor features such as pixel color filters and pixel-specific micro-lenses. To learn the real-world blur field of a given device, we formulate a generalized non-blind deconvolution problem that directly optimizes the MLP weights using a small set of focal stacks as the only input. We also provide a first-of-its-kind dataset of 5D blur fields—for smartphone cameras, camera bodies equipped with a variety of lenses, etc. Lastly, we show that acquired 5D blur fields are expressive and accurate enough to reveal, for the first time, differences in optical behavior of smartphone devices of the same make and model.