23 November 2018 in IEEE Transactions on Computational Imaging
Signal reconstruction is a challenging aspect of computational imaging as it often involves solving ill-posed inverse problems. Recently, deep feed-forward neural networks have led to state-of-the-art results in solving various inverse imaging problems. However, being task specific, these networks have to be learned for each inverse problem. On the other hand, a more flexible approach would be to learn a deep generative model once and then use it as a signal prior for solving various inverse problems. In this paper, we show that among the various state-of-the-art deep generative models, autoregressive models are especially suitable for our purpose for the following reasons. First, they explicitly model the pixel level dependencies and hence are capable of reconstructing low-level details, such as texture patterns and edges better. Second, they provide an explicit expression for the image prior, which can then be used for MAP-based inference along with the forward model. Third, they can model long range dependencies in images which make them ideal for handling global multiplexing as encountered in various compressive imaging systems. We demonstrate the efficacy of our proposed approach in solving three computational imaging problems: Single Pixel Camera, LiSens, and FlatCam. For both real and simulated cases, we obtain better reconstructions than the state-of-the-art methods in terms of perceptual and quantitative metrics.
19 December 2025 in Science Advances
Natural selection has produced diverse vision systems, from simple patches of photoreceptors to complex camera eyes, representing just one set of evolutionary outcomes. Computational evolution offers a way to systematically test hypotheses, isolate individual factors, and ask the “why” questions behind vision. We recreate vision evolution by coevolving eyes and behaviors in embodied agents and use this to illuminate principles shaping vision across different levels of the Marr’s hierarchy. This leads to three key findings: First, we provide computational evidence that task-specific selection drives bifurcation in eye evolution. Second, we reveal how optical innovations naturally emerge to resolve fundamental trade-offs between light collection and spatial precision. Third, we uncover scaling laws between visual acuity and neural processing that provide insights into long-standing hypothesis behind eye and brain size. Our work introduces a paradigm that uses embodied artificial intelligence (AI) as hypothesis-testing machines that can help accelerate discoveries in vision science.
8 April 2025 in ACM Transactions on Graphics
Photoelasticity enables full-field stress analysis in transparent objects through stress-induced birefringence. Existing techniques are limited to two-dimensional (2D) slices and require destructively slicing the object. Recovering the internal three-dimensional (3D) stress distribution of the entire object is challenging, as it involves solving a tensor tomography problem and handling phase wrapping ambiguities. We introduce NeST, an analysis-by-synthesis approach for reconstructing 3D stress tensor fields as neural implicit representations from polarization measurements. Our key insight is to jointly handle phase unwrapping and tensor tomography using a differentiable forward model based on Jones calculus. Our non-linear model faithfully matches real captures, unlike prior linear approximations. We develop an experimental multi-axis polariscope setup to capture 3D photoelasticity and experimentally demonstrate that NeST reconstructs the internal stress distribution for objects with varying shape and force conditions. Additionally, we showcase novel applications in stress analysis, such as visualizing photoelastic fringes by virtually slicing the object and viewing photoelastic fringes from unseen viewpoints. NeST paves the way for scalable non-destructive 3D photoelastic analysis.