12 August 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Through formulating the image restoration as a generation problem, the conditional diffusion model has been applied to low-light image enhancement (LIE) to restore the details in dark regions. However, in the previous diffusion model based LIE methods, the conditions used for guiding generation are degraded images, such as low-light image, signal-to-noise ratio map and color map, which suffer from severe degradation and are simply fed into diffusion model by rigidly concatenating with the noise. To avoid using degraded conditions resulting in sub-optimal performance in recovering details and enhancing brightness, we use the image intrinsic components originating from the Retinex model as guidance, whose multi-scale features are flexibly integrated into the diffusion model, and propose a novel conditional diffusion model for LIE. Specifically, the input low-light image is decomposed into reflectance and illumination by a Retinex decomposition module, where two components contain abundant physical property and lighting conditions of the scene. Then, we extract the latent features from two conditions through a component-dependent feature extraction module, which is designed according to the physical property of components. Finally, instead of previous rigid concatenation manner, a well-designed feature fusion mechanism is equipped to adaptively embed generative conditions into diffusion model. Extensive experimental results demonstrate that our method outperforms the state-of-the-art methods, and is capable of effectively restoring the local details while brightening the dark regions.
18 April 2025 in IEEE Transactions on Circuits and Systems for Video Technology
It remains to be extremely difficult to capture high quality photographs of low-light scenes. Low light causes the low signal-to-noise ratio (SNR) problem which makes the image noisy. Such scenes almost always have the high dynamic range (HDR) problem caused by uneven lighting where a small area surrounding the light source is very bright while the rest of the scene is very dark, making it very difficult to simultaneously obtain high quality signals in both the dark and bright areas. This paper presents a new image restoration method for tackling the problems in low-light scenes. Fundamentally differing from existing approaches, the new method borrows ideas from inverse graphics rendering and re-renders the image with a canonical light source thus correcting the image from first principle. A deep learning based simplified inverse rendering model (SIRM) featuring implicit regularization is first developed for correcting uneven lighting and then an end-to-end convolutional neural network is constructed for reducing noise. Extensive experimental results are presented to demonstrate that the new method outperforms state-of-the-art methods, and is capable of effectively brightening up dark image regions while at the same time preserving details and color consistency. Our code is available at: https://github.com/pj0927/SIRNet.