Cameras currently allow access to two image states: (i) a minimally processed linear raw-RGB image state (i.e., raw sensor data); or (ii) a highly-processed nonlinear image state (e.g., sRGB). There are many computer vision tasks that work best with a linear image state, such as image deblurring and image dehazing. Unfortunately, the vast majority of images are saved in the nonlinear image state. Because of this, a number of methods have been proposed to “unprocess” nonlinear images back to a raw-RGB state. However, existing unprocessing methods have a drawback because raw-RGB images are sensor-specific. As a result, it is necessary to know which camera produced the sRGB output and use a method or network tailored for that sensor to properly unprocess it. This paper addresses this limitation by exploiting another camera image state that is not available as an output, but it is available inside the camera pipeline. In particular, cameras apply a colorimetric conversion step to convert the raw-RGB image to a device-independent space based on the CIE XYZ color space before they apply the nonlinear photo-finishing. Leveraging this canonical image state, we propose a deep learning framework, CIE XYZ Net, that can unprocess a nonlinear image back to the canonical CIE XYZ image. This image can then be processed by any low-level computer vision operator and re-rendered back to the nonlinear image. We demonstrate the usefulness of the CIE XYZ Net on several low-level vision tasks and show significant gains that can be obtained by this processing framework. Code and dataset are publicly available at https://github.com/mahmoudnafifi/CIE_XYZ_NET .

Image enhancement methods can be formulated as global transformations, local transformations, pixel-wise processing, or a mixture of these operations. Global transformations are limited in enhancing local image regions. Existing local and pixel-wise methods mitigate this issue, but give rise to the additional challenge of limited interpretability. Bridging the gap between global and local methods, we propose a local tone mapping network (LTMNet) that learns a grid of tone curves to locally enhance an image. Tone curves are commonly used by photo-editing software and offer an intuitive representation to photographers, facilitating subsequent customization of the image. Tone curves are also widely used in image signal processors (ISPs), making our method easy to deploy on cameras. Because existing datasets contain image enhancement and photofinishing beyond global and local tone mapping, we also propose a new dataset representative of local tone mapping—the LTM dataset. We evaluate our method on this new dataset as well as MIT-Adobe and HDR+ datasets. We show that the proposed LTMNet outperforms existing methods in local tone mapping while achieving competitive performance modeling additional photofinishing. Furthermore, we show that our method can be assistive in user-interactive photo-editing tools. Our code, model, and data will be released publicly.

The authenticity of digital images posted online and shared on social media is often questioned due to the ability of photo-editing software to alter image content and generative AI methods that can produce visually compelling deepfakes . Only images directly produced by cameras are deemed unaltered and beyond suspicion, as they have not undergone any modifications. However, there is a recent trend among camera manufacturers to integrate AI-based modules into the dedicated onboard hardware, specifically the image signal processor (ISP), responsible for processing the captured sensor image into the final saved image for users. Many of these AI modules utilize perceptual or generative losses during training, which can “hallucinate” image content. While this hallucinated content often manifests as small details and textures, there are instances where these regions unintentionally impact the interpretation of the entire image. This paper aims to bring attention to this issue and advocate for in-camera strategies to validate the authenticity of camera-captured images at a pixel level. We propose the creation of an "authenticity" mask that could be stored as additional metadata with each image. This information can be extracted and overlaid on the image to easily identify the hallucinated regions. Considering the widespread implications of image authenticity (e.g., in courtroom evidence, news broadcasts, and other media forms), we anticipate that authentication metadata will become a standard practice for any ISP utilizing AI.