Densely-sampled light field (LF) image is drawing increased attention for its wide applications in 3D reconstruction, digital refocusing, depth estimation, and virtual/augmented reality, et al. In order to reconstruct a densely-sampled LF with high angular resolution, many computational methods have been proposed. However, most existing methods consider LF angular reconstruction based on neighboring views, local epipolar plane images (EPIs) or EPI volume information, which overlook the rich LF angular information and fail to restore more texture details, especially for occlusion regions. In order to mitigate this problem, we introduce a multi-angular epipolar geometry (MA-EG) structure for LF angular reconstruction. The MA-EG structure contains multi-angular directional LF geometry information and can provide multi-angular geometry characteristics in LF reconstruction process, which benefits in recovering more texture details. Based on the MA-EG structure, we further put forward a multi-angular LF angular reconstruction network (MALFRNet) to learn a mapping from sparse LF to a densely-sampled LF. The proposed MALFRNet adopts a multi-stream framework, which can fully explore rich LF angular information and implicitly learn LF angular consistency and spatial geometry information from LF MA-EG. Comprehensive experiments on real-world and synthetic LF scenes demonstrate that the proposed MALFRNet can recover more texture details and achieve a better reconstruction quality. Moreover, ablation studies and LF depth estimation applications also illustrate the advantages of using more angular information in LF angular reconstruction through the proposed MALFRNet.

Light field (LF) imaging enables new possibilities for digital imaging, such as digital refocusing, changing of focus plane, changing of viewpoint, scene-depth estimation, and 3D scene reconstruction, by capturing both spatial and angular information of light rays. However, one main problem in dealing with LF data is its sheer volume. In this context, efficient compression methods are needed for such a particular type of content. In this paper, we propose a content-based LF image-compression method with Gaussian process regression to improve the compression efficiency and accelerate the prediction procedure. First, the LF image is fed to the intra-frame codec of HEVC. In the prediction procedure, the prediction units (PUs) are classified as non-homogenous texture units, homogenous texture units, and visually flat units, based on the content property of the LF image. For each category, we design a corresponding Gaussian process regression (GPR)-based prediction method. Moreover, we propose a classification mechanism to exactly decide to which category the current PU belongs, so as to adjust the trade-off between the computational burden and the LF image coding efficiency. Experimental results demonstrate that the proposed LF image compression method is superior to several other state-of-the-art compression methods in terms of different quality metrics. Furthermore, the proposed method can also achieve a good visual quality of views rendered from decoded LF contents.

The quality evaluation is an indispensable link in light field (LF) image processing. Most of existing LF objective evaluation indexes do not make effective use of the angular characteristic of LF, so the evaluation results are unsatisfactory. In this letter, the quality evaluation of LF image is constructed based on human visual system (HVS) and LF angular-spatial characteristics. Based on the fact that HVS has different sensitivity to different parallaxes, we assume that LF image quality perceived by human eyes has the optimal parallax range. A dual-fan filter is used to constrain the parallax range. Then, the overall quality of the LF is represented by combining the spatial and angular quality, which performed from the central sub-aperture image and the focus stack, respectively. In addition, because the difference of Gaussian (DoG) operator can simulate the process of extracting texture structure by human eyes. The structural similarity of DoG texture feature is utilized in the spatial quality evaluation. Extensive comparison experiments show that the proposed method is more consistent with the characteristics of LF.

Light field (LF) imaging, which captures both intensity and directional information of light rays, extends the capabilities of traditional imaging techniques. In this paper, we introduce a task in the field of LF imaging, sparse-to-dense inbetweening, which focuses on generating dense novel views from sparse multi-view LFs. By synthesizing intermediate views from sparse inputs, this task enhances LF view synthesis through filling in interperspective gaps within an expanded field of view and increasing data robustness by leveraging complementary information between light rays from different perspectives, which are limited by non-robust single-view synthesis and the inability to handle sparse inputs effectively. To address these challenges, we construct a high-quality multi-view LF dataset, consisting of 60 indoor scenes and 59 outdoor scenes. Building upon this dataset, we propose a baseline method. Specifically, we introduce an adaptive alignment module to dynamically align information by capturing relative displacements. Next, we explore angular consistency and hierarchical information using a multi-level feature decoupling module. Finally, a multi-level feature refinement module is applied to enhance features and facilitate reconstruction. Additionally, we introduce a universally applicable artifact-aware loss function to effectively suppress visual artifacts. Experimental results demonstrate that our method outperforms existing approaches, establishing a benchmark for sparse-to-dense inbetweening. The code is available at https://github.com/Starmao1/MutiLF.

Light field imaging can simultaneously record the position and direction information of light rays; thus, digital refocusing and full depth-of-field extension — functions that are inaccessible for conventional images — can be achieved using the structural consistency of light field data. To meet the challenges of limited bandwidth and storage, such vast numbers of light field data must be compressed to a low bitrate. However, current compression solutions ignore the intrinsic consistency of light fields in pursuit of a low bitrate, thereby leading to the loss of light field capabilities. To solve this issue, this work focuses on structural consistency to achieve efficient light field compression with a low bitrate. The proposed light field compression method encodes the sparsely selected sub-aperture images (SAIs) and the disparity maps corresponding to the unselected SAIs. From the perspective of geometry consistency, the consistency of the initially estimated disparity maps is improved by using a color-guided refinement algorithm, thereby reducing the bitrate of the disparity maps. From the perspective of content consistency, the consistency of the SAI-transformed pseudo sequence is improved by the proposed content-similarity-based arrangement algorithm along with a specific prediction structure; thereby, the bitrate of the sparsely selected SAIs is reduced. The experimental results show that the proposed compression method can reduce the total bitrate while preserving good structural consistency.