In this paper, we propose an advanced three-dimensional visualization method for an integral imaging microscope system to simultaneously improve the resolution and quality of the reconstructed image. The main advance of the proposed method is that it generates a high-quality three-dimensional model without limitation of resolution by combining the high-resolution two-dimensional color image with depth data obtained through a fully convolutional neural network. First, the high-resolution two-dimensional image and an elemental image array for a specimen are captured, and the orthographic-view image is reconstructed from the elemental image array. Then, via a convolutional neural network-based depth estimation after the brightness of input images are uniformed, a more accurate and improved depth image is generated; and the noise of result depth image is filtered. Subsequently, the estimated depth data is combined with the high-resolution two-dimensional image and transformed into a high-quality three-dimensional model. In the experiment, it was confirmed that the displayed high-quality three-dimensional model could be visualized very similarly to the original image.

Four-dimensional (4D) light-field (LF) microscopes can acquire 3D information about target objects using a microlens array (MLA). However, the resolution and quality of sub-images in the LF images are reduced because of the spatial multiplexing of rays by the element lenses of the MLA. To overcome these limitations, this study proposes an LF one-shot learning technique that can convert LF sub-images into high-quality images similar to the 2D images of conventional optical microscopes obtained without any external training datasets for image enhancement. The proposed convolutional neural network model was trained using only one training dataset comprising a high-resolution reference image captured without an MLA as the ground truth. Further, its input was the central view of the LF image. After LF one-shot learning, the trained model should be able to convert well the other LF sub-images of various directional views that were not used in the main training process. Therefore, novel learning techniques were designed for LF one-shot learning. These novel techniques include an autoencoder-based model initialization method, a feature map-based learning algorithm to prevent the overfitting of the model, and cut loss to prevent saturation. The experimental results verified that the proposed technique effectively enhances the LF image quality and resolution using a reference image. Moreover, this method enhances the resolution by up to 13 times, decreases the noise amplification effect, and restores the lost details of microscopic objects. The proposed technique is stable and yields superior experimental results compared with those of the existing resolution-enhancing methods.

We propose a light-field microscopy display system that provides improved image quality and realistic three-dimensional (3D) measurement information. Our approach acquires both high-resolution two-dimensional (2D) and light-field images of the specimen sequentially. We put forward a matting Laplacian-based depth estimation algorithm to obtain nearly realistic 3D surface data, allowing the calculation of depth data, which is relatively close to the actual surface, and measurement information from the light-field images of specimens. High-reliability area data of the focus measure map and spatial affinity information of the matting Laplacian are used to estimate nearly realistic depths. This process represents a reference value for the light-field microscopy depth range that was not previously available. A 3D model is regenerated by combining the depth data and the high-resolution 2D image. The element image array is rendered through a simplified direction-reversal calculation method, which depends on user interaction from the 3D model and is displayed on the 3D display device. We confirm that the proposed system increases the accuracy of depth estimation and measurement and improves the quality of visualization and 3D display images.