The digital construction of cultural heritage promotes communication and sharing of digital cultural resources across time and space. Digital storage serves as the foundation for the digital construction of cultural artifacts. In the digital storage of Dunhuang murals, image stitching plays a critical role in restoring the complete image of the cave murals. Traditional image stitching methods are constrained by the detection accuracy of feature points and are not fit for stitching low-texture murals. Despite deep learning-based image stitching methods, parallax misalignment and ghosting are still prevalent issues. For this reason, we perform the first Dunhuang mural stitching based on deep learning in this paper. This is in response to the need for digitizing and storing Dunhuang murals. Two mural stitching datasets are constructed, and we design a progressive regression image alignment network and a feature differential reconstruction soft-coded seam stitching network. We also introduce a soft-coded seam quality evaluation method. The algorithm presented in this paper achieves state-of-the-art alignment and stitching performance in the mural stitching task through unsupervised learning with a smaller number of model parameters, which provides technical support for the digitization and preservation of Dunhuang murals. The codes and models will be available at https://github.com/MmelodYy/DunHuangStitch .
19 June 2024 in IEEE Transactions on Consumer Electronics
Image rectangling research aims to solve the problem of irregular boundaries in stitched images, which helps to enhance the visual satisfaction of electronic consumers with the imaging of electronic products. To do so, it learns the mesh warping rules of the image, allowing for the generation of a rectangular image with content fidelity. Existing image rectangling solutions either require two stages of warping processing, or one stage but multiple levels of warping processing to achieve the rectangling effect, both of which are inefficient at rectangling and are prone to cumulative errors and distortions of images. Based on an optimization exploration of a single-stage and multi-level warping approach, we construct an image rectangling knowledge distillation task. An image rectangling network with single-stage and single-level warping is proposed after analyzing the distillation task construction impact factors. The proposed method, combined with our multi-teacher guided instance-level knowledge distillation strategy, utilizes only 2.53% of the parameters employed by the state-of-the-art method. It achieves improvements of 4.68% in SSIM, 1.94% in PSNR, and 1.01% in NIQE metrics, while also achieving a speedup of 81.38%. The codes and models will be available at https://github.com/MmelodYy/Distill_Rectangling.
To better preserve the global semantic information and edge information of the original image during the multi-focus image fusion process, we propose an encoder–decoder network, which combines pixel loss function, multiscale structural similarity loss function and total variation loss function to further reduce the detail loss in the image reconstruction process. The model introduces dynamic convolution and global context network to improve its feature expression ability and global context modeling ability. In the fusion stage, the fusion strategy based on the edge feature map is used to fuse the two feature maps output by the encoder to obtain the final decision map. Combining the gradient information of the deep feature map and the edge feature map, the fusion strategy can more effectively extract the flat parts of the focused and defocused regions and enhance the edge features of the image. Finally, the proposed algorithm is compared with seven advanced fusion algorithms, and the results are superior to other fusion algorithms in subjective and objective evaluation indexes.