Thanka murals are important cultural heritages of Tibet, but many precious murals were damaged during history. Thanka mural restoration is very important for the protection of Tibetan cultural heritage. Partial convolution has great potential for Thanka mural restoration due to its outstanding performance for inpainting irregular holes. However, three challenges prevent the existing partial convolution-based methods from solving Thanka restoration problems: 1) the features of multi-scale objects in Thanka murals cannot be extracted correctly because of single-scale partial convolution; 2) the stroke-like Thanka inpainting mode cannot be effectively simulated and learned by existing rectangular or arbitrary masks; and 3) the original content of damaged Thanka murals cannot be restored. To resolve these problems, we propose a Thanka mural inpainting method based on multi-scale adaptive partial convolution and stroke-like masks. The proposed method consists of three parts: 1) a kernel-level multi-scale adaptive partial convolution (MAPConv) to accurately discriminate valid pixels from invalid pixels, and to extract the features of multi-scale objects; 2) a parameter-configurable stroke-like mask generation method to simulate and learn the stroke-like Thanka inpainting mode; and 3) a 2-phase learning framework based on MAPConv Unet and different loss functions to restore the original content of Thanka murals. Experiments on both simulated and real damages of Thanka murals demonstrated that our approach works well on a small dataset (N=2780), generates realistic mural content, and restores the damaged Thanka murals with high speed (600 ms for multiple holes in 512×512 images). The proposed end-to-end method can be applied to other small datasets-based inpainting tasks.

Thangka is an important intangible cultural heritage of Tibet. Due to the complexity, and time-consuming nature of the Thangka painting technique, this technique is currently facing the risk of being lost. It is important to preserve the art of Thangka through digital painting methods. Machine learning-based auto-sketch colorization is one of the vital steps for digital Thangka painting. However, existing learning-based sketch colorization methods face two challenges in solving the problem of colorizing Thangka: (1) the extremely rich colors of the Thangka make it difficult to color accurately with existing algorithms, and (2) the line density of the Thangka brings extreme challenges for algorithms to define what semantic information the lines imply. To resolve these problems, we propose a Thangka sketch colorization method based on multi-level adaptive-instance-normalized color fusion (MACF) and skip connection attention (SCA). The proposed method consists of two parts: (1) a multi-level adaptive-instance-normalized color fusion (MACF) to fuse sketch feature and color feature; and (2) a skip connection attention (SCA) mechanism to distinguish the semantic information implied by the sketch lines. Experiments on colorizing Thangka sketches show that our method works well on two small datasets—the Danbooru 2019 dataset and the Thangka dataset. Our approach can generate exquisite Thangka.

Thangka murals hold significant cultural value in Tibet, but unfortunately, many of these precious murals have sustained damage over time. Therefore, the restoration of Thangka murals is crucial in preserving Tibet’s cultural heritage. The potential of Gated Convolution for Thangka mural restoration is significant due to its impressive performance. However, existing Gated Convolution-based approaches for Thangka restoration face two challenges: (1) restoration results using Gated Convolution still show blurring at the pixel level, and (2) Gated Convolution remains dependent on the perceptual field of the convolution during image inpainting. To address two challenges, we propose a novel Thangka mural inpainting method that incorporates two key components: (1) an Edge-assisted Feature Fusion (EFF) method that leverages edge information to assist in inpainting the texture details of the image and enhance the consistency of textures and edges, and (2) a Self Attention based Local Refine (SALR) network that effectively enhances the perceptual field of the convolution and captures long-distance dependencies. The experimental results demonstrate that our proposed method is effective in repairing broken areas of Thangka murals, even on a small dataset (N=5605). To validate the effectiveness of our proposed architecture and its components, we conduct a series of ablation experiments. The results demonstrate the necessity of our approach and highlight the contributions of each component. Moreover, the proposed end-to-end approach exhibits broad applicability and can be extended to other profiling tasks based on small datasets.

Thangka murals are precious cultural heritage for Tibetan history, literature, and art. Digital line drawing of Thangka murals plays a vital role not only as an abstracted expression of Thangka for art appreciation but also as a fundamental digital resource for Thangka protection. Digital Thangka line drawing can be categorized as image edge detection, which as a fundamental problem for computer vision, aims to extract visually salient edges from images. Varieties of high-level computer vision tasks depend on edge detection. Although existing non-learning and learning-based edge detection methods have progressed, they failed to generate semantically plausible thin edges, especially thin in-object edges. We propose a novel deep supervised edge detection solution Richer In-object Thin Edge Network (RITE-Net) to generate line drawings of Thangka mural images. Compared to existing studies, firstly a new Cross Dense Residual architecture (CDR) is proposed to propagate abundant edge features effectively from shallow layers to deep layers of CNN using a long-range feature memory; Secondly, a new Hard Pixel Balancing (HPB) based loss function strategy is designed to focus on hard pixel distinguishment. Experiments and tests on different datasets show that the proposed RITE-Net is able to produce more visually plausible and richer thin edge maps comparing to the existing methods. Both objective and subjective evaluations validated the competitive performance of our method.

Style transfer is a challenging computer vision task. As an important cultural heritage in the Tibetan region, Thangka murals cover history, culture, and religion. The stylistic recreation of Thangka images not only promotes better appreciation of Thangka art but also provides a new art form for Thangka. Two challenges prevent the existing methods from solving Thangka stylization problems: (1) feature point based attention networks ignore manifold alignment of features between content images and style images and are unable to achieve regional consistency of stylized Thangka images well. (2) Due to the lack of comprehensive understanding of content and style losses in previous studies, the style-content trade-off of the stylized Thangka is unable to be guaranteed. To solve these problems, we propose a progressive style-attentional network (PSANet) and a multi-level loss function strategy for Thangka style transfer. The proposed method consists of two parts: (1) a PSANet to align content manifolds and style manifolds. (2) A multi-level loss function strategy to achieve a balance between content and style of the stylized Thangka images and to ensure better content preservation. Qualitative and quantitative experiments show that our proposed method is able to achieve satisfactory stylized effects on Thangka images.