22 December 2014 in Computer Graphics Forum
23 January 2018 in IEEE Transactions on Multimedia
In order to retrieve unlabeled images by textual queries, cross-media similarity computation is a key ingredient. Although novel methods are continuously introduced, little has been done to evaluate these methods together with large-scale query log analysis. Consequently, how far have these methods brought us in answering real-user queries is unclear. Given baseline methods that use relatively simple text/image matching, how much progress have advanced models made is also unclear. This paper takes a pragmatic approach to answering the two questions. Queries are automatically categorized according to the proposed query visualness measure and later connected to the evaluation of multiple cross-media similarity models on three test sets. Such a connection reveals that the success of the state of the art is mainly attributed to their good performance on visual-oriented queries, which account for only a small part of real-user queries. To quantify the current progress, we propose a simple text2image method, representing a novel query by a set of images selected from large-scale query log. Consequently, computing cross-media similarity between the query and a given image boils down to comparing the visual similarity between the given image and the selected images. Image retrieval experiments on the challenging Clickture dataset show that the proposed text2image is a strong baseline, comparing favorably to recent deep learning alternatives.
1 August 2019 in Journal of Physics: Conference Series
It is a difficult and challenging task to restore the digital murals to a visually pleasant result, and even the result is similar to the original murals without corruption. In this paper, to address the above problem, we propose an image inpainting strategy called PCSW for digital Dunhuang murals using partial convolutions and sliding window method. Specially, a deep neural network based on partial convolutions is used as the underlying model for image inpainting. Because the murals are somewhat damaged or even large areas are missing, in addition, digital murals are large and high resolution, it is unreasonable and impractical to use the original digital murals for training and then restoring the missing areas. Therefore, a data augmentation method based on sliding window technique is applied to increase samples and then improve the model accuracy. Experimental results have shown that the proposed strategy has a certain effect on the restoration of digital Dunhuang murals.
12 September 2021 in Neural Computing and Applications
21 January 2014 in Journal of Shanghai Jiaotong University (Science)