We investigate the problem of sketch generation for sketch-based image retrieval (SBIR). Solving this problem is important to obtain better retrieval results, because a powerful feature extraction algorithm is inefficient. Transforming photos from raw pixels into pseudo-sketches closes the gap between these two domains and plays a significant role in SBIR. This problem is relatively challenging because of: 1) the complexity of the image background and 2) the complexity of the resulting edge map. In this paper, we develop a system to generate pseudo-sketches from photos. Saliency detection is used to extract the major objects from a photo. Then, a Gabor filter is designed to further capture the “real-major” object. Finally, the Sobel operator is used to obtain the final pseudo-sketch. Experiments are conducted using the Flickr15k data set. The results show that the obtained pseudo-sketches are reasonable and that the SBIR process produces the state-of-the-art results in certain categories.
Depth estimation is a key problem in 3D computer vision and has a wide variety of applications. In this paper we explore whether deep learning network can predict depth map accurately by learning multi-scale spatio-temporal features from sequences and recasting the depth estimation from a regression task to an ordinal classification task. We design an encoder-decoder network with several multi-scale strategies to improve its performance and extract spatio-temporal features with ConvLSTM. The results of our experiments show that the proposed method has an improvement of almost 10% in error metrics and up to 2% in accuracy metrics. The results also tell us that extracting spatio-temporal features can dramatically improve the performance in depth estimation task. We consider to extend this work to a self-supervised manner to get rid of the dependence on large-scale labeled data.
25 November 2014 in Journal of the Optical Society of America A
In this paper, a block sparse discriminative classification framework (BSDC) is proposed under the assumption that a block or group structure exists in sparse coefficients on classification. First, we propose a block discriminative dictionary-learning (BDDL) algorithm, which learns class-specific subdictionaries and forces the sparse coefficients to be block sparse. An efficient gradient-based optimization strategy of BDDL also is developed, and the block sparse constraint of the sparse coefficient leads to a least-squares solution of nonzero entries in the sparse coding stage of dictionary learning. Second, to take advantage of the structures when a new test sample is given, conventional sparse coding algorithms are discarded, and structured sparse coding methods are adopted. Experiments validate the effectiveness of the proposed framework in face recognition and texture classification. We also show that BSDC is robust to noise.
In recent years, sketches have attracted much attention as a form of free media, and at the same time, the automatic coloring task of hand-drawn sketches has become one of the research hotspots in the field of vision. Sketch coloring is a challenging task, mainly due to: a) lack of inherent color information; b) the result of coloring is prone to cross-color and overflow; c) poor controllability of the coloring process. Based on this, we design a unique deep network that combines self-attention mechanism to alleviate this problem in this paper. Specifically, we first use the encoding and decoding structure of U-net to capture the delicate details in the sketch; a unique self-attention module is then integrated to automatically capture key areas of the input sketch, thereby improving color distribution modeling and contextual awareness, enhancing the results of automatic coloring. Experiments on a large-scale anime sketch dataset show that the proposed method exhibits significant performance improvement, capturing delicate details in sketches while maintaining faithful color restoration. In addition, our model also supports guided coloring based on provided clues, which providing users more controllability.