18 May 2021 in Proceedings of the AAAI Conference on Artificial Intelligence
Bilinear pooling has achieved an excellent performance in many computer vision tasks such as fine-grained classification, scene recognition and texture recognition. However, the high-dimension features from bilinear pooling can sometimes be inefficient and prone to over-fitting. Random Maclaurin (RM) is a widely used GPU-friendly approximation method to reduce the dimensionality of bilinear features. However, to achieve good performance, large projection matrices are usually required in practice, making it costly in computation and memory. In this paper, we propose a Shifted Random Maclaurin (SRM) strategy for fast and compact bilinear pooling. With merely negligible extra computational cost, the proposed SRM provides an estimator with a provably smaller variance than RM, which benefits accurate kernel approximation and thus the learning performance. Using a small projection matrix, the proposed SRM achieves a comparable estimation performance as RM based on a large projection matrix, and thus boosts the efficiency. Furthermore, we upgrade the proposed SRM to SRM+ to further improve the efficiency and make the compact bilinear pooling compatible with fast matrix normalization. Fast and Compact Bilinear Network (FCBN) built upon the proposed SRM+ is devised, achieving an end-to-end training. Systematic experiments conducted on four public datasets demonstrate the effectiveness and efficiency of the proposed FCBN.
25 October 2022 in IEEE Transactions on Intelligent Transportation Systems
Due to their low storage capacity and fast retrieval speed, hashing techniques have received much attention in cross-modal retrieval. However, there are some issues that need to be further explored. First, some existing hashing methods use the labels to construct the semantic similarity matrix between pairwise data, ignoring the potential manifold structure between heterogeneous data. Second, some existing methods underestimate the importance of multi-label and the gaps between different class labels, making the learned hash codes less discriminative. Third, few of them embed both manifold and balanced structures within the same model, and the relaxation of discrete constraints will lead to an increasing quantization error. To mitigate these problems, this paper proposes a novel supervised hashing method, termed Label Guided Discrete Hashing (LGDH), which simultaneously preserves the comprehensive manifold structure and discriminative balanced codes that are both constructed by label information into Hamming space. We develop a local category distribution of the nearest neighbors, to excavate the underlying manifold structure of heterogeneous data. To maximize the gaps of different categories, a balanced matrix is constructed by labels to generate hash codes with balanced bits. For multi-label data, we also design a novel multi-label manifold and balanced structure matrix to adapt the real-world scenarios. An effective discrete optimization method is used to optimize our proposed objective function instead of the relaxation one. Extensive experiments on three benchmark datasets verify the effectiveness of LGDH. The comparison results demonstrat that LGDH achieves about 2 $\%$ and 3 $\%$ improved to different cross-modal tasks on average.