Attention mechanisms have significantly boosted the performance of video classification neural networks thanks to the utilization of perspective contexts. However, the current research on video attention generally focuses on adopting a specific aspect of contexts (e.g., channel, spatial/temporal, or global context) to refine the features and neglects their underlying correlation when computing attentions. This leads to incomplete context utilization and hence bears the weakness of limited performance improvement. To tackle the problem, this paper proposes an efficient attention-in-attention (AIA) method for element-wise feature refinement, which investigates the feasibility of inserting the channel context into the spatio-temporal attention learning module, referred to as CinST, and also its reverse variant, referred to as STinC. Specifically, we instantiate the video feature contexts as dynamics aggregated along a specific axis with global average and max pooling operations. The workflow of an AIA module is that the first attention block uses one kind of context information to guide the gating weights calculation of the second attention that targets at the other context. Moreover, all the computational operations in attention units act on the pooled dimension, which results in quite few computational cost increase (< 0.02%). To verify our method, we densely integrate it into two classical video network backbones and conduct extensive experiments on several standard video classification benchmarks. The source code of our AIA is available at https://github.com/haoyanbin918/Attention-in-Attention .

Capturing cross-pose correlation from a sequence of frame-level 2D poses is essential for 3D human pose estimation (3D-HPE) in the video. Recent studies have shown the promising potential of modeling the pose relation with feature-mixing operations on the temporal domain. However, they seldom consider the interaction across poses in the frequency domain. This paper studies a Frequency-Temporal Collaborative Module (FTCM) to explore the feasibility of encoding the cross-pose correlations in both frequency and temporal domains. FTCM aims to jointly capture the global and local cross-pose correlations with a more lightweight network model. Specifically, FTCM splits the pose features into two groups along the channel dimension and separately models the frequency and temporal interactions across poses with different feature-mixing operations in parallel. To achieve this goal, we purposely design two pose-mixing units, i.e., the frequency pose-mixing (FPM) and the temporal pose-mixing (TPM). Particularly, FPM is designed to reap the global correlations among different pose frequencies with the representation obtained by converting the original pose signals with Fast Fourier transform (FFT). Unlike the pose-mixing used by previous methods like Transformers that influences an individual pose with all other poses, TPM locally calibrates the pose with dynamics aggregated within several adjacent poses in the temporal domain, explicitly weighting neighboring poses more with respect to the far-away ones so as to enforce a strict locality constraint. Besides, the group strategy significantly reduces the model complexity. To verify the effectiveness of FTCM, we conduct extensive experiments on two benchmarks (i.e., Human3.6M and MPI-INF-3DHP). Experimental results not only exhibit favorable accuracy/complexity trade-offs of our FTCM but also show superior or comparable performance to state-of-the-art methods on both datasets. The code and model are publicly available at: https://github.com/zhenhuat/FTCM.

Sentiment analysis is an important topic concerning identification of feelings, attitudes, emotions and opinions from text. A critical challenge for automating such analysis is the high manual annotation cost when conducting large-scale learning. However, the cross-domain technique is a key solution for this. It reuses annotated reviews across domains and its success principally relies on the learning of a robust common representation space across domains to support the sentiment classifier transfer. In the recent years, significant effort has been invested to improve the cross-domain representation learning by designing increasingly more complex and elaborate model inputs and architectures. We support that it is not necessary to focus on design complexity as this inevitably consumes more time for model training. Instead, we propose to explore through a simple mapping the word polarity and occurrence information and encode such information more accurately whilst aiming at lower computational costs. The proposed approach is unique and takes advantage of the stochastic embedding technique to tackle cross-domain sentiment alignment. Its effectiveness is benchmarked with over ten data tasks constructed from two review corpora, and is compared against ten classical and state-of-the-art methods.

Cross-modal hashing is an effective approach for information retrieval from large and heterogeneous cross-modal datasets, owing to its low storage cost and high computational speed. However, conventional cross-modal hashing techniques for generating hashing codes rely on cross-space dimensional compression, which results in two types of information loss: quantization information loss and dimension reduction loss. To address these limitations, we propose a novel method that decouples the one-step hashing (Fig.1a) strategy into two sub-steps (Fig.1b). Specifically, in the first step, we introduce a novel differentiable hash method, which utilizes a smooth hash module for binary quantization. This method allows our model to reduce the quantization information loss and make the model optimized by gradient descent. In the second step, we design a long-short Hamming space transformation approach to project the long code into a short one, which is effective in preserving the dimension information between long and short and mitigating the dimension reduction loss. We demonstrate the effectiveness of our approach through extensive experiments on several popular cross-modal datasets, achieving a significant improvement in cross-modal retrieval performance.