Forecasting 3-dimensional skeleton-based human poses from the historical sequence is a classic task, which shows enormous potential in robotics, computer vision, and graphics. Currently, the state-of-the-art methods resort to graph convolutional networks (GCNs) to access the relationships of human joint pairs to formulate this problem. However, human action involves complex interactions among multiple joints, which presents a higher-order correlation overstepping the pairwise (2-order) connection of GCNs. Moreover, joints are typically activated by the parent joint, rather than driving their parent joints, whereas in existing methods, this specific direction of information transmission is ignored. In this work, we propose a novel hybrid directed hypergraph convolution network (H-DHGCN) to model the high-order relationships of the human skeleton with directionality. Specifically, our H-DHGCN mainly involves 2 core components. One is the static directed hypergraph, which is pre-defined according to the human body structure, to effectively leverage the natural relations of human joints. The second is dynamic directed hypergraph (D-DHG). D-DHG is learnable and can be constructed adaptively, to learn the unique characteristics of the motion sequence. In contrast to the typical GCNs, our method brings a richer and more refined topological representation of skeleton data. On several large-scale benchmarks, experimental results show that the proposed model consistently surpasses the latest techniques.

Few-Shot Action Recognition (FSAR) aims to recognize novel class action with limited annotated training data from the same class. Most FSAR methods subconsciously follow the few-shot image classification solutions by solely focusing on appearance-level matching between support and query videos, such as part-level matching, frame-level matching, and segment-level matching. However, these methods, almost always, have two main limitations: 1) generally ignore the relationship among these part-, frame- and segment-level features; and 2) may mismatch the same class actions under fast-term and slow-term dynamics. To this end, we present a novel Hierarchical Motion-enhanced Matching (HM ${^{2}}$ ) framework to hierarchically learn the relation-aware multi-modal features, and jointly promote the multi-modal matching, including appearance-level matching on segments, frames, and parts, as well as the motion-level matching on dynamics. Specifically, we first propose a new Hierarchical Tokenizer (HT) to learn multi-modal features, namely utilizing a hierarchical Transformer to learn appearance-level features, along with a Slow-Fast Aware Motion (SFAM) strategy to learn motion-level features covering fast- and slow-term dynamics. Next, we propose a new Relation-aware Matcher (RM) to match the multi-modal features, by leveraging a Hierarchical Relational Graph Convolutional Network (H-RGCN) to capture the relationship among these appearance-level features. Further, a Dual Sample-to-Class Matching (DSCM) strategy is proposed to measure the bidirectional similarities among appearance- and motion-modal features by sample-to-class matching and class-to-sample matching. Extensive experiments on four golden FSAR datasets demonstrate significant performance improvements of HM ${^{2}}$ compared with the state-of-the-art methods.

Existing Graph Convolutional Networks to achieve human motion prediction largely adopt a one-step scheme, which output the prediction straight from history input, failing to exploit human motion patterns. We observe that human motions have transitional patterns and can be split into snippets representative of each transition. Each snippet can be reconstructed from its starting and ending poses referred to as the transitional poses. We propose a snippet-to-motion multi-stage framework that breaks motion prediction into sub-tasks easier to accomplish. Each sub-task integrates three modules: transitional pose prediction, snippet reconstruction, and snippet-to-motion prediction. Specifically, we propose to first predict only the transitional poses. Then we use them to reconstruct the corresponding snippets, obtaining a close approximation to the true motion sequence. Finally we refine them to produce the final prediction output. To implement the network, we propose a novel unified graph modeling, which allows for direct and effective feature propagation compared to existing approaches which rely on separate space-time modeling. Extensive experiments on Human 3.6M, CMU Mocap and 3DPW datasets verify the effectiveness of our method which achieves state-of-the-art performance.