Occlusion and pose variations, which can change facial appearance significantly, are two major obstacles for automatic Facial Expression Recognition (FER). Though automatic FER has made substantial progresses in the past few decades, occlusion-robust and pose-invariant issues of FER have received relatively less attention, especially in real-world scenarios. This paper addresses the real-world pose and occlusion robust FER problem in the following aspects. First, to stimulate the research of FER under real-world occlusions and variant poses, we annotate several in-the-wild FER datasets with pose and occlusion attributes for the community. Second, we propose a novel Region Attention Network (RAN), to adaptively capture the importance of facial regions for occlusion and pose variant FER. The RAN aggregates and embeds varied number of region features produced by a backbone convolutional neural network into a compact fixed-length representation. Last, inspired by the fact that facial expressions are mainly defined by facial action units, we propose a region biased loss to encourage high attention weights for the most important regions. We validate our RAN and region biased loss on both our built test datasets and four popular datasets: FERPlus, AffectNet, RAF-DB, and SFEW. Extensive experiments show that our RAN and region biased loss largely improve the performance of FER with occlusion and variant pose. Our method also achieves state-of-the-art results on FERPlus, AffectNet, RAF-DB, and SFEW. Code and the collected test data will be publicly available.

With the deepening of research on Large Language Models (LLMs), significant progress has been made in recent years on the development of Large Multimodal Models (LMMs), which are gradually moving toward Artificial General Intelligence. This paper aims to summarize the recent progress from LLMs to LMMs in a comprehensive and unified way. First, we start with LLMs and outline various conceptual frameworks and key techniques. Then, we focus on the architectural components, training strategies, fine-tuning guidance, and prompt engineering of LMMs, and present a taxonomy of the latest vision–language LMMs. Finally, we provide a summary of both LLMs and LMMs from a unified perspective, make an analysis of the development status of large-scale models in the view of globalization, and offer potential research directions for large-scale models.

Diffusion models have swiftly taken the lead in generative modeling, establishing unprecedented standards for producing high-quality, varied outputs. Unlike Generative Adversarial Networks (GANs)—once considered the gold standard in this realm—diffusion models bring several unique benefits to the table. They are renowned for generating outputs that more accurately reflect the complexity of real-world data, showcase a wider array of diversity, and are based on a training approach that is comparatively more straightforward and stable. This survey aims to offer an exhaustive overview of both the theoretical underpinnings and practical achievements of diffusion models. We explore and outline three core approaches to diffusion modeling: denoising diffusion probabilistic models, score-based generative models, and stochastic differential equations. Subsequently, we delineate the algorithmic enhancements of diffusion models across several pivotal areas. A notable aspect of this review is an in-depth analysis of leading generative models, examining how diffusion models relate to and evolve from previous generative methodologies, offering critical insights into their synergy. A comparative analysis of the merits and limitations of different generative models is a vital component of our discussion. Moreover, we highlight the applications of diffusion models across computer vision, multi-modal generation, and beyond, culminating in significant conclusions and suggesting promising avenues for future investigation.

Despite Graph Neural Networks (GNNs) demonstrating considerable promise in graph representation learning tasks, GNNs predominantly face significant issues with overfitting and over-smoothing as they go deeper as models of computer vision (CV) realm. The success of artificial intelligence in computer vision and natural language processing largely stems from its ability to train deep models effectively. We have thus conducted a systematic study on deep GNN models. Our findings indicate that the current success of deep GNNs primarily stems from (I) the adoption of innovations from CNNs, such as residual/skip connections, or (II) the tailor-made aggregation algorithms like DropEdge. However, these algorithms often lack intrinsic interpretability and indiscriminately treat all nodes within a given layer in a similar manner, thereby failing to capture the nuanced differences among various nodes. In this paper, we introduce the Snowflake Hypothesis -- a novel paradigm underpinning the concept of "one node, one receptive field''. The hypothesis draws inspiration from the unique and individualistic patterns of each snowflake, proposing a corresponding uniqueness in the receptive fields of nodes in the GNNs. We employ the simplest gradient and node-level cosine distance as guiding principles to regulate the aggregation depth for each node, and conduct comprehensive experiments including: (1) different training scheme; (2) various shallow and deep GNN backbones; (3) various numbers of layers (8, 16, 32, 64) on multiple benchmarks; (4) compare with different aggregation strategies. The observational results demonstrate that our framework can serve as a universal operator for a range of tasks, and it displays tremendous potential on deep GNNs. Code is available at: https://github.com/CunWang520/Snowhypothe.