Text-based visual question answering (TextVQA) requires reasoning over images containing rich textual content, often involving knowledge beyond what is directly observable. Existing methods fuse visual objects and OCR tokens but struggle when questions require external knowledge. Moreover, naively incorporating retrieved knowledge often introduces irrelevant or misleading information, which may hinder reasoning rather than support it. To address these challenges, we propose a TextVQA framework that integrates external prior knowledge to support multimodal reasoning. Given an image and question, a task-aware knowledge retrieval module selects relevant candidates, which are then filtered and verified by a knowledge verification module leveraging large language models. The verified knowledge and question are compressed into compact embeddings via a perceiver-based semantic resampler and jointly processed with visual and OCR features in a multimodal reasoning module. Experiments on the TextVQA and ST-VQA datasets demonstrate that our approach effectively leverages external knowledge to improve performance on knowledge-intensive questions.
Although generative models have achieved impressive visual plausibility in image inpainting, they still face challenges in high-fidelity content restoration, as the generated structures may deviate from real-world appearances. In this paper, to achieve accurate inpainting results, we propose the Focus-and-Disperse Inpainting (FDInpaint), a fine-to-broad context aggregation network that follows a focus-and-disperse paradigm. Specifically, FDInpaint first focuses on accurately identifying and reconstructing subtle details and textures through optimized gradient propagation, and subsequently disperses these refined cues across the entire image to seamlessly integrate restored regions with known content, thereby maintaining global visual consistency. Firstly, considering that image textures are intrinsically characterized by local intensity variations and geometric transitions, gradient information provides essential structural priors for accurately recovering missing content. By capturing complementary edge variations and curvature changes, gradient propagation effectively guides the reconstruction of sharp boundaries, delicate textures, and coherent structures, while reducing ambiguity and artifacts introduced during content generation. According to that, we design the Multi-Granularity Gradient-based Detail Perceptron (MGDP) within a scale-cross augmentation framework to explicitly localize and restore subtle micro-structures that are difficult to recover through semantic features alone. Furthermore, we propose the Patch-based Hybrid Consistency Aggregation (PHCA), a hierarchical consistency propagation module that progressively enlarges the optimization region from small local patches to larger contextual patches and ultimately the whole image. Through this patch-to-image aggregation process, PHCA continuously aligns reconstructed content with known regions by modeling local details and global dependencies, thereby achieving seamless transitions and maintaining global visual consistency. This refinement process enables seamless transitions between inpainted and known areas while maintaining global structural coherence without relying on stochastic sampling. Extensive experiments on Places2, CelebA-HQ, ICDAR, and COCO datasets demonstrate that FDInpaint achieves superior restoration accuracy and visual quality compared with state-of-the-art inpainting methods.
Human activity recognition has become an important component of human–computer interaction and ubiquitous computing. Among various sensing technologies, WiFi-based gesture recognition has attracted increasing attention due to its contactless nature and robustness to visual occlusion. However, environmental variations and user-specific differences often lead to significant performance degradation, particularly in cross-user scenarios. Existing methods primarily focus on learning domain-invariant representations, which may overlook user-specific characteristics that are essential for accurate recognition. To address this issue, we propose the Domain-invariant and User-Adaptive Learning Network (DUAL-Net), a dual-branch framework that jointly models domain-invariant and user-adaptive representations. Specifically, DUAL-Net incorporates a contrastive fusion learning (CFL) module with modality-specific encoders to learn complementary representations from WiFi and vision modalities. Furthermore, a spatial matrix difference (SMD)-guided cross-modal generation (CMG) module is introduced to generate user-adaptive WiFi features by incorporating structural priors derived from skeletal representations. To improve deployment efficiency, DUAL-Net adopts a two-stage learning framework, where adaptation is conducted offline to reduce online computational overhead. Experiments on the MM-Fi dataset and a self-collected dataset show that DUAL-Net achieves superior cross-user recognition performance compared with existing single-modality and multimodal methods. In addition, SMD-guided conditioning improves recognition accuracy by up to 8.79% over diffusion generation without structural guidance.