Recently, sketch-based 3D shape retrieval has received growing attention in the community of computer graphics and computer vision. Most previous works focus on the problem of how to reduce the large cross-modality difference between 2D sketch and 3D shape data and make significant progress. Nevertheless, little attention has been paid to another important problem of how to deal with noise in the sketch data . For the first time, this work investigates the problem of noisy sketch data. It firstly provides qualitative and insightful analysis on the impact of noise, revealing that the noisy data are a key factor for unsatisfactory retrieval performance, as they cause severe over fitting and impair feature learning. Thus, the issue is worthy of serious treatment. Then, we propose to estimate sketch noise as data uncertainty, motivated by existing ideas that model data uncertainty with a distributional representation. We present methods with simple network structure and loss functions. They achieve strong results and establish new state-of-the-art on two benchmarks. Comprehensive experiment results, ablation studies, and insightful analysis validate the effectiveness of our methods, revealing that sketch feature learning with uncertainty is crucial for noise resistant sketch based 3D shape retrieval.

Visual relationships are crucial for visual perception and reasoning, and cover tasks like Scene Graph Generation, Human-Object Interaction, and object affordance. Despite significant efforts, this field still suffers from the following limitations: specialists for a specific task without considering similar ones, strict and complex task formulations with limited flexibility, and underexploited reasoning with language and knowledge. To solve these limitations, we seek to build a new framework, one model for all tasks, over Large Multimodal Models (LMMs). LMMs offer the potential of unifying tasks, flexible forms, and reasoning with language. However, they fail to handle visual relationship tasks well. We find the obstacles include the conflicts between different tasks and insufficient instance-level information. We solve these problems by reforming the data for LMMs, rather than architectures, considering their strong language-in language-out capability. We propose to disassemble tasks into simple and common sub-tasks, verbally estimate instance confidence, and augment instance diversity, all without additional modules. These strategies help us build a visual relationship generalist, RelationLMM, with a simple architecture. Exhaustive experiments demonstrate RelationLMM is strong, generalizable and flexible to different tasks, with one model and one suite of weight.

Weakly supervised action localization is a challenging problem in video understanding and action recognition. Existing models usually formulate the training process as direct classification using video-level supervision. They tend to only locate the most discriminative parts of action instances and produce temporally incomplete detection results. A natural solution for this problem, the adversarial erasing strategy, is to remove such parts from training so that models can attend to complementary parts. Previous works do it in an offline and heuristic way. They adopt a multi-stage pipeline, where discriminative regions are determined and erased under the guidance of detection results from last stage. Such a pipeline can be both ineffective and inefficient, possibly hindering the overall performance. On the contrary, we combine adversarial erasing with dropout mechanism and propose a Temporal Dropout Module that learns where to remove in a data-driven and online manner. This plug-and-play module is trained without iterative stages, which not only simplifies the pipeline but also makes the regularization during training easier and more adaptive. Experiments show that the proposed method outperforms previous erasing-based methods by a large margin. More importantly, it achieves universal improvement when plugged into various direct classification methods and obtains state-of-the-art performance.