9 October 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Unsupervised Domain Adaptation (UDA) makes predictions for the target domain data while manual annotations are only available in the source domain. Previous methods minimize the domain discrepancy neglecting the class information, which may lead to misalignment and poor generalization performance. To tackle this issue, this paper proposes Contrastive Adaptation Network (CAN) that optimizes a new metric named Contrastive Domain Discrepancy explicitly modeling the intra-class domain discrepancy and the inter-class domain discrepancy. To optimize CAN, two technical issues need to be addressed: 1) the target labels are not available and 2) the conventional mini-batch sampling is imbalanced. Thus we design an alternating update strategy to optimize both the target label estimations and the feature representations. Moreover, we develop class-aware sampling to enable more efficient and effective training. Our framework can be generally applied to the single-source and multi-source domain adaptation scenarios. In particular, to deal with multiple source domain data, we propose 1) multi-source clustering ensemble which exploits the complementary knowledge of distinct source domains to make more accurate and robust target label estimations, and 2) boundary-sensitive alignment to make the decision boundary better fitted to the target. Experiments conducted on three real-world benchmarks, demonstrating CAN performs favorably against previous state-of-the-arts.
5 May 2017 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Recent years have witnessed the success of deep neural networks in dealing with a plenty of practical problems. Dropout has played an essential role in many successful deep neural networks, by inducing regularization in the model training. In this paper, we present a new regularized training approach: Shakeout. Instead of randomly discarding units as Dropout does at the training stage, Shakeout randomly chooses to enhance or reverse each unit's contribution to the next layer. This minor modification of Dropout has the statistical trait: the regularizer induced by Shakeout adaptively combines L 0 , L 1 and L 2 regularization terms. Our classification experiments with representative deep architectures on image datasets MNIST, CIFAR-10 and ImageNet show that Shakeout deals with over-fitting effectively and outperforms Dropout. We empirically demonstrate that Shakeout leads to sparser weights under both unsupervised and supervised settings. Shakeout also leads to the grouping effect of the input units in a layer. Considering the weights in reflecting the importance of connections, Shakeout is superior to Dropout, which is valuable for the deep model compression. Moreover, we demonstrate that Shakeout can effectively reduce the instability of the training process of the deep architecture.
2 June 2025 in IEEE Transactions on Image Processing
Previous methods utilize the Neural Radiance Field (NeRF) for panoptic lifting, while their training and rendering speed are unsatisfactory. In contrast, 3D Gaussian Splatting (3DGS) has emerged as a prominent technique due to its rapid training and rendering speed. However, unlike NeRF, the conventional 3DGS may not satisfy the basic smoothness assumption as it does not rely on any parameterized structures to render (e.g., MLPs). Consequently, the conventional 3DGS is, in nature, more susceptible to noisy 2D mask supervision. In this paper, we propose a new method called PLGS that enables 3DGS to generate consistent panoptic segmentation masks from noisy 2D segmentation masks while maintaining superior efficiency compared to NeRF-based methods. Specifically, we build a panoptic-aware structured 3D Gaussian model to introduce smoothness and design effective noise reduction strategies. For the semantic field, instead of initialization with structure from motion, we construct reliable semantic anchor points to initialize the 3D Gaussians. We then use these anchor points as smooth regularization during training. Additionally, we present a self-training approach using pseudo labels generated by merging the rendered masks with the noisy masks to enhance the robustness of PLGS. For the instance field, we project the 2D instance masks into 3D space and match them with oriented bounding boxes to generate cross-view consistent instance masks for supervision. Experiments on various benchmarks demonstrate that our method outperforms previous state-of-the-art methods in terms of both segmentation quality and speed.
7 December 2023 in IEEE Transactions on Multimedia
Domain generalization (DG) aims to train a model with access to a limited number of source domains for generalizing it across various unseen target domains. The key to solving the DG problem is disentangling domain-invariant features ( i.e., semantic factors) from domain-specific features ( i.e., variation factors) to facilitate generalizable representation learning. Previous studies either implicitly model the semantic and variation factors or ineffectively constrain the disentangling process, thus rendering the disentanglement incomplete and ineffective. In this study, we propose a novel approach, named DualVAE , to explicitly model and disentangle both the semantic and variation factors. DualVAE is based on the variational autoencoder (VAE) architecture. However, it differs from the conventional VAE in that it consists of two paths, which explicitly model the semantic and variation factors. In addition to the reconstruction loss of VAE and the classification loss, three types of regularizations, namely statistical independence regularization, factorized prior regularization, and prediction consistency regularization, are proposed to further facilitate the disentanglement of factors. Experimental results on representative DG benchmarks show that our method performs favourably against previous state-of-the-art methods. Ablation and visualization results demonstrate that semantic and variation factors can be effectively disentangled.