27 June 2023 in IEEE Transactions on Intelligent Transportation Systems
Empowered by labeled datasets, supervised pre-training based transfer learning (SPTL) has made significant advances for image classification applications. However, due to privacy-preserving protocol and unaccessible annotation, it emerges as a novel problem in federated learning scenarios whether unsupervised pre-training based transfer learning (UPTL) is available for semantic segmentation. In this work, we define federated transfer learning (FedTL) in the absence of source domain label, and track research progress on pavement crack benchmark. The main challenges of FedTL include: i) a privacy-protecting distributed training framework that extends UPTL to the constraints of federated settings, and ii) a self-learning semantic segmentation approach that develops self-supervised learning paradigm to simultaneously learn category and shape representations. Motivated by that, we propose a FedCrack model to absorb feature disentanglement and prototype clustering into vision Transformer, which obtains the pre-trained encoder on source domain without accessing annotation. Thereafter, a fine-tuning stage is presented to learn decoder with scaling attention on target domain for fine-grained crack segmentation. The effectiveness of proposed FedCrack can be demonstrated with superior performance of 82.14% on mIoU and 9.85 FPS on speed in extensive experiments. To the best of our knowledge, it is the first work in FedTL to gain weights of unsupervised pre-training representations on source domain locally, gradients of which are then aggregated to a federated central model that also fine-tunes the transferable parameters by target domain.
25 December 2024 in IEEE Transactions on Multimedia
Semantic foggy scene understanding (SFSU) emerges a challenging task under out-of-domain distribution (OD) due to uncertain cognition caused by degraded visibility. With the strong assumption of data centralization, unsupervised domain adaptation (UDA) reduces vulnerability under OD scenario. Whereas, enlarged domain gap and growing privacy concern heavily challenge conventional UDA. Motivated by gap decomposition and data decentralization, we establish a decentralized domain adaptation (DDA) framework called Translate thEn Adapt (abbr. TEA) for privacy preservation. Our highlights lie in: (1) Regarding federated hallucination translation, a Disentanglement and Contrastive-learning based Generative Adversarial Network (abbr. DisCoGAN) is proposed to impose contrastive prior and disentangle latent space in cycle-consistent translation. To yield domain hallucination, client minimizes cross-entropy of local classifier but maximizes entropy of global model to train translator. (2) Regarding source-free regularization adaptation, a Prototypical-knowledge based Regularization Adaptation (abbr. ProRA) is presented to align joint distribution in output space. Soft adversarial learning relaxes binary label to rectify inter-domain discrepancy and inner-domain divergence. Structure clustering and entropy minimization drive intra-class features closer and inter-class features apart. Extensive experiments exhibit efficacy of our TEA which achieves 55.26% or 46.25% mIoU in adaptation from GTA5 to Foggy Cityscapes or Foggy Zurich, outperforming other DDA methods for SFSU. Code will be released at URL.