Kaidong Zhang

Kaidong Zhang

Auto-generated profile, based on publicly available information

Kaidong Zhang

MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 November 2022
Verified
Book Chapter
Closed access
PDF file
KZKaidong Zhang
Kaidong Zhang
JFJingjing Fu
Jingjing Fu
DLDong Liu
Dong Liu

17 October 2023 in arXiv
Verified
Preprint
Open access
PDF file
KZKaidong Zhang
Kaidong Zhang
DLDong Liu
Dong Liu
We propose SAMed, a general solution for medical image segmentation. Different from the previous methods, SAMed is built upon the large-scale image segmentation model, Segment Anything Model (SAM), to explore the new research paradigm of customizing large-scale models for medical image segmentation. SAMed applies the low-rank-based (LoRA) finetuning strategy to the SAM image encoder and finetunes it together with the prompt encoder and the mask decoder on labeled medical image segmentation datasets. We also observe the warmup finetuning strategy and the AdamW optimizer lead SAMed to successful convergence and lower loss. Different from SAM, SAMed could perform semantic segmentation on medical images. Our trained SAMed model achieves 81.88 DSC and 20.64 HD on the Synapse multi-organ segmentation dataset, which is on par with the state-of-the-art methods. We conduct extensive experiments to validate the effectiveness of our design. Since SAMed only updates a small fraction of the SAM parameters, its deployment cost and storage cost are quite marginal in practical usage. The code of SAMed is available at https://github.com/hitachinsk/SAMed.

1 February 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
KZKaidong Zhang
Kaidong Zhang
JPJialun Peng
Jialun Peng
JFJingjing Fu
Jingjing Fu
DLDong Liu
Dong Liu
Transformers have been widely used for video processing owing to the multi-head self attention (MHSA) mechanism. However, the MHSA mechanism encounters an intrinsic difficulty for video inpainting, since the features associated with the corrupted regions are degraded and incur inaccurate self attention. This problem, termed query degradation, may be mitigated by first completing optical flows and then using the flows to guide the self attention, which was verified in our previous work – flow-guided transformer (FGT). We further exploit the flow guidance and propose FGT++ to pursue more effective and efficient video inpainting. First, we design a lightweight flow completion network by using local aggregation and edge loss. Second, to address the query degradation, we propose a flow guidance feature integration module, which uses the motion discrepancy to enhance the features, together with a flow-guided feature propagation module that warps the features according to the flows. Third, we decouple the transformer along the temporal and spatial dimensions, where flows are used to select the tokens through a temporally deformable MHSA mechanism, and global tokens are combined with the inner-window local tokens through a dual-perspective MHSA mechanism. FGT++ is experimentally evaluated to be outperforming the existing video inpainting networks qualitatively and quantitatively.

17 May 2024 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
CLChang Liu
Chang Liu
SXShunxin Xu
Shunxin Xu
JPJialun Peng
Jialun Peng
KZKaidong Zhang
Kaidong Zhang
DLDong Liu
Dong Liu
One tough problem of image inpainting is to restore complex structures in the corrupted regions. It motivates interactive image inpainting which leverages additional hints, e.g., sketches, to assist the inpainting process. A sketch is simple and intuitive for end users to provide, but meanwhile has free forms with much randomness. Such randomness may confuse the inpainting models, and incur severe artifacts in completed images. To better facilitate image inpainting with sketch guidance, we propose a two-stage image inpainting system, termed SketchRefiner. The first stage of our approach serves as a data provider that simulates real sketches and derives the capability of sketch calibration from the simulated data. In the second stage, our approach aligns the sketch guidance with the inpainting process so as to elevate image inpainting with sketches. We also propose a real-world test protocol to address the evaluation of inpainting methods upon practical applications with user sketches. Experimental results on three prevailing benchmark datasets, i.e., CelebA-HQ, Places2, and ImageNet, and the proposed test protocol demonstrate the state-of-the-art performance of our approach, and its great potentials upon real-world applications. Further analyses illustrate that our approach effectively utilizes sketch information as guidance and eliminates the artifacts due to the free-form sketches.

2 December 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
CLChang Liu
Chang Liu
RLRui Li
Rui Li
KZKaidong Zhang
Kaidong Zhang
YLYunwei Lan
Yunwei Lan
DLDong Liu
Dong Liu
Recent advancements in generative artificial intelligence have significantly promoted content creation and editing, where prevailing studies further extend this exciting progress to video editing. These studies mainly transfer the inherent motion patterns from the source videos to the edited ones, where they often produce inferior results with inconsistency to user intentions, especially when shape changes between the edited and original objects might occur, due to the lack of particular alignments between the delivered motions and edited content. To address this limitation, we present a shape-consistent video editing method, namely StableV2V. Our method decomposes the entire editing pipeline into several sequential procedures, where we first edit the initial video frame, then simulate the shape-aware alignment between the delivered motions and edited sequence, and propagate the edited content to all other frames based on such alignment. Furthermore, we curate a testing benchmark, namely DAVIS-Edit, to offer a comprehensive evaluation of video editing, considering various types of prompts and difficulties. Experimental results and analyses illustrate the superior performance, visual consistency, and inference efficiency of our proposed method compared to existing state-of-the-art video editing studies.

23 January 2026 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
CLChang Liu
Chang Liu
RLRui Li
Rui Li
KZKaidong Zhang
Kaidong Zhang
YLYunwei Lan
Yunwei Lan
XLXin Luo
Xin Luo
DLDong Liu
Dong Liu
Diffusion models have demonstrated impressive abilities in generating photo-realistic and creative images. To offer more controllability for the generation process of diffusion models, previous studies normally adopt extra modules to integrate condition signals by manipulating the intermediate features of the noise predictors, where they often fail in conditions not seen in the training. Although subsequent studies are motivated to handle multi-condition control, they are mostly resource-consuming to implement, where more generalizable and efficient solutions are expected for controllable visual generation. In this paper, we present a late-constraint controllable visual generation method, namely LaCon, which enables generalization across various modalities and granularities for each single-condition control. LaCon establishes an alignment between the external condition and specific diffusion timesteps, and guides diffusion models to produce conditional results based on this built alignment. Experimental results on prevailing benchmark datasets illustrate the promising performance and generalization capability of LaCon under various conditions and settings. Ablation studies analyze different components in LaCon, illustrating its great potential to offer flexible condition controls for different backbones.