29 July 2024 in IEEE Access
Image colorization, a pivotal aspect of computer vision, employs advanced algorithms to transform grayscale images into realistic colors. This task is inherently challenging due to the need to balance colorfulness and fidelity while preserving local spatial structures and eliminating ghosting effect. To address these issues, Our research introduces a novel pipeline leveraging Stable Diffusion for image colorization, guided by color hint points or textual descriptions. Compared to current text-to-image model, Our key contributions include multi-modal input flexibility, a trainable pixel-level encoder and a controllable feature modulation block. The multi-modal input flexibility allows for the simultaneous use of grayscale images with color hint points and textual descriptions, facilitating the generation of colorized outputs with greater precision and alignment with user instructions. The trainable pixel-level encoder extracts multi-scale features from input images, guiding the diffusion process to capture generative diffusion prior for image colorization, thereby achieving better consistency between the input and output images. Additionally, the controllable feature modulation block is introduced to strike a balance between colorfulness and precision through an adjustable coefficient α. By integrating Stable Diffusion with these innovative guidance advancements, our model overcomes previous limitations and showcases the potential of advanced generative models to produce highly realistic and contextually appropriate colorized images, significantly impacting applications such as historical restoration and contemporary creative processes.
16 May 2025 in Neurocomputing
13 November 2025 in IEEE Transactions on Broadcasting
Although anime super-resolution (SR) has garnered increasing interest in recent years, many existing approaches still inherit design principles from photorealistic imagery, overlooking the unique visual traits of anime. In this work, we revisit the anime SR task from an anime-specific perspective and propose a unified solution tailored for real-world restoration. First, we present A4K, the first 4K-resolution dataset specifically curated for anime SR, built via a dual-criterion selection pipeline combining perceptual quality and structural complexity, resulting in cleaner and more informative training samples. Second, we introduce AniFusionNet, a hybrid CNN–Transformer architecture that dynamically fuses local convolutional features with global self-attention, effectively balancing fine detail reconstruction and global coherence. Finally, we introduce a targeted ground truth (GT) enhancement strategy that selectively strengthens hand-drawn line structures, enabling more accurate learning of anime-specific textures. Extensive experiments on public benchmarks demonstrate that our approach achieves state-of-the-art performance in edge sharpness, color consistency, and artifact suppression. The project is available at https://github.com/wasai67/Anime4K
3 February 2026 in Multimedia Systems