Shanxin Yuan

Dr. Shanxin Yuan


Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 March 2024 in The Visual Computer
Verified
Journal Article
Open access
PDF file
HSHenry Senior
Henry Senior
GSGreg Slabaugh
Greg Slabaugh
SYShanxin Yuan
Shanxin Yuan
LRLuca Rossi
Luca Rossi
2D image understanding is a complex problem within computer vision, but it holds the key to providing human-level scene comprehension. It goes further than identifying the objects in an image, and instead, it attempts to understand the scene. Solutions to this problem form the underpinning of a range of tasks, including image captioning, visual question answering (VQA), and image retrieval. Graphs provide a natural way to represent the relational arrangement between objects in an image, and thus, in recent years graph neural networks (GNNs) have become a standard component of many 2D image understanding pipelines, becoming a core architectural component, especially in the VQA group of tasks. In this survey, we review this rapidly evolving field and we provide a taxonomy of graph types used in 2D image understanding approaches, a comprehensive list of the GNN models used in this domain, and a roadmap of future potential developments. To the best of our knowledge, this is the first comprehensive survey that covers image captioning, visual question answering, and image retrieval techniques that focus on using GNNs as the main part of their architecture.

26 June 2023 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
RCRufeng Chen
Rufeng Chen
BZBolun Zheng
Bolun Zheng
HZHua Zhang
Hua Zhang
QCQuan Chen
Quan Chen
CYChenggang Yan
Chenggang Yan
SYShanxin Yuan
Shanxin Yuan
Reconstructing a High Dynamic Range (HDR) image from several Low Dynamic Range (LDR) images with different exposures is a challenging task, especially in the presence of camera and object motion. Though existing models using convolutional neural networks (CNNs) have made great progress, challenges still exist, e.g., ghosting artifacts. Transformers, originating from the field of natural language processing, have shown success in computer vision tasks, due to their ability to address a large receptive field even within a single layer. In this paper, we propose a transformer model for HDR imaging. Our pipeline includes three steps: alignment, fusion, and reconstruction. The key component is the HDR transformer module. Through experiments and ablation studies, we demonstrate that our model outperforms the state-of-the-art by large margins on several popular public datasets.

26 February 2024 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
LLLin Liu
Lin Liu
JAJunfeng An
Junfeng An
SYShanxin Yuan
Shanxin Yuan
WZWengang Zhou
Wengang Zhou
HLHouqiang Li
Houqiang Li
QTQi Tian
Qi Tian
Demoiréing is the task of removing moiré patterns, which are commonly caused by the interference between the screen and digital cameras. Although research on single image demoiréing has made great progress, research on video demoiréing has received less attention from the community. Video demoiréing poses a new set of challenges. First, most existing video restoration algorithms rely on multi-resolution pixel-based alignment, which can cause damage to the details of the predicted results. Second, these algorithms are based on flow-based loss or relation-based loss, making it difficult to handle the large motions of adjacent frames while keeping temporal consistency intact. To address these challenges, we present a novel deep learning-based approach called the Deep Temporal Color Embedding network (DTCENet) that employs an invertible network to align distortion color patches in a patch-based embedding framework. DTCENet can well preserve details while eliminate color distortions. Furthermore, we introduce a video-image invertible loss function to effectively handle the color inconsistent problem of adjacent frames. Our approach shows promising results in demoiréing videos, with improved performance over existing state-of-the-art algorithms. Our method gets about 10% improvements in terms of LPIPS and 10.3% improvements in terms of FID compared with the recent SOTA methods.

9 November 2025
Verified
Book Chapter
Closed access
PDF file
HSHenry Senior
Henry Senior
LRLuca Rossi
Luca Rossi
GSGreg Slabaugh
Greg Slabaugh
SYShanxin Yuan
Shanxin Yuan

27 October 2025
Verified
Conference Paper
Open access
QFQifan Fu
Qifan Fu
XCXu Chen
Xu Chen
MAMuhammad Asad
Muhammad Asad
SYShanxin Yuan
Shanxin Yuan
COChangjae Oh
Changjae Oh
GSGreg Slabaugh
Greg Slabaugh

11 March 2025 in arXiv
Verified
Preprint
Open access
PDF file
HSHenry Senior
Henry Senior
LRLuca Rossi
Luca Rossi
GSGreg Slabaugh
Greg Slabaugh
SYShanxin Yuan
Shanxin Yuan
It has been a longstanding goal within image captioning to move beyond a dependence on object detection. We investigate using superpixels coupled with Vision Language Models (VLMs) to bridge the gap between detector-based captioning architectures and those that solely pretrain on large datasets. Our novel superpixel approach ensures that the model receives object-like features whilst the use of VLMs provides our model with open set object understanding. Furthermore, we extend our architecture to make use of multi-resolution inputs, allowing our model to view images in different levels of detail, and use an attention mechanism to determine which parts are most relevant to the caption. We demonstrate our model's performance with multiple VLMs and through a range of ablations detailing the impact of different architectural choices. Our full model achieves a competitive CIDEr score of $136.9$ on the COCO Karpathy split.