Chun Yang

Dr. Chun Yang


Research Focus & Metrics

Academic Career Length

13Years (2014-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
21 October 2021 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
ZJZan-Xia Jin
Zan-Xia Jin
HWHeran Wu
Heran Wu
CYChun Yang
Chun Yang
FZFang Zhou
Fang Zhou
JQJingyan Qin
Jingyan Qin
XYXu-Cheng Yin
Xu-Cheng Yin
Text-based visual question answering (VQA) requires to read and understand text in an image to correctly answer a given question. However, most current methods simply add optical character recognition (OCR) tokens extracted from the image into the VQA model without considering contextual information of OCR tokens and mining the relationships between OCR tokens and scene objects. In this paper, we propose a novel text-centered method called RUArt (Reading, Understanding and Answering the Related Text) for text-based VQA. Taking an image and a question as input, RUArt first reads the image and obtains text and scene objects. Then, it understands the question, OCRed text and objects in the context of the scene, and further mines the relationships among them. Finally, it answers the related text for the given question through text semantic matching and reasoning. We evaluate our RUArt on two text-based VQA benchmarks (ST-VQA and TextVQA) and conduct extensive ablation studies for exploring the reasons behind RUArt's effectiveness. Experimental results demonstrate that our method can effectively explore the contextual information of the text and mine the stable relationships between the text and objects.

10 July 2024
Verified
Conference Paper
Open access
ZFZhiyu Fang
Zhiyu Fang
SLShuai-Long Lei
Shuai-Long Lei
Xiaobin ZhuXiaobin Zhu
Xiaobin Zhu
CYChun Yang
Chun Yang
SZShi-Xue Zhang
Shi-Xue Zhang
JQJingyan Qin
Jingyan Qin
Temporal Knowledge Graph (TKG) reasoning often involves completing missing factual elements along the timeline. Although existing methods can learn good embeddings for each factual element in quadruples by integrating temporal information, they often fail to infer the evolution of temporal facts. This is mainly because of (1) insufficiently exploring the internal structure and semantic relationships within individual quadruples and (2) inadequately learning a unified representation of the contextual and temporal correlations among different quadruples. To overcome these limitations, we propose a novel Transformer-based reasoning model (dubbed ECEformer) for TKG to learn the Evolutionary Chain of Events (ECE). Specifically, we unfold the neighborhood subgraph of an entity node in chronological order, forming an evolutionary chain of events as the input for our model. Subsequently, we utilize a Transformer encoder to learn the embeddings of intra-quadruples for ECE. We then craft a mixed-context reasoning module based on the multi-layer perceptron (MLP) to learn the unified representations of inter-quadruples for ECE while accomplishing temporal knowledge reasoning. In addition, to enhance the timeliness of the events, we devise an additional time prediction task to complete effective temporal information within the learned unified representation. Extensive experiments on six benchmark datasets verify the state-of-the-art performance and the effectiveness of our method.

21 January 2022 in Machine Intelligence Research
Verified
Journal Article
Open access
PDF file
CYChun Yang
Chun Yang
CLChang Liu
Chang Liu
XYXu-Cheng Yin
Xu-Cheng Yin
Various few-shot image classification methods indicate that transferring knowledge from other sources can improve the accuracy of the classification. However, most of these methods work with one single source or use only closely correlated knowledge sources. In this paper, we propose a novel weakly correlated knowledge integration (WCKI) framework to address these issues. More specifically, we propose a unified knowledge graph (UKG) to integrate knowledge transferred from different sources (i.e., visual domain and textual domain). Moreover, a graph attention module is proposed to sample the subgraph from the UKG with low complexity. To avoid explicitly aligning the visual features to the potentially biased and weakly correlated knowledge space, we sample a task-specific subgraph from UKG and append it as latent variables. Our framework demonstrates significant improvements on multiple few-shot image classification datasets.

1 January 2014 in TransNav, the International Journal on Marine Navigation and Safety of Sea Transportation
Verified
Journal Article
Open access
PDF file
QHQinyou Hu
Qinyou Hu
FCF. Cai
F. Cai
CYChun Yang
Chun Yang
CSChaojian Shi
Chaojian Shi

11 November 2024 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
CLChang Liu
Chang Liu
CYChun Yang
Chun Yang
ZFZhiyu Fang
Zhiyu Fang
HQHai-Bo Qin
Hai-Bo Qin
XYXu-Cheng Yin
Xu-Cheng Yin
The open-set text recognition task is a generalized form of the (close-set) text recognition task, where the model is further challenged to spot and incrementally recognize novel characters in new languages, which also indicate a shift in the language model. In this work, we propose to alleviate the confounding effect such biases under an open-set setup by learning individual character representations out of their context.We propose a Character-First Open-Set Text Recognition framework that treats individual characters as first-class citizens by cotraining the feature extractor with two context-free learning tasks. First, a Context Isolation Learning task is proposed to wipe the context for each character, utilizing a character mask learned in a weak supervision manner. Second, the framework adopts an Individual Character Learning task, which is a single-character classification task with synthetic samples. Our framework can reliably spot unseen characters in Japanese with F1-score over 64%, and can adapt to recognize unseen characters in Japanese, Korean, Greek, and other languages without retraining. The framework also shows decent many-shot performance on close-set text benchmarks with 91.5% line accuracy on IIIT5k and a speed of over 69 FPS single-batched, making it a feasible universal lightweight OCR solution.