Chaorui Deng

Chaorui Deng

Auto-generated profile, based on publicly available information

Chaorui Deng

School of Computer Science, The University of Adelaide, Adelaide, SA, Australia

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 December 2020 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
YQYanyuan Qiao
Yanyuan Qiao
CDChaorui Deng
Chaorui Deng
QWQi Wu
Qi Wu
Referring expression comprehension (REC) aims to localize a target object in an image described by a referring expression phrased in natural language. Different from the object detection task that queried object labels have been pre-defined, the REC problem only can observe the queries during the test. It is more challenging than a conventional computer vision problem. This task has attracted a lot of attention from both computer vision and natural language processing community, and several lines of work have been proposed, from CNN-RNN model, modular network to complex graph-based model. In this survey, we first examine the state-of-the-art by comparing modern approaches to the problem. We classify methods by their mechanism to encode the visual and textual modalities. In particular, we examine the common approach of joint embedding images and expressions to a common feature space. We also discuss modular architectures and graph-based models that interface with structured graph representation. In the second part of this survey, we review the datasets available for training and evaluating REC approaches. We then group results according to the datasets, backbone models, settings so that they can be fairly compared. Finally, we discuss promising future directions for this field, in particular the compositional referring expression comprehension that requires more reasoning steps to address.

21 September 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
CDChaorui Deng
Chaorui Deng
QWQi Wu
Qi Wu
Qingyao WuQingyao Wu
Qingyao Wu
FHFuyuan Hu
Fuyuan Hu
FLFan Lyu
Fan Lyu
MTMingkui Tan
Mingkui Tan
Visual Grounding (VG) aims to locate the most relevant object or region in an image, based on a natural language query. In real-world VG applications, however, we usually have to deal with ambiguous queries and images with complicated scene structures. Identifying the target based on highly redundant and correlated information can be very challenging, leading to unsatisfactory performance. To tackle this, in this paper, we exploit an attention module for each kind of information to reduce the internal redundancies. We then propose the Accumulated Attention mechanism to reason among all the attention modules jointly, thus the correlations among different kinds of information can be explicitly captured. Moreover, to improve the performance and robustness of our VG models, we introduce some noises into the training procedure to bridge the distribution gap between the human-labeled training data and the real-world poor quality data. With this ``noised'' training strategy, we further learn a bounding box regressor, which can be used to refine the bounding box of the target object. We evaluate the proposed methods on four benchmark datasets. The experimental results show that our methods significantly outperform all previous works on every dataset in terms of both speed and accuracy.

6 November 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
NDNing Ding
Ning Ding
CDChaorui Deng
Chaorui Deng
MTMingkui Tan
Mingkui Tan
QDQing Du
Qing Du
ZGZhiwei Ge
Zhiwei Ge
QWQi Wu
Qi Wu
Image captioning is a core challenge in computer vision, attracting significant attention. Traditional methods prioritize caption quality, often overlooking style control. Our research enhances method controllability, enabling descriptions of varying detail. By integrating a length level embedding into current models, they can produce detailed or concise captions, increasing diversity. We introduce a length-level reranking transformer to correlate image and text complexity, optimizing caption length for informativeness without redundancy. Additionally, with caption length increase, computational complexity grows due to the autoregressive (AR) design of existing methods. To address this, our non-autoregressive (NAR) model maintains constant complexity regardless of caption length. We've developed a training approach that includes refinement sequence training and sequence-level knowledge distillation to close the performance gap between NAR and AR models. In testing, our models set new standards for caption quality on the MS COCO dataset and offer enhanced controllability and diversity. Our NAR model excels over AR models in these aspects and shows greater efficiency with longer captions. With advanced training techniques, our NAR's caption quality rivals that of leading AR models.