Hao Wang

Dr. Hao Wang

School of Computer Science and Engineering, Nanyang Technological University, Singapore, Central Singapore, Singapore
orcid

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024
Verified
Conference Paper
Open access
PDF file
YLYihuai Lan
Yihuai Lan
ZHZhiqiang Hu
Zhiqiang Hu
LWLei Wang
Lei Wang
YWYang Wang
Yang Wang
DYDeheng Ye
Deheng Ye
HWHao Wang
Hao Wang
Yihuai Lan, Zhiqiang Hu, Lei Wang, Yang Wang, Deheng Ye, Peilin Zhao, Ee-Peng Lim, Hui Xiong, Hao Wang. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

24 May 2021 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
HWHao Wang
Hao Wang
DSDoyen Sahoo
Doyen Sahoo
CLChenghao Liu
Chenghao Liu
KSKe Shu
Ke Shu
PAPalakorn Achananuparp
Palakorn Achananuparp
SHSteven C. H. Hoi
Steven C. H. Hoi
Food retrieval is an important task to perform analysis of food-related information, where we are interested in retrieving relevant information about the queried food item such as ingredients, cooking instructions, etc. In this paper, we investigate cross-modal retrieval between food images and cooking recipes. The goal is to learn an embedding of images and recipes in a common feature space, such that the corresponding image-recipe embeddings lie close to one another. Two major challenges in addressing this problem are 1) large intra-variance and small inter-variance across cross-modal food data; and 2) difficulties in obtaining discriminative recipe representations. To address these two problems, we propose Semantic-Consistent and Attention-based Networks (SCAN), which regularize the embeddings of the two modalities through aligning output semantic probabilities. Besides, we exploit a self-attention mechanism to improve the embedding of recipes. We evaluate the performance of the proposed method on the large-scale Recipe1M dataset, and show that we can outperform several state-of-the-art cross-modal retrieval strategies for food images and cooking recipes by a significant margin.

10 June 2022 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
HWHao Wang
Hao Wang
GLGuosheng Lin
Guosheng Lin
SHSteven C. H. Hoi
Steven C. H. Hoi
CMChunyan Miao
Chunyan Miao
Food is significant to human daily life. In this paper, we are interested in learning structural representations for lengthy recipes, that can benefit the recipe generation and food cross-modal retrieval tasks. Different from the common vision-language data, here the food images contain mixed ingredients and target recipes are lengthy paragraphs, where we do not have annotations on structure information. To address the above limitations, we propose a novel method to unsupervisedly learn the sentence-level tree structures for the cooking recipes. Our approach brings together several novel ideas in a systematic framework: (1) exploiting an unsupervised learning approach to obtain the sentence-level tree structure labels before training; (2) generating trees of target recipes from images with the supervision of tree structure labels learned from (1); and (3) integrating the learned tree structures into the recipe generation and food cross-modal retrieval procedure. Our proposed model can produce good-quality sentence-level tree structures and coherent recipes. We achieve the state-of-the-art recipe generation and food cross-modal retrieval performance on the benchmark Recipe1M dataset.

21 May 2024 in International Journal of Computer Vision
Verified
Journal Article
Closed access
PDF file
HWHao Wang
Hao Wang
GLGuosheng Lin
Guosheng Lin
AMAna García del Molino
Ana García del Molino
AWAnran Wang
Anran Wang
JFJiashi Feng
Jiashi Feng
ZSZHIQI SHEN
ZHIQI SHEN

28 July 2022 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
HWHao Wang
Hao Wang
GLGuosheng Lin
Guosheng Lin
SHSteven C. H. Hoi
Steven C. H. Hoi
CMChunyan Miao
Chunyan Miao
Video captioning targets interpreting the complex visual contents as text descriptions, which requires the model to fully understand video scenes including objects and their interactions. Prevailing methods adopt off-the-shelf object detection networks to give object proposals and use the attention mechanism to model the relations between objects. They often miss some undefined semantic concepts of the pretrained model and fail to identify exact predicate relationships between objects. In this paper, we investigate an open research task of generating text descriptions for the given videos, and propose Cross-Modal Graph (CMG) with meta concepts for video captioning. Specifically, to cover the useful semantic concepts in video captions, we weakly learn the corresponding visual regions for text descriptions, where the associated visual regions and textual words are named cross-modal meta concepts. We further build meta concept graphs dynamically with the learned cross-modal meta concepts. We also construct holistic video-level and local frame-level video graphs with the predicted predicates to model video sequence structures. We validate the efficacy of our proposed techniques with extensive experiments and achieve state-of-the-art results on two public datasets.

1 January 2025
Verified
Conference Paper
Open access
PDF file
HFHonghao Fu
Honghao Fu
JRJunlong Ren
Junlong Ren
QCQi Chai
Qi Chai
DYDeheng Ye
Deheng Ye
YCYujun Cai
Yujun Cai
HWHao Wang
Hao Wang
Honghao Fu, Junlong Ren, Qi Chai, Deheng Ye, Yujun Cai, Hao Wang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.