Zhe Gan

Dr. Zhe Gan


Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 October 2022
Verified
Book Chapter
Closed access
PDF file
ZYZhengyuan Yang
Zhengyuan Yang
ZGZhe Gan
Zhe Gan
JWJianfeng Wang
Jianfeng Wang
XHXiaowei Hu
Xiaowei Hu
FAFaisal Ahmed
Faisal Ahmed
LWLijuan Wang
Lijuan Wang

1 January 2019
Verified
Conference Paper
Open access
PDF file
MJMing Jiang
Ming Jiang
QHQiuyuan Huang
Qiuyuan Huang
LZLei Zhang
Lei Zhang
XWXin Wang
Xin Wang
PZPengchuan Zhang
Pengchuan Zhang
JGJianfeng Gao
Jianfeng Gao
Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, Jianfeng Gao. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

23 April 2020 in arXiv
Verified
Preprint
Open access
PDF file
CZChen Zhu
Chen Zhu
YCYu Cheng
Yu Cheng
ZGZhe Gan
Zhe Gan
SSSiqi Sun
Siqi Sun
TGTom Goldstein
Tom Goldstein
JLJingjing Liu
Jingjing Liu
Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models. In this work, we propose a novel adversarial training algorithm, FreeLB, that promotes higher invariance in the embedding space, by adding adversarial perturbations to word embeddings and minimizing the resultant adversarial risk inside different regions around input samples. To validate the effectiveness of the proposed approach, we apply it to Transformer-based models for natural language understanding and commonsense reasoning tasks. Experiments on the GLUE benchmark show that when applied only to the finetuning stage, it is able to improve the overall test scores of BERT-base model from 78.3 to 79.4, and RoBERTa-large model from 88.5 to 88.8. In addition, the proposed approach achieves state-of-the-art single-model test accuracies of 85.44\% and 67.75\% on ARC-Easy and ARC-Challenge. Experiments on CommonsenseQA benchmark further demonstrate that FreeLB can be generalized and boost the performance of RoBERTa-large model on other tasks as well. Code is available at \url{https://github.com/zhuchen03/FreeLB .

18 July 2020 in arXiv
Verified
Preprint
Open access
PDF file
YCYen-Chun Chen
Yen-Chun Chen
LLLinjie Li
Linjie Li
LYLicheng Yu
Licheng Yu
AKAhmed El Kholy
Ahmed El Kholy
FAFaisal Ahmed
Faisal Ahmed
JLJingjing Liu
Jingjing Liu
Joint image-text embedding is the bedrock for most Vision-and-Language (V+L) tasks, where multimodality inputs are simultaneously processed for joint visual and textual understanding. In this paper, we introduce UNITER, a UNiversal Image-TExt Representation, learned through large-scale pre-training over four image-text datasets (COCO, Visual Genome, Conceptual Captions, and SBU Captions), which can power heterogeneous downstream V+L tasks with joint multimodal embeddings. We design four pre-training tasks: Masked Language Modeling (MLM), Masked Region Modeling (MRM, with three variants), Image-Text Matching (ITM), and Word-Region Alignment (WRA). Different from previous work that applies joint random masking to both modalities, we use conditional masking on pre-training tasks (i.e., masked language/region modeling is conditioned on full observation of image/text). In addition to ITM for global image-text alignment, we also propose WRA via the use of Optimal Transport (OT) to explicitly encourage fine-grained alignment between words and image regions during pre-training. Comprehensive analysis shows that both conditional masking and OT-based WRA contribute to better pre-training. We also conduct a thorough ablation study to find an optimal combination of pre-training tasks. Extensive experiments show that UNITER achieves new state of the art across six V+L tasks (over nine datasets), including Visual Question Answering, Image-Text Retrieval, Referring Expression Comprehension, Visual Commonsense Reasoning, Visual Entailment, and NLVR$^2$. Code is available at https://github.com/ChenRocks/UNITER.

28 November 2017 in arXiv
Verified
Preprint
Open access
PDF file
TXTao Xu
Tao Xu
PZPengchuan Zhang
Pengchuan Zhang
QHQiuyuan Huang
Qiuyuan Huang
HZHan Zhang
Han Zhang
ZGZhe Gan
Zhe Gan
XHXiaodong He
Xiaodong He
In this paper, we propose an Attentional Generative Adversarial Network (AttnGAN) that allows attention-driven, multi-stage refinement for fine-grained text-to-image generation. With a novel attentional generative network, the AttnGAN can synthesize fine-grained details at different subregions of the image by paying attentions to the relevant words in the natural language description. In addition, a deep attentional multimodal similarity model is proposed to compute a fine-grained image-text matching loss for training the generator. The proposed AttnGAN significantly outperforms the previous state of the art, boosting the best reported inception score by 14.14% on the CUB dataset and 170.25% on the more challenging COCO dataset. A detailed analysis is also performed by visualizing the attention layers of the AttnGAN. It for the first time shows that the layered attentional GAN is able to automatically select the condition at the word level for generating different parts of the image.

18 July 2020 in arXiv
Verified
Preprint
Open access
PDF file
YCYen-Chun Chen
Yen-Chun Chen
ZGZhe Gan
Zhe Gan
YCYu Cheng
Yu Cheng
JLJingzhou Liu
Jingzhou Liu
JLJingjing Liu
Jingjing Liu
Large-scale pre-trained language model such as BERT has achieved great success in language understanding tasks. However, it remains an open question how to utilize BERT for language generation. In this paper, we present a novel approach, Conditional Masked Language Modeling (C-MLM), to enable the finetuning of BERT on target generation tasks. The finetuned BERT (teacher) is exploited as extra supervision to improve conventional Seq2Seq models (student) for better text generation performance. By leveraging BERT's idiosyncratic bidirectional nature, distilling knowledge learned in BERT can encourage auto-regressive Seq2Seq models to plan ahead, imposing global sequence-level supervision for coherent text generation. Experiments show that the proposed approach significantly outperforms strong Transformer baselines on multiple language generation tasks such as machine translation and text summarization. Our proposed model also achieves new state of the art on IWSLT German-English and English-Vietnamese MT datasets. Code is available at https://github.com/ChenRocks/Distill-BERT-Textgen.