Xiujun Li

Dr. Xiujun Li

orcid

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2019
Verified
Conference Paper
Open access
PDF file
XLXiujun Li
Xiujun Li
CLChunyuan Li
Chunyuan Li
QXQiaolin Xia
Qiaolin Xia
YBYonatan Bisk
Yonatan Bisk
ACAsli Celikyilmaz
Asli Celikyilmaz
YCYejin Choi
Yejin Choi
Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah A. Smith, Yejin Choi. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

5 September 2019 in arXiv
Verified
Preprint
Open access
PDF file
XLXiujun Li
Xiujun Li
CLChunyuan Li
Chunyuan Li
QXQiaolin Xia
Qiaolin Xia
YBYonatan Bisk
Yonatan Bisk
ACAsli Celikyilmaz
Asli Celikyilmaz
YCYejin Choi
Yejin Choi
Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments. In this paper, we report two simple but highly effective methods to address these challenges and lead to a new state-of-the-art performance. First, we adapt large-scale pretrained language models to learn text representations that generalize better to previously unseen instructions. Second, we propose a stochastic sampling scheme to reduce the considerable gap between the expert actions in training and sampled actions in test, so that the agent can learn to correct its own mistakes during long sequential action decoding. Combining the two techniques, we achieve a new state of the art on the Room-to-Room benchmark with 6% absolute gain over the previous best result (47% -> 53%) on the Success Rate weighted by Path Length metric.

9 March 2020 in arXiv
Verified
Preprint
Open access
PDF file
QXQiaolin Xia
Qiaolin Xia
XLXiujun Li
Xiujun Li
CLChunyuan Li
Chunyuan Li
YBYonatan Bisk
Yonatan Bisk
ZSZhifang Sui
Zhifang Sui
NSNoah A. Smith
Noah A. Smith
Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and under-specified. In this paper, we present a novel training paradigm, Learn from EveryOne (LEO), which leverages multiple instructions (as different views) for the same trajectory to resolve language ambiguity and improve generalization. By sharing parameters across instructions, our approach learns more effectively from limited training data and generalizes better in unseen environments. On the recent Room-to-Room (R2R) benchmark dataset, LEO achieves 16% improvement (absolute) over a greedy agent as the base agent (25.3% $\rightarrow$ 41.4%) in Success Rate weighted by Path Length (SPL). Further, LEO is complementary to most existing models for vision-and-language navigation, allowing for easy integration with the existing techniques, leading to LEO+, which creates the new state of the art, pushing the R2R benchmark to 62% (9% absolute improvement).

2 April 2019 in arXiv
Verified
Preprint
Open access
PDF file
LKLiyiming Ke
Liyiming Ke
XLXiujun Li
Xiujun Li
YBYonatan Bisk
Yonatan Bisk
AHAri Holtzman
Ari Holtzman
ZGZhe Gan
Zhe Gan
SSSiddhartha Srinivasa
Siddhartha Srinivasa
We present the Frontier Aware Search with backTracking (FAST) Navigator, a general framework for action decoding, that achieves state-of-the-art results on the Room-to-Room (R2R) Vision-and-Language navigation challenge of Anderson et. al. (2018). Given a natural language instruction and photo-realistic image views of a previously unseen environment, the agent was tasked with navigating from source to target location as quickly as possible. While all current approaches make local action decisions or score entire trajectories using beam search, ours balances local and global signals when exploring an unobserved environment. Importantly, this lets us act greedily but use global signals to backtrack when necessary. Applying FAST framework to existing state-of-the-art models achieved a 17% relative gain, an absolute 6% gain on Success rate weighted by Path Length (SPL).