Zilong Wang

Zilong Wang

Auto-generated profile, based on publicly available information

Zilong Wang

University of California San Diego

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024
Verified
Conference Paper
Open access
PDF file
LZLi Zhong
Li Zhong
ZWZilong Wang
Zilong Wang
JSJingbo Shang
Jingbo Shang
Li Zhong, Zilong Wang, Jingbo Shang. Findings of the Association for Computational Linguistics ACL 2024. 2024.

24 March 2024 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
LZLi Zhong
Li Zhong
ZWZilong Wang
Zilong Wang
Recently, large language models (LLMs) have shown an extraordinary ability to understand natural language and generate programming code. It has been a common practice for software engineers to consult LLMs when encountering coding questions. Although efforts have been made to avoid syntax errors and align the code with the intended semantics, the reliability, and robustness of the code generation from LLMs have not yet been thoroughly studied. The executable code is not equivalent to reliable and robust code, especially in the context of real-world software development. For example, the misuse of APIs in the generated code could lead to severe problems, such as resource leaks, program crashes, etc. Existing code evaluation benchmarks and datasets focus on crafting small tasks such as programming questions in coding interviews, which, however, deviates from the problem that developers would ask LLM for real-world coding help. To fill the missing piece, in this work, we propose a dataset RobustAPI for evaluating the reliability and robustness of code generated by LLMs. We collect 1208 coding questions from Stack Overflow on 18 representative Java APIs. We summarize the common misuse patterns of these APIs and evaluate them on current popular LLMs. The evaluation results show that even for GPT-4, 62% of the generated code contains API misuses, which would cause unexpected consequences if the code is introduced into real-world software.

1 January 2020
Verified
Conference Paper
Open access
PDF file
ZWZilong Wang
Zilong Wang
MZMingjie Zhan
Mingjie Zhan
XLXuebo Liu
Xuebo Liu
DLDing Liang
Ding Liang
Zilong Wang, Mingjie Zhan, Xuebo Liu, Ding Liang. Findings of the Association for Computational Linguistics: EMNLP 2020. 2020.

4 August 2023
Verified
Conference Paper
Open access
ZWZilong Wang
Zilong Wang
YZYichao Zhou
Yichao Zhou
WWWei Wei
Wei Wei
CLChen-Yu Lee
Chen-Yu Lee
STSandeep Tata
Sandeep Tata
Understanding visually-rich business documents to extract structured data and automate business workflows has been receiving attention both in academia and industry. Although recent multi-modal language models have achieved impressive results, we find that existing benchmarks do not reflect the complexity of real documents seen in industry. In this work, we identify the desiderata for a more comprehensive benchmark and propose one we call Visually Rich Document Understanding (VRDU). VRDU contains two datasets that represent several challenges: rich schema including diverse data types as well as hierarchical entities, complex templates including tables and multi-column layouts, and diversity of different layouts (templates) within a single document type. We design few-shot and conventional experiment settings along with a carefully designed matching algorithm to evaluate extraction results. We report the performance of strong baselines and offer three observations: (1) generalizing to new document templates is still very challenging, (2) few-shot performance has a lot of headroom, and (3) models struggle with hierarchical fields such as line-items in an invoice. We plan to open source the benchmark and the evaluation toolkit. We hope this helps the community make progress on these challenging tasks in extracting structured data from visually rich documents.

1 January 2022
Verified
Conference Paper
Open access
PDF file
JGJinyu Guo
Jinyu Guo
KSKai Shuang
Kai Shuang
JLJijie Li
Jijie Li
ZWZihan Wang
Zihan Wang
YLYixuan Liu
Yixuan Liu
Jinyu Guo, Kai Shuang, Jijie Li, Zihan Wang, Yixuan Liu. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

1 January 2024
Verified
Conference Paper
Open access
PDF file
LPLetian Peng
Letian Peng
YZYuwei Zhang
Yuwei Zhang
ZWZilong Wang
Zilong Wang
JSJayanth Srinivasa
Jayanth Srinivasa
GLGaowen Liu
Gaowen Liu
JSJingbo Shang
Jingbo Shang
Letian Peng, Yuwei Zhang, Zilong Wang, Jayanth Srinivasa, Gaowen Liu, Zihan Wang, Jingbo Shang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.