Hao Fang

Hao Fang

Auto-generated profile, based on publicly available information

Hao Fang

Microsoft Semantic Machines

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024
Verified
Conference Paper
Open access
PDF file
BWBoshi Wang
Boshi Wang
HFHao Fang
Hao Fang
JEJason Eisner
Jason Eisner
BDBenjamin Van Durme
Benjamin Van Durme
YSYu Su
Yu Su
Boshi Wang, Hao Fang, Jason Eisner, Benjamin Van Durme, Yu Su. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

13 October 2021 in IEEE Access
Verified
Journal Article
Open access
HFHao Fang
Hao Fang
JLJun-Qing Liu
Jun-Qing Liu
KXKai Xie
Kai Xie
PWPeng Wu
Peng Wu
XZXin-Yu Zhang
Xin-Yu Zhang
JHJian-Biao He
Jian-Biao He
Head pose estimation based on a single image is a challenging endeavor because of the complex background conditions and characteristics of the human face. In this report, we propose a Multi stage Regression-Capsule Network (MR-CapsNet) to predict head posture based on a single image input. In the study, we used the residual attention block and squeeze-and-excitation block to extract features in three levels. CapsNet overcomes the shortcomings of the traditional convolutional neural network and implements module aggregation to describe the spatial relationship of features after aggregation, in addition to realizing a compact and robust model using a multi-stage regression scheme. We tested our method on the AFLW2000 and BIWI datasets obtaining mean absolute errors of 4.26% and 3.95%, respectively. In addition, we discuss the accuracy of our method in the case of eye or mouth occlusion. The results of comprehensive experiments reveal that our method can accurately predict head posture.

1 January 2023
Verified
Conference Paper
Open access
PDF file
KLKevin Lin
Kevin Lin
PXPatrick Xia
Patrick Xia
HFHao Fang
Hao Fang
Kevin Lin, Patrick Xia, Hao Fang. Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings). 2023.

7 March 2024 in arXiv
Verified
Preprint
Open access
PDF file
BWBoshi Wang
Boshi Wang
HFHao Fang
Hao Fang
JEJason Eisner
Jason Eisner
BDBenjamin Van Durme
Benjamin Van Durme
YSYu Su
Yu Su
Tools are essential for large language models (LLMs) to acquire up-to-date information and take consequential actions in external environments. Existing work on tool-augmented LLMs primarily focuses on the broad coverage of tools and the flexibility of adding new tools. However, a critical aspect that has surprisingly been understudied is simply how accurately an LLM uses tools for which it has been trained. We find that existing LLMs, including GPT-4 and open-source LLMs specifically fine-tuned for tool use, only reach a correctness rate in the range of 30% to 60%, far from reliable use in practice. We propose a biologically inspired method for tool-augmented LLMs, simulated trial and error (STE), that orchestrates three key mechanisms for successful tool use behaviors in the biological system: trial and error, imagination, and memory. Specifically, STE leverages an LLM's 'imagination' to simulate plausible scenarios for using a tool, after which the LLM interacts with the tool to learn from its execution feedback. Both short-term and long-term memory are employed to improve the depth and breadth of the exploration, respectively. Comprehensive experiments on ToolBench show that STE substantially improves tool learning for LLMs under both in-context learning and fine-tuning settings, bringing a boost of 46.7% to Mistral-Instruct-7B and enabling it to outperform GPT-4. We also show effective continual learning of tools via a simple experience replay strategy.

18 September 2023 in arXiv
Verified
Preprint
Open access
PDF file
KLKevin Lin
Kevin Lin
PXPatrick Xia
Patrick Xia
HFHao Fang
Hao Fang
We evaluate the ability of semantic parsers based on large language models (LLMs) to handle contextual utterances. In real-world settings, there typically exists only a limited number of annotated contextual utterances due to annotation cost, resulting in an imbalance compared to non-contextual utterances. Therefore, parsers must adapt to contextual utterances with a few training examples. We examine four major paradigms for doing so in conversational semantic parsing i.e., Parse-with-Utterance-History, Parse-with-Reference-Program, Parse-then-Resolve, and Rewrite-then-Parse. To facilitate such cross-paradigm comparisons, we construct SMCalFlow-EventQueries, a subset of contextual examples from SMCalFlow with additional annotations. Experiments with in-context learning and fine-tuning suggest that Rewrite-then-Parse is the most promising paradigm when holistically considering parsing accuracy, annotation cost, and error types.