Peiqi Wang

Peiqi Wang

Auto-generated profile, based on publicly available information

Peiqi Wang

MIT Department of Electrical Engineering and Computer Science 77 Massachusetts Ave 02139 Cambridge Long Beach MA CA USA

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 August 2023 in arXiv
Verified
Preprint
Open access
PDF file
ZGZhen Guo
Zhen Guo
PWPeiqi Wang
Peiqi Wang
YWYanwei Wang
Yanwei Wang
SYShangdi Yu
Shangdi Yu
Large Language Models (LLMs) have made remarkable advancements in the field of natural language processing. However, their increasing size poses challenges in terms of computational cost. On the other hand, Small Language Models (SLMs) are known for their efficiency, but they often struggle with limited capacity and training data, especially in specific domains. In this paper, we introduce a novel method aimed at improving SLMs in the medical domain using LLM-based generative data augmentation. The objective of our approach is to develop more efficient and capable models that are specifically tailored for specialized applications. Through experiments conducted on the PubMedQA dataset, we demonstrate the effectiveness of LLMs in refining and diversifying existing question-answer pairs. This refinement process leads to improved performance in a significantly smaller model after fine-tuning. Notably, our best SLM, with under 1.6 billion parameters, outperforms the few-shot GPT-4 on the PubMedQA dataset. Our code and generated data are publicly available to facilitate further explorations.

8 June 2023
Verified
Book Chapter
Closed access
PDF file
PWPeiqi Wang
Peiqi Wang
WWWilliam M. Wells
William M. Wells
SBSeth Berkowitz
Seth Berkowitz
SHSteven Horng
Steven Horng
PGPolina Golland
Polina Golland

1 January 2025
Verified
Conference Paper
Open access
PDF file
PWPeiqi Wang
Peiqi Wang
SPShengYun Peng
ShengYun Peng
XZXuewen Zhang
Xuewen Zhang
HYHanchao Yu
Hanchao Yu
YYYibo Yang
Yibo Yang
QWQifan Wang
Qifan Wang
Peiqi Wang, ShengYun Peng, Xuewen Zhang, Hanchao Yu, Yibo Yang, Lifu Huang, Fujun Liu, Qifan Wang. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

4 February 2024 in arXiv
Verified
Preprint
Open access
PDF file
PWPeiqi Wang
Peiqi Wang
YSYikang Shen
Yikang Shen
ZGZhen Guo
Zhen Guo
MSMatthew Stallone
Matthew Stallone
YKYoon Kim
Yoon Kim
RPRameswar Panda
Rameswar Panda
We aim to select data subsets for the fine-tuning of large language models to more effectively follow instructions. Prior work has emphasized the importance of diversity in dataset curation but relied on heuristics such as the number of tasks. In this paper, we use determinantal point processes to capture the diversity and quality of instruction tuning datasets for subset selection. We propose to measure dataset diversity with log determinant distance that is the distance between the dataset of interest and a maximally diverse reference dataset. Our experiments demonstrate that the proposed diversity measure in the normalized weight gradient space is correlated with downstream instruction-following performance. Consequently, it can be used to inform when data selection is the most helpful and to analyze dataset curation strategies. We demonstrate the utility of our approach on various instruction tuning datasets.

9 March 2023 in arXiv
Verified
Preprint
Open access
PDF file
PWPeiqi Wang
Peiqi Wang
WWWilliam M. Wells
William M. Wells
SBSeth Berkowitz
Seth Berkowitz
SHSteven Horng
Steven Horng
PGPolina Golland
Polina Golland
Image-text multimodal representation learning aligns data across modalities and enables important medical applications, e.g., image classification, visual grounding, and cross-modal retrieval. In this work, we establish a connection between multimodal representation learning and multiple instance learning. Based on this connection, we propose a generic framework for constructing permutation-invariant score functions with many existing multimodal representation learning approaches as special cases. Furthermore, we use the framework to derive a novel contrastive learning approach and demonstrate that our method achieves state-of-the-art results in several downstream tasks.

24 May 2025 in arXiv
Verified
Preprint
Open access
PDF file
PWPeiqi Wang
Peiqi Wang
SPShengYun Peng
ShengYun Peng
XZXuewen Zhang
Xuewen Zhang
HYHanchao Yu
Hanchao Yu
YYYibo Yang
Yibo Yang
QWQifan Wang
Qifan Wang
This work investigates the optimal allocation of inference compute across three key scaling factors in video vision language models: language model size, frame count, and the number of visual tokens per frame. While prior works typically focuses on optimizing model efficiency or improving performance without considering resource constraints, we instead identify optimal model configuration under fixed inference compute budgets. We conduct large-scale training sweeps and careful parametric modeling of task performance to identify the inference compute-optimal frontier. Our experiments reveal how task performance depends on scaling factors and finetuning data size, as well as how changes in data size shift the compute-optimal frontier. These findings translate to practical tips for selecting these scaling factors.