Yiyi Huang

Yiyi Huang

Auto-generated profile, based on publicly available information

Yiyi Huang

University of California, San Diego, San Diego, California, USA

Research Focus & Metrics

Representative Publications

View All Publications

25 April 2025
Verified
Conference Paper
Open access
ZZZhongyi Zhou
Zhongyi Zhou
JJJing Jin
Jing Jin
VPVrushank Phadnis
Vrushank Phadnis
XYXiuxiu Yuan
Xiuxiu Yuan
JJJun Jiang
Jun Jiang
RDRuofei Du
Ruofei Du

11 May 2024
Verified
Conference Paper
Open access
ZZZhongyi Zhou
Zhongyi Zhou
JJJing Jin
Jing Jin
VPVrushank Phadnis
Vrushank Phadnis
XYXiuxiu Yuan
Xiuxiu Yuan
JJJun Jiang
Jun Jiang
RDRuofei Du
Ruofei Du
Foundational multi-modal models have democratized AI access, yet the construction of complex, customizable machine learning pipelines by novice users remains a grand challenge. This paper demonstrates a visual programming system that allows novices to rapidly prototype multimodal AI pipelines. We first conducted a formative study with 58 contributors and collected 236 proposals of multimodal AI pipelines that served various practical needs. We then distilled our findings into a design matrix of primitive nodes for prototyping multimodal AI visual programming pipelines, and implemented a system with 65 nodes. To support users’ rapid prototyping experience, we built InstructPipe, an AI assistant based on large language models (LLMs) that allows users to generate a pipeline by writing text-based instructions. We believe InstructPipe enhances novice users onboarding experience of visual programming and the controllability of LLMs by offering non-experts a platform to easily update the generation.