Jiahao Nick Li

Jiahao Nick Li

Auto-generated profile, based on publicly available information

Jiahao Nick Li

University of California, Los Angeles, Los Angeles, California, USA

Research Focus & Metrics

Representative Publications

View All Publications

11 May 2024
Verified
Conference Paper
Open access
XLXingyu Bruce Liu
Xingyu Bruce Liu
JLJiahao Nick Li
Jiahao Nick Li
DKDavid Kim
David Kim
XCXiang 'Anthony' Chen
Xiang 'Anthony' Chen
RDRuofei Du
Ruofei Du
Situationally Induced Impairments and Disabilities (SIIDs) can significantly hinder user experience in contexts such as poor lighting, noise, and multi-tasking. While prior research has introduced algorithms and systems to address these impairments, they predominantly cater to specific tasks or environments and fail to accommodate the diverse and dynamic nature of SIIDs. We introduce Human I/O, a unified approach to detecting a wide range of SIIDs by gauging the availability of human input/output channels. Leveraging egocentric vision, multimodal sensing and reasoning with large language models, Human I/O achieves a 0.22 mean absolute error and a 82% accuracy in availability prediction across 60 in-the-wild egocentric video recordings in 32 different scenarios. Furthermore, while the core focus of our work is on the detection of SIIDs rather than the creation of adaptive user interfaces, we showcase the efficacy of our prototype via a user study with 10 participants. Findings suggest that Human I/O significantly reduces effort and improves user experience in the presence of SIIDs, paving the way for more adaptive and accessible interactive systems in the future.


13 October 2024
Verified
Conference Paper
Open access
JLJiahao Nick Li
Jiahao Nick Li
ZZZhuohao (Jerry) Zhang
Zhuohao (Jerry) Zhang
JMJiaju Ma
Jiaju Ma
We present OmniQuery, an interactive system that augments users’ personal photo albums and enables free-form question answering on users’ past memories. OmniQuery processes multimodal media data in personal albums, aggregates them into related episodic memory databases in different levels, and infers semantic knowledge including personal facts like social relationships, preferences, and experiences. OmniQuery then allows users to interact with their database using natural language, giving media that directly matches the query or an exact answer supported by related media as a result.