Jun Wang

Jun Wang

Auto-generated profile, based on publicly available information

Jun Wang

Paul G. Allen School of Computer Science & Engineering, University of Washington, United States

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 May 2024
Verified
Conference Paper
Open access
JLJaewook Lee
Jaewook Lee
JWJun Wang
Jun Wang
EBElizabeth Brown
Elizabeth Brown
LCLiam Chu
Liam Chu
SRSebastian S. Rodriguez
Sebastian S. Rodriguez
JFJon Froehlich
Jon Froehlich
Voice assistants (VAs) like Siri and Alexa are transforming human-computer interaction; however, they lack awareness of users’ spatiotemporal context, resulting in limited performance and unnatural dialogue. We introduce GazePointAR, a fully-functional context-aware VA for wearable augmented reality that leverages eye gaze, pointing gestures, and conversation history to disambiguate speech queries. With GazePointAR, users can ask “what’s over there?” or “how do I solve this math problem?” simply by looking and/or pointing. We evaluated GazePointAR in a three-part lab study (N=12): (1) comparing GazePointAR to two commercial systems, (2) examining GazePointAR’s pronoun disambiguation across three tasks; (3) and an open-ended phase where participants could suggest and try their own context-sensitive queries. Participants appreciated the naturalness and human-like nature of pronoun-driven queries, although sometimes pronoun use was counter-intuitive. We then iterated on GazePointAR and conducted a first-person diary study examining how GazePointAR performs in-the-wild. We conclude by enumerating limitations and design considerations for future context-aware VAs.


11 October 2024
Verified
Conference Paper
Open access
JWJun Wang
Jun Wang
CCChun-Cheng Chang
Chun-Cheng Chang
JDJiafei Duan
Jiafei Duan
DFDieter Fox
Dieter Fox
RKRanjay Krishna
Ranjay Krishna
The increasing affordability of robot hardware is accelerating the integration of robots into everyday activities. However, training a robot to automate a task requires expensive trajectory data where a trained human annotator moves a physical robot to train it. Consequently, only those with access to robots produce demonstrations to train robots. In this work, we remove this restriction with EVE, an iOS app that enables everyday users to train robots using intuitive augmented reality visualizations, without needing a physical robot. With EVE, users can collect demonstrations by specifying waypoints with their hands, visually inspecting the environment for obstacles, modifying existing waypoints, and verifying collected trajectories. In a user study (N = 14, D = 30) consisting of three common tabletop tasks, EVE outperformed three state-of-the-art interfaces in success rate and was comparable to kinesthetic teaching—physically moving a physical robot—in completion time, usability, motion intent communication, enjoyment, and preference (meanp = 0.30). EVE allows users to train robots for personalized tasks, such as sorting desk supplies, organizing ingredients, or setting up board games. We conclude by enumerating limitations and design considerations for future AR-based demonstration collection systems for robotics.

29 October 2023
Verified
Conference Paper
Open access
JLJaewook Lee
Jaewook Lee
JWJun Wang
Jun Wang
EBElizabeth Brown
Elizabeth Brown
LCLiam Chu
Liam Chu
SRSebastian S. Rodriguez
Sebastian S. Rodriguez
JFJon Froehlich
Jon Froehlich
Voice assistants (VAs) like Siri and Alexa have transformed how humans interact with technology; however, their inability to consider a user’s spatiotemporal context, such as surrounding objects, drammatically limits natural dialogue. In this demo paper, we introduce GazePointAR, a wearable augmented reality (AR) system that resolves ambiguity in speech queries using eye gaze, pointing gesture, and conversation history. With GazePointAR, a user can ask “what’s over there?” or “how do I solve this math problem?” simply by looking and/or pointing. We describe GazePointAR’s design and highlight supported use cases.