Erin McGowan

Ms. Erin McGowan

Rutgers University-New Brunswick
orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 January 2022 in Sensors
Verified
Journal Article
Open access
PDF file
EMErin McGowan
Erin McGowan
VGVidita Gawade
Vidita Gawade
WGWeihong "Grace" Guo
Weihong "Grace" Guo
Physics-informed machine learning is emerging through vast methodologies and in various applications. This paper discovers physics-based custom loss functions as an implementable solution to additive manufacturing (AM). Specifically, laser metal deposition (LMD) is an AM process where a laser beam melts deposited powder, and the dissolved particles fuse to produce metal components. Porosity, or small cavities that form in this printed structure, is generally considered one of the most destructive defects in metal AM. Traditionally, computer tomography scans measure porosity. While this is useful for understanding the nature of pore formation and its characteristics, purely physics-driven models lack real-time prediction ability. Meanwhile, a purely deep learning approach to porosity prediction leaves valuable physics knowledge behind. In this paper, a hybrid model that uses both empirical and simulated LMD data is created to show how various physics-informed loss functions impact the accuracy, precision, and recall of a baseline deep learning model for porosity prediction. In particular, some versions of the physics-informed model can improve the precision of the baseline deep learning-only model (albeit at the expense of overall accuracy).

2 November 2023 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
SCSonia Castelo
Sonia Castelo
JRJoao Rulff
Joao Rulff
EMErin McGowan
Erin McGowan
BSBea Steers
Bea Steers
GWGuande Wu
Guande Wu
CSClaudio Silva
Claudio Silva
The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi-year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine-tune the components of the AR assistant.

9 September 2024 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
SCSonia Castelo
Sonia Castelo
JRJoao Rulff
Joao Rulff
PSParikshit Solunke
Parikshit Solunke
EMErin McGowan
Erin McGowan
GWGuande Wu
Guande Wu
CSClaudio Silva
Claudio Silva
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach through an in-depth examination of a think-aloud experiment with five domain experts.

10 March 2025 in IEEE Computer Graphics and Applications
Verified
Journal Article
Closed access
EMErin McGowan
Erin McGowan
JRJoao Rulff
Joao Rulff
SCSonia Castelo
Sonia Castelo
GWGuande Wu
Guande Wu
SCShaoyu Chen
Shaoyu Chen
CSClaudio Silva
Claudio Silva
The concept of an AI assistant for task guidance is rapidly shifting from a science fiction staple to an impending reality. Such a system is inherently complex, requiring models for perceptual grounding, attention, and reasoning, an intuitive interface that adapts to the performer's needs, and the orchestration of data streams from many sensors. Moreover, all data acquired by the system must be readily available for post-hoc analysis to enable developers to understand performer behavior and quickly detect failures. We introduce TIM, the first end-to-end AI-enabled task guidance system in augmented reality which is capable of detecting both the user and scene as well as providing adaptable, just-in-time feedback. We discuss the system challenges and propose design solutions. We also demonstrate how TIM adapts to domain applications with varying needs, highlighting how the system components can be customized for each scenario.

7 June 2026
Verified
Conference Paper
Open access
GWGuande Wu
Guande Wu
DTDishita Turakhia
Dishita Turakhia
EWEden Wu
Eden Wu
SQSonia Castelo Quispe
Sonia Castelo Quispe
JRJoão Rulff
João Rulff
CSClaudio Silva
Claudio Silva
The growing use of head-mounted AR devices has produced a surge of egocentric recordings of procedural tasks—step-by-step activities such as medical procedures—creating rich opportunities for performance analysis. Yet current analyses are time-consuming, requiring substantial manual inspection to identify events, actions, and objects from videos, with little automated support for surfacing insights or errors. We introduce InsightAR, a visual analytics tool that integrates event detection, action recognition, and multi-modal language models to create video summaries capturing key errors and insights. Its interface further enables users to refine generated insights and errors collaboratively. A user study (N=16) found that InsightAR identifies significantly more errors and marginally significant insight numbers than the baseline. Our work demonstrated the feasibility and potential benefits of future human-AI collaborative video analysis.

17 July 2024 in arXiv
Verified
Preprint
Open access
PDF file
SCSonia Castelo
Sonia Castelo
JRJoao Rulff
Joao Rulff
PSParikshit Solunke
Parikshit Solunke
EMErin McGowan
Erin McGowan
GWGuande Wu
Guande Wu
CSClaudio Silva
Claudio Silva
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach by conducting an in-depth examination of a think-aloud experiment with five domain experts.