Anna Li Trella

Ms. Anna Li Trella

Harvard University, Cambridge, Maryland, USA
orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 July 2022 in Algorithms
Verified
Journal Article
Open access
PDF file
Anna Li TrellaAnna Li Trella
Anna Li Trella
KZKelly W. Zhang
Kelly W. Zhang
INInbal Nahum-Shani
Inbal Nahum-Shani
VSVivek Shetty
Vivek Shetty
FDFinale Doshi-Velez
Finale Doshi-Velez
SMSusan A. Murphy
Susan A. Murphy
Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To guide how one can tackle these challenges, we extend the PCS (predictability, computability, stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning to the design of RL algorithms for the digital interventions setting. Furthermore, we provide guidelines on how to design simulation environments, a crucial tool for evaluating RL candidate algorithms using the PCS framework. We show how we used the PCS framework to design an RL algorithm for Oralytics, a mobile health study aiming to improve users’ tooth-brushing behaviors through the personalized delivery of intervention messages. Oralytics will go into the field in late 2022.

26 June 2023 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
Anna Li TrellaAnna Li Trella
Anna Li Trella
KZKelly W. Zhang
Kelly W. Zhang
INInbal Nahum-Shani
Inbal Nahum-Shani
VSVivek Shetty
Vivek Shetty
FDFinale Doshi-Velez
Finale Doshi-Velez
SMSusan A. Murphy
Susan A. Murphy
While dental disease is largely preventable, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. One of the main challenges in developing such an algorithm is ensuring that the algorithm considers the impact of current actions on the effectiveness of future actions (i.e., delayed effects), especially when the algorithm has been designed to run stably and autonomously in a constrained, real-world setting characterized by highly noisy, sparse data. We address this challenge by designing a quality reward that maximizes the desired health outcome (i.e., high-quality brushing) while minimizing user burden. We also highlight a procedure for optimizing the hyperparameters of the reward by building a simulation environment test bed and evaluating candidates using the test bed. The RL algorithm discussed in this paper will be deployed in Oralytics. To the best of our knowledge, Oralytics is the first mobile health study utilizing an RL algorithm designed to prevent dental disease by optimizing the delivery of motivational messages supporting oral self-care behaviors.

11 April 2025 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
Anna Li TrellaAnna Li Trella
Anna Li Trella
KZKelly W. Zhang
Kelly W. Zhang
HJHinal Jajal
Hinal Jajal
INInbal Nahum-Shani
Inbal Nahum-Shani
VSVivek Shetty
Vivek Shetty
SMSusan A. Murphy
Susan A. Murphy
Dental disease is a prevalent chronic condition associated with substantial financial burden, personal suffering, and increased risk of systemic diseases. Despite widespread recommendations for twice-daily tooth brushing, adherence to recommended oral self-care behaviors remains sub-optimal due to factors such as forgetfulness and disengagement. To address this, we developed Oralytics, a mHealth intervention system designed to complement clinician-delivered preventative care for marginalized individuals at risk for dental disease. Oralytics incorporates an online reinforcement learning algorithm to determine optimal times to deliver intervention prompts that encourage oral self-care behaviors. We have deployed Oralytics in a registered clinical trial. The deployment required careful design to manage challenges specific to the clinical trials setting in the U.S. In this paper, we (1) highlight key design decisions of the RL algorithm that address these challenges and (2) conduct a re-sampling analysis to evaluate algorithm design decisions. A second phase (randomized control trial) of Oralytics is planned to start in spring 2025.

6 March 2026 in Journal of Medical Internet Research
Verified
Journal Article
Open access
AGAsim Gazi
Asim Gazi
DGDaiqi Gao
Daiqi Gao
SGSusobhan Ghosh
Susobhan Ghosh
ZXZiping Xu
Ziping Xu
Anna Li TrellaAnna Li Trella
Anna Li Trella
SMSusan A Murphy
Susan A Murphy
In the context of digital health, just-in-time adaptive interventions (JITAIs) are nascent precision medicine systems that can extend personalized health care support to everyday life. A challenge in designing JITAIs is that personalized support often involves sophisticated decision-making algorithms. These decision-making algorithms can require numerous nontrivial design decisions that must be made between successive JITAI deployments (eg, hyperparameter selection for an artificial intelligence algorithm). Making design decisions between deployments—rather than during deployment—ensures intervention fidelity and enhances the ability to replicate results. Yet, each deployment can be costly, precluding the use of A/B testing for every design decision. How should design decisions be made strategically between JITAI deployments? This paper introduces “digital twins for just-in-time adaptive interventions (JITAI-Twins)” to address this question. JITAI-Twins are “digital twins of a subpopulation” (term used in the 2023 National Academies workshop proceedings on digital twins). JITAI-Twins are used to virtually simulate the potential outcomes of a JITAI’s design decisions for an upcoming deployment. Based on simulation results, design decisions are made for the deployed JITAI. To continually improve the JITAI, data collected during deployment are used to update the JITAI-Twin—and this bidirectional feedback between deployments and simulation environments continues. JITAI-Twins are thus “fit-for-purpose” (term used in the National Academies 2024 consensus report on digital twins) instantiations of the digital twin concept. In this paper, we elucidate the specifics and design process of JITAI-Twins, with examples of prior use in clinical settings. JITAI-Twins highlight continuity over the course of a JITAI’s optimization and continual improvement, emphasizing the need for bidirectional feedback between versions of a simulation environment and a JITAI’s deployments.

2 June 2022 in Preprints.org
Verified
Preprint
Open access
PDF file
Anna Li TrellaAnna Li Trella
Anna Li Trella
KZKelly W. Zhang
Kelly W. Zhang
INInbal Nahum-Shani
Inbal Nahum-Shani
VSVivek Shetty
Vivek Shetty
FDFinale Doshi-Velez
Finale Doshi-Velez
SMSusan A. Murphy
Susan A. Murphy
Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To guide how one can tackle these challenges, we extend the PCS (Predictability, Computability, Stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning (Yu and Kumbier, 2020), to the design of RL algorithms for the digital interventions setting. Further, we provide guidelines on how to design simulation environments, a crucial tool for evaluating RL candidate algorithms using the PCS framework. We illustrate the use of the PCS framework for designing an RL algorithm for Oralytics, a mobile health study aiming to improve users' tooth-brushing behaviors through the personalized delivery of intervention messages. Oralytics will go into the field in late 2022.

Verified
Journal Article
Closed access
Anna Li TrellaAnna Li Trella
Anna Li Trella
WDWalter Dempsey
Walter Dempsey
AGAsim H. Gazi
Asim H. Gazi
ZXZiping Xu
Ziping Xu
FDFinale Doshi-Velez
Finale Doshi-Velez
SMSusan A. Murphy
Susan A. Murphy