Phillip Olla

Dr. Phillip Olla


Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 October 2024 in Information
Verified
Journal Article
Open access
PDF file
POPhillip Olla
Phillip Olla
LELauren Elliott
Lauren Elliott
MAMustafa Abumeeiz
Mustafa Abumeeiz
KMKaren Mihelich
Karen Mihelich
JOJoshua Olson
Joshua Olson
This study investigates the integration of generative AI in higher education and the development of the SPARRO framework, a structured approach to improving human–AI interaction in academic settings. This ethnographic study explores the integration of generative AI in healthcare and nursing education, detailing the development of the SPARRO framework based on observations of student and faculty interactions with AI tools across five courses. The study identifies key challenges such as AI hallucination, mistrust of AI-generated summaries, and the difficulty in formulating effective prompts. The SPARRO framework addresses these challenges, offering a step-by-step guide for planning, prompt design, reviewing, and refining AI outputs. While the framework shows promise in improving AI integration, future research is needed to validate its applicability across other academic disciplines and assess its long-term impact on critical thinking and academic integrity. This study contributes to the growing body of research on AI in education, offering practical solutions for ethically and effectively integrating AI tools in academic settings.

31 July 2025 in Nursing Reports
Verified
Journal Article
Open access
PDF file
POPhillip Olla
Phillip Olla
NWNadine K Wodwaski
Nadine K Wodwaski
TLTaylor K Long
Taylor K Long
Background/Objectives: The integration of AI chatbots in nursing education, particularly in simulation-based learning, is advancing rapidly. However, there is a lack of structured evaluation models, especially to assess AI-generated simulations. This article introduces the AI-Integrated Method for Simulation (AIMS) evaluation framework, a dual-phase evaluation framework adapted from the FAITA model, designed to evaluate both prompt design and chatbot performance in the context of nursing education. Methods: This simulation-based study explored the application of an AI chatbot in an emergency planning course. The AIMS framework was developed and applied, consisting of six prompt-level domains (Phase 1) and eight performance criteria (Phase 2). These domains were selected based on current best practices in instructional design, simulation fidelity, and emerging AI evaluation literature. To assess the chatbots educational utility, the study employed a scoring rubric for each phase and incorporated a structured feedback loop to refine both prompt design and chatbox interaction. To demonstrate the framework’s practical application, the researchers configured an AI tool referred to in this study as “Eval-Bot v1”, built using OpenAI’s GPT-4.0, to apply Phase 1 scoring criteria to a real simulation prompt. Insights from this analysis were then used to anticipate Phase 2 performance and identify areas for improvement. Participants (three individuals)—all experienced healthcare educators and advanced practice nurses with expertise in clinical decision-making and simulation-based teaching—reviewed the prompt and Eval-Bot’s score to triangulate findings. Results: Simulated evaluations revealed clear strengths in the prompt alignment with course objectives and its capacity to foster interactive learning. Participants noted that the AI chatbot supported engagement and maintained appropriate pacing, particularly in scenarios involving emergency planning decision-making. However, challenges emerged in areas related to personalization and inclusivity. While the chatbot responded consistently to general queries, it struggled to adapt tone, complexity and content to reflect diverse learner needs or cultural nuances. To support replication and refinement, a sample scoring rubric and simulation prompt template are provided. When evaluated using the Eval-Bot tool, moderate concerns were flagged regarding safety prompts and inclusive language, particularly in how the chatbot navigated sensitive decision points. These gaps were linked to predicted performance issues in Phase 2 domains such as dialog control, equity, and user reassurance. Based on these findings, revised prompt strategies were developed to improve contextual sensitivity, promote inclusivity, and strengthen ethical guidance within chatbot-led simulations. Conclusions: The AIMS evaluation framework provides a practical and replicable approach for evaluating the use of AI chatbots in simulation-based education. By offering structured criteria for both prompt design and chatbot performance, the model supports instructional designers, simulation specialists, and developers in identifying areas of strength and improvement. The findings underscore the importance of intentional design, safety monitoring, and inclusive language when integrating AI into nursing and health education. As AI tools become more embedded in learning environments, this framework offers a thoughtful starting point for ensuring they are applied ethically, effectively, and with learner diversity in mind.

10 June 2025 in Computers
Verified
Journal Article
Open access
PDF file
POPhillip Olla
Phillip Olla
ABAshlee Barnes
Ashlee Barnes
LELauren Elliott
Lauren Elliott
MAMustafa Abumeeiz
Mustafa Abumeeiz
VOVenus Olle
Venus Olle
JTJoseph Tan
Joseph Tan
Rising mental health challenges among postsecondary students have increased the demand for scalable, ethical solutions. This paper presents the design, development, and safety evaluation of Luna, a GPT-4-based mental health chatbot. Built using a modular PHP architecture, Luna integrates multi-layered prompt engineering, safety guardrails, and referral logic. The Institutional Review Board (IRB) at the University of Detroit Mercy (Protocol #23-24-38) reviewed the proposed study and deferred full human subject approval, requesting technical validation prior to deployment. In response, we conducted a pilot test with a variety of users—including clinicians and students who simulated at-risk student scenarios. Results indicated that 96% of expert interactions were deemed safe, and 90.4% of prompts were considered useful. This paper describes Luna’s architecture, prompt strategy, and expert feedback, concluding with recommendations for future human research trials.

6 August 2024 in Preprints.org
Verified
Preprint
Open access
PDF file
POPhillip Olla
Phillip Olla
LELauren Elliott
Lauren Elliott
MAMustafa Abumeeiz
Mustafa Abumeeiz
The rapid integration of Generative AI technologies in academic and research environments marks the beginning of a new digital era, characterized by enhanced efficiency and innovative capabilities. This technological advancement, however, brings forth significant challenges and ethical dilemmas. One of the primary concerns is the potential diminishment of authentic human intellect in scholarly works, fueled by the capabilities of AI to generate ambiguous, fabricated, or even biased and inappropriate content. These issues highlight the critical need for developing comprehensive frameworks and establishing a dedicated field of study to govern the use of AI-generated content in academic and professional contexts. This autoethnography explores the impact of generative AI on educational practices, detailing the development of the SPARRO framework in healthcare and nursing classes. Through personal reflections, interviews, and observations, it addresses specific challenges, offering insights into integrating GenAI ethically and effectively to enhance teaching and learning.

3 January 2024 in Research Square
Verified
Preprint
Open access
PDF file
POPhillip Olla
Phillip Olla
LELauren Elliot
Lauren Elliot
MAMustapha Abumeeiz
Mustapha Abumeeiz
EPElaina Pardalis
Elaina Pardalis
This manuscript meticulously explores the approach for interacting with Artificial Intelligence (AI) Large Language Models (LLMs) to elicit optimal outputs. The generation of high-caliber prompts serves as a pivotal element in achieving the sought-after outcomes from these computational models. The discourse herein delineates various categories of prompts, substantiated with exemplars within each domain of application under investigation. This manuscript highlights the categories of prompts related to the particular utility of each application domain, especially accentuating their relevance to educational stakeholders such as students and educators in medical education. The Application of Learning Domains (ALDs) proposed within this article, endeavor to demarcate areas that may find the most utility from AI LLMs, facilitating knowledge dissemination, practice and training, simulated personas, and augmented interactivity across a spectrum of users in the educational milieu and beyond.

21 May 2025 in Archives of Psychiatric Nursing
Verified
Journal Article
Closed access
POPhillip Olla
Phillip Olla