Bayu Aryoyudanta

Bayu Aryoyudanta

Auto-generated profile, based on publicly available information

Bayu Aryoyudanta


Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
31 July 2024 in JMIR Formative Research
Verified
Journal Article
Open access
BPBambang Parmanto
Bambang Parmanto
BABayu Aryoyudanta
Bayu Aryoyudanta
TSTimothius Wilbert Soekinto
Timothius Wilbert Soekinto
ISI Made Agus Setiawan
I Made Agus Setiawan
YWYuhan Wang
Yuhan Wang
YCYong Kyung Choi
Yong Kyung Choi
Background: In the United States, 1 in 5 adults currently serves as a family caregiver for an individual with a serious illness or disability. Unlike professional caregivers, family caregivers often assume this role without formal preparation or training. Thus, there is an urgent need to enhance the capacity of family caregivers to provide quality care. Leveraging technology as an educational tool or an adjunct to care is a promising approach that has the potential to enhance the learning and caregiving capabilities of family caregivers. Large language models (LLMs) can potentially be used as a foundation technology for supporting caregivers. An LLM can be categorized as a foundation model (FM), which is a large-scale model trained on a broad data set that can be adapted to a range of different domain tasks. Despite their potential, FMs have the critical weakness of “hallucination,” where the models generate information that can be misleading or inaccurate. Information reliability is essential when language models are deployed as front-line help tools for caregivers. Objective: This study aimed to (1) develop a reliable caregiving language model (CaLM) by using FMs and a caregiving knowledge base, (2) develop an accessible CaLM using a small FM that requires fewer computing resources, and (3) evaluate the model’s performance compared with a large FM. Methods: We developed a CaLM using the retrieval augmented generation (RAG) framework combined with FM fine-tuning for improving the quality of FM answers by grounding the model on a caregiving knowledge base. The key components of the CaLM are the caregiving knowledge base, a fine-tuned FM, and a retriever module. We used 2 small FMs as candidates for the foundation of the CaLM (LLaMA [large language model Meta AI] 2 and Falcon with 7 billion parameters) and adopted a large FM (GPT-3.5 with an estimated 175 billion parameters) as a benchmark. We developed the caregiving knowledge base by gathering various types of documents from the internet. We focused on caregivers of individuals with Alzheimer disease and related dementias. We evaluated the models’ performances using the benchmark metrics commonly used in evaluating language models and their reliability for providing accurate references with their answers. Results: The RAG framework improved the performance of all FMs used in this study across all measures. As expected, the large FM performed better than the small FMs across all metrics. Interestingly, the small fine-tuned FMs with RAG performed significantly better than GPT 3.5 across all metrics. The fine-tuned LLaMA 2 with a small FM performed better than GPT 3.5 (even with RAG) in returning references with the answers. Conclusions: The study shows that a reliable and accessible CaLM can be developed using small FMs with a knowledge base specific to the caregiving domain.

15 June 2026 in JMIR Preprints
Verified
Preprint
Open access
MYMaria Yuliana
Maria Yuliana
BABayu Aryoyudanta
Bayu Aryoyudanta
TSTimothius Wilbert Soekinto
Timothius Wilbert Soekinto
ISI Made Agus Setiawan
I Made Agus Setiawan
YCYong Kyung Choi
Yong Kyung Choi
BPBambang Parmanto
Bambang Parmanto
BACKGROUND Timely, accurate, and understandable health information is critical for individuals with spinal cord injury (SCI) during rehabilitation, yet shortened stays, staff shortages, and information complexity leave many patients and caregivers underprepared at discharge. Large Language Model (LLM) chatbots show promise for delivering accessible patient education in SCI rehabilitation and typically augmented with domain-specific knowledge bases (KBs) to enhance accuracy and response quality, yet how KB source selection influences these systems remains understudied. OBJECTIVE This study investigates how different knowledge sources—curated website articles, journal articles, and their combination, as well as a no-KB baseline—affect the response quality of an LLM-driven SCI patient education chatbot. By identifying which configurations produce responses that are both clinically accurate and genuinely actionable, this study aims to guide KB development that support patient self-management and reducing preventable complications. METHODS Using a Retrieval-Augmented Generation (RAG) framework, we developed chatbots with GPT-4o, Llama 3.3-70B, and Claude 3.5 Sonnet across four KB configurations and evaluated their responses against a curated clinical question set. Response quality was assessed across five dimensions: semantic similarity to clinician-provided answers, understandability, actionability, factuality, and comprehensiveness. RESULTS The curated website-based KB achieved the highest performance overall, provided the highest semantic similarity, understandability and actionability (score = 73.29%, κ = 0.459, AC1 = 0.652), and zero non-factual responses. The vanilla model performed comparably in semantic similarity but produced one non-factual response and lower inter-rater agreement (score = 72.56%, κ = 0.325, AC1 = 0.553), indicating that relying solely on the internal model knowledge yields less reliable outputs. The combined KB showed no improvement over the curated websites KB or vanilla model from merging both sources (score = 72.46%, κ = 0.496, AC1 = 0.665). Finally, the journal article–based KB generated the most comprehensive responses but scored lowest overall (score = 65.48%, κ = 0.449, AC1 = 0.546), likely due to technical language and insufficient step-by-step guidance, and also produced one non-factual response. CONCLUSIONS Knowledge source selection substantially shapes the response quality of LLM-driven chatbot for SCI patient education. Among the configuration tested, a KB constructed from curated websites delivered the most balanced and reliable performance across five dimensions.

7 February 2026 in JMIR Preprints
Verified
Preprint
Open access
BABayu Aryoyudanta
Bayu Aryoyudanta
MYMaria Yuliana
Maria Yuliana
ISI Made Agus Setiawan
I Made Agus Setiawan
ASAndi Saptono
Andi Saptono
BDBrad Dicianno
Brad Dicianno
BPBambang Parmanto
Bambang Parmanto
BACKGROUND Spinal Cord Injury (SCI) is a life-altering condition that creates a critical need for accurate and accessible patient education materials (PEMs). Retrieval Augmented Generation (RAG) with Large Language Models (LLMs) holds promise for answering consumer health questions (CHQs) using existing PEMs. However, readability remains a concern. Our preliminary findings indicate that RAG-generated answers frequently exceed the commonly recommended 8th-grade readability level, thereby widening accessibility gaps for patients and family caregivers seeking SCI-related consumer health information (CHI). OBJECTIVE This study aimed to evaluate strategies for enhancing the readability of RAG-generated answers to SCI-related CHQs with a focus on comparing a self-reflection approach to existing prompt-based approaches. METHODS A curated corpus of 172 SCI-related PEMs was collected and categorized into three groups (FAQs, self-management guidelines, and insurance/legal documents), forming the foundation of a RAG knowledge base. From these PEMs, synthetic CHQs were generated and answered using a RAG framework across multiple LLMs, including OpenAI GPT 4o, Claude Sonnet 3.5, Google Gemini 1.5 Pro, Meta Llama 3.3 70B Instruct and OpenAI GPT OSS 20B. The generated answers were rewritten using four strategies: (1) few-shot prompting, (2) explicit readability prompt, (3) post-generation rewriting, and (4) self-reflection and refinement (READ). Readability was assessed using the Flesch-Kincaid Grade Level (FKGL) and semantic similarity was evaluated with embedding-based cosine similarity. RESULTS Baseline RAG answers consistently exceeded recommended readability levels, scoring higher than the PEMs from which they were derived. Few-shot prompting yielded only minimal gain and no significant improvement in readability. Both explicit readability prompting and post-generation rewriting produced significant improvement yet still failed to consistently achieve the 8th-grade readability target. In contrast, the READ approach significantly improved readability across all tested LLMs, with nearly all outputs written at or below the 8th-grade level. However, improvements in readability were accompanied by decline in semantic similarity, highlighting a potential trade-off between accessibility and semantic similarity. CONCLUSIONS The READ approach provides an effective and robust framework for improving the readability of RAG-generated answers to SCI-related CHQs, outperforming prompt-based strategies and demonstrating strong potential to enhance the accessibility of patient-facing health information. Future studies should incorporate expert evaluation and real-world testing to ensure clinical accuracy, usability, and successful integration into clinical settings.

16 November 2023 in JMIR Preprints
Verified
Preprint
Open access
BPBambang Parmanto
Bambang Parmanto
BABayu Aryoyudanta
Bayu Aryoyudanta
TSTimothius Wilbert Soekinto
Timothius Wilbert Soekinto
ISI Made Agus Setiawan
I Made Agus Setiawan
YWYuhan Wang
Yuhan Wang
YCYong Kyung Choi
Yong Kyung Choi
BACKGROUND In the United States, 1 in 5 adults currently serves as a family caregiver for an individual with a serious illness or disability. Unlike professional caregivers, family caregivers often assume this role without formal preparation or training. Thus, there is an urgent need to enhance the capacity of family caregivers to provide quality care. Leveraging technology as an educational tool or an adjunct to care is a promising approach that has the potential to enhance the learning and caregiving capabilities of family caregivers. Large language models (LLMs) can potentially be used as a foundation technology for supporting caregivers. An LLM can be categorized as a foundation model (FM), which is a large-scale model trained on a broad data set that can be adapted to a range of different domain tasks. Despite their potential, FMs have the critical weakness of “hallucination,” where the models generate information that can be misleading or inaccurate. Information reliability is essential when language models are deployed as front-line help tools for caregivers. OBJECTIVE This study aimed to (1) develop a reliable caregiving language model (CaLM) by using FMs and a caregiving knowledge base, (2) develop an accessible CaLM using a small FM that requires fewer computing resources, and (3) evaluate the model’s performance compared with a large FM. METHODS We developed a CaLM using the retrieval augmented generation (RAG) framework combined with FM fine-tuning for improving the quality of FM answers by grounding the model on a caregiving knowledge base. The key components of the CaLM are the caregiving knowledge base, a fine-tuned FM, and a retriever module. We used 2 small FMs as candidates for the foundation of the CaLM (LLaMA [large language model Meta AI] 2 and Falcon with 7 billion parameters) and adopted a large FM (GPT-3.5 with an estimated 175 billion parameters) as a benchmark. We developed the caregiving knowledge base by gathering various types of documents from the internet. We focused on caregivers of individuals with Alzheimer disease and related dementias. We evaluated the models’ performances using the benchmark metrics commonly used in evaluating language models and their reliability for providing accurate references with their answers. RESULTS The RAG framework improved the performance of all FMs used in this study across all measures. As expected, the large FM performed better than the small FMs across all metrics. Interestingly, the small fine-tuned FMs with RAG performed significantly better than GPT 3.5 across all metrics. The fine-tuned LLaMA 2 with a small FM performed better than GPT 3.5 (even with RAG) in returning references with the answers. CONCLUSIONS The study shows that a reliable and accessible CaLM can be developed using small FMs with a knowledge base specific to the caregiving domain.