Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: After an mHealth app is created, one important step is to evaluate the usability of the app before it is released to the public. There are multiple ways of conducting a usability study, one of which is collecting target users’ feedback with a usability questionnaire. Different groups have used different questionnaires for mHealth app usability evaluation; commonly used questionnaires are the System Usability Scale (SUS) and Post-Study System Usability Questionnaire (PSSUQ); however, SUS and PSSUQ were not designed for evaluating the usability of mHealth apps. Self-written questionnaires are also commonly used for mHealth app usability evaluation but they have not been validated. Objective: The goal of this project was to develop and validate a new mobile health app usability questionnaire. Methods: An mHealth app usability questionnaire (MAUQ) was designed by the research team based on a number of existing questionnaires used in previous mobile app usability studies, especially the well-validated questionnaires. MAUQ, SUS, and PSSUQ were then all used to evaluate the usability of two mHealth apps: one an interactive mHealth app and the other a standalone mHealth app. The reliability and validity of the new questionnaire were evaluated. The correlation coefficients among MAUQ and SUS, PSSUQ were calculated as well. Results: In this study, 128 study participants provided responses to the questionnaire statements in this study. Psychometric analysis indicated that the MAUQ has three subscales and their internal consistency reliability is high. The relevant subscales correlated well with the subscales of the PSSUQ. The overall scale also strongly correlated with the PSSUQ and SUS. Four versions of the MAUQ were created related to type of app (interactive or standalone) and target user of the app (patient or provider). A website has been created to make it convenient for mHealth app developers to use this new questionnaire to assess the usability of their mHealth apps. Conclusions: The newly created mHealth app usability questionnaire – MAUQ – has the reliability and validity required to assess mHealth app usability.

Persons with chronic conditions and disabilities (PwCCDs) are vulnerable to secondary complications. Many of these secondary complications are preventable with proactive self-management and proper support. To enhance PwCCDs' self-management skills and conveniently receive desired support, we have developed a mobile health (mHealth) system called iMHere. In 2 previous clinical trials, iMHere was successfully used to improve health outcomes of adult participants with spina bifida and spinal cord injury. To further expand use of iMHere among people with various types of disabilities and chronic diseases, the system needs to be more adaptive to address 3 unique challenges: 1) PwCCDs have very diverse needs with regards to self-management support, 2) PwCCDs' self-management needs may change over time, and 3) it is a challenge to keep PwCCDs engaged and interested in long-term self-management. The aim of this study was to develop an adaptive mHealth system capable of supporting long-term self-management and adapting to the various needs and conditions of PwCCDs. A scalable and adaptive architecture was designed and implemented for the new version, iMHere 2.0. In this scalable architecture, a set of mobile app modules was created to provide various types of self-management support to PwCCDs with the ability to add more as needed. The adaptive architecture empowers PwCCDs with personally relevant app modules and allows clinicians to adapt these modules in response to PwCCDs' evolving needs and conditions over time. Persuasive technologies, social support, and personalization features were integrated into iMHere 2.0 to engage and motivate PwCCDs and support long-term usage. Two initial studies were performed to evaluate the usability and feasibility of the iMHere 2.0 system. The iMHere 2.0 system consists of cross-platform client and caregiver apps, a Web-based clinician portal, and a secure 2-way communication protocol for providing interactions among these 3 front-end components, all supported by a back-end server. The client and caregiver apps have 12 adaptive app modules to support various types of self-management tasks. The adaptive architecture makes it possible for PwCCDs to receive personalized app modules relevant to their conditions with or without support from various types of caregivers. The personalization and persuasive technologies in the architecture can be used to engage PwCCDs for long-term usage of the iMHere 2.0 system. Participants of the usability study were satisfied with the iMHere 2.0 client app. The feasibility evaluation revealed several practical issues to consider when implementing the system on a large scale. We developed an adaptive mHealth system as a novel method to support diverse needs in self-management for PwCCDs that can dynamically change over time. The usability of the client app is high, and it was feasible for PwCCDs to use in supporting personalized and evolving self-care needs.

Background: In the United States, 1 in 5 adults currently serves as a family caregiver for an individual with a serious illness or disability. Unlike professional caregivers, family caregivers often assume this role without formal preparation or training. Thus, there is an urgent need to enhance the capacity of family caregivers to provide quality care. Leveraging technology as an educational tool or an adjunct to care is a promising approach that has the potential to enhance the learning and caregiving capabilities of family caregivers. Large language models (LLMs) can potentially be used as a foundation technology for supporting caregivers. An LLM can be categorized as a foundation model (FM), which is a large-scale model trained on a broad data set that can be adapted to a range of different domain tasks. Despite their potential, FMs have the critical weakness of “hallucination,” where the models generate information that can be misleading or inaccurate. Information reliability is essential when language models are deployed as front-line help tools for caregivers. Objective: This study aimed to (1) develop a reliable caregiving language model (CaLM) by using FMs and a caregiving knowledge base, (2) develop an accessible CaLM using a small FM that requires fewer computing resources, and (3) evaluate the model’s performance compared with a large FM. Methods: We developed a CaLM using the retrieval augmented generation (RAG) framework combined with FM fine-tuning for improving the quality of FM answers by grounding the model on a caregiving knowledge base. The key components of the CaLM are the caregiving knowledge base, a fine-tuned FM, and a retriever module. We used 2 small FMs as candidates for the foundation of the CaLM (LLaMA [large language model Meta AI] 2 and Falcon with 7 billion parameters) and adopted a large FM (GPT-3.5 with an estimated 175 billion parameters) as a benchmark. We developed the caregiving knowledge base by gathering various types of documents from the internet. We focused on caregivers of individuals with Alzheimer disease and related dementias. We evaluated the models’ performances using the benchmark metrics commonly used in evaluating language models and their reliability for providing accurate references with their answers. Results: The RAG framework improved the performance of all FMs used in this study across all measures. As expected, the large FM performed better than the small FMs across all metrics. Interestingly, the small fine-tuned FMs with RAG performed significantly better than GPT 3.5 across all metrics. The fine-tuned LLaMA 2 with a small FM performed better than GPT 3.5 (even with RAG) in returning references with the answers. Conclusions: The study shows that a reliable and accessible CaLM can be developed using small FMs with a knowledge base specific to the caregiving domain.

Background: Over the past decade, a large number of mobile health (mHealth) apps have been created to help individuals to better manage their own health. However, very few of these mHealth apps were specifically designed for people with disabilities, and only a few of them have been assessed for accessibility for people with disabilities. As a result, people with disabilities have difficulties using many of these mHealth apps. Objective: The objective of this study was to identify an approach that can be generally applied to improve the accessibility of mHealth apps. Methods: We recruited 5 study participants with a primary diagnosis of cerebral palsy or spinal cord injury. All the participants had fine motor impairment or lack of dexterity, and hence, they had difficulties using some mHealth apps. These 5 study participants were first asked to use multiple modules in the client app of a novel mHealth system (iMHere 2.0), during which their performance was observed. Interviews were conducted post use to collect study participants’ desired accessibility features. These accessibility features were then implemented into the iMHere 2.0 client app as customizable options. The 5 participants were asked to use the same modules in the app again, and their performance was compared with that in the first round. A brief interview and a questionnaire were then performed at the end of the study to collect the 5 participants’ comments and impression of the iMHere 2.0 app in general and of the customizable accessibility features. Results: Study results indicate that the study participants on their first use of the iMHere 2.0 client app experienced various levels of difficulty consistent with the severity of their lack of dexterity. Their performance was improved after their desired accessibility features were added into the app, and they liked the customizable accessibility features. These participants also expressed an interest in using this mHealth system for their health self-management tasks. Conclusions: The accessibility features identified in this study improved the accessibility of the mHealth app for people with dexterity issues. Our approach for improving mHealth app accessibility may also be applied to other mHealth apps to make those apps accessible to people with disabilities.

The objective of this project was to measure Veteran and provider satisfaction with a home-based telerehabilitation assessment for wheelchair seating and mobility. Forty-three Veterans were seen remotely at their place of residence by a provider, using a VA Video Connect synchronous videoconferencing system. Veteran and provider satisfaction were collected using the Telerehabilitation Questionnaire (TRQ). Mean individual TRQ scores for both Veterans and providers were significantly higher than the scale midpoint of 3.5. Veterans had higher scores than providers for five individual items on the TRQ. Higher scores by Veterans on the technology and quality and clarity of the video and audio likely correspond to the differences in environmental settings in which the visit occurred for the Veteran compared with the provider. High satisfaction scores with the telerehabilitation assessments are likely attributed to the positive working relationship between the provider and the rehabilitation technician, who provided in-person technical support to the Veteran in the home during the wheeled mobility evaluation. Overall, the results indicate a high level of Veteran and provider satisfaction using telerehabilitation for wheelchair seating and mobility evaluations.