Floodlight Open was a global, open-access, digital-only study designed to understand the drivers and barriers in deployment and use of a smartphone app in a naturalistic setting and broad study population of people with and without multiple sclerosis (MS). The study utilised the Floodlight Open app: a ‘bring-your-own-device’ solution that remotely measures a user’s mood, cognition, hand motor function, and gait and postural stability via smartphone sensor-based tests requiring active user input (‘active tests’). Levels of mobility of study participants (‘life-space measurement’) were passively measured. Study data from these tests were made available via an open-access platform. Data from 1350 participants with self-declared MS and 1133 participants with self-declared non-MS from 17 countries across four continents were included in this report. Overall, MS participants provided active test data for a mean duration of 5.6 weeks or a mean duration of 19 non-consecutive days. This duration increased among MS participants who persisted beyond the first week to a mean of 10.3 weeks or 36.5 non-consecutive days. Passively collected life-space measurement data were generated by MS participants for a mean duration of 9.8 weeks or 50.6 non-consecutive days. This duration increased to 16.3 weeks/85.1 non-consecutive days among MS participants who persisted beyond the first week. Older age, self-declared MS disease status, and clinical supervision as part of concomitant clinical research were all significantly associated with higher persistence of the use of the Floodlight Open app. MS participants performed significantly worse than non-MS participants on four out of seven active tests. The findings from this multinational study inform future research to improve the dynamics of persistence of use of digital monitoring tools and further highlight challenges and opportunities in applying them to support MS clinical care.

Background: The Floodlight Open app is a digital health technology tool (DHTT) that comprises remote, smartphone sensor–based tests (daily activities) for assessing symptoms of multiple sclerosis (MS). User acquisition, engagement, and retention remain a barrier to successfully deploying such tools. Objective: This study aims to quantitatively and qualitatively investigate key user experience (UX) factors associated with the Floodlight Open app. Methods: Floodlight Open is a global, open-access, digital-only study designed to understand the drivers and barriers in deploying a DHTT in a naturalistic setting without supervision and onboarding by a clinician. Daily activities included tests assessing cognition (Information Processing Speed and Information Processing Speed Digit–Digit), hand-motor function (Pinching Test and Draw a Shape Test), and postural stability and gait (Static Balance Test, U-Turn Test, and Two-Minute Walk Test [2MWT]). All daily activities except the 2MWT were taken in a fixed sequence. Qualitative UX was studied through semistructured interviews in a substudy of US participants with MS. The quantitative UX analysis investigated the impact of new UX design features on user engagement and retention in US participants for 3 separate test series: all daily activities included in the fixed sequence (DA), all daily activities included in the fixed sequence except the Static Balance Test and U-Turn Test (DAx), and the 2MWT. Results: The qualitative UX substudy (N=22) revealed the need for 2 new UX design features: a more seamless user journey during the activation process that eliminates the requirement of switching back and forth between the app and the email that the participants received upon registration, and configurable reminders and push notifications to help plan and remind the participants to complete their daily activities. Both UX design features were assessed in the quantitative UX analysis. Introducing the more seamless user journey (original user journey: n=608; more seamless user journey: n=481) improved the conversion rate of participants who enrolled in the study and proceeded to successfully activate the app from 53.9% (328/608) to 74.6% (359/481). Introducing reminders and push notifications (with reminders and notifications: n=350; without reminders and notifications: n=172) improved continuous usage time (proportion of participants with ≥3 consecutive days of usage: DA and DAx: ~30% vs ~12%; 2MWT: ~30% vs ~20%); test completion rates (maximum number of test series completed: DA: 279 vs 64; DAx: 283 vs 126; 2MWT: 302 vs 76); and user retention rates (at day 30: DA: 53/172, 30.8% vs 34/350, 9.7%; DAx: 53/172, 30.8% vs 60/350, 17.1%; 2MWT: 39/172, 22.6% vs 22/350, 6.2%). Inactivity times remained comparable. Conclusions: The remote assessment of MS with DHTTs is a relatively nascent but growing field of research. The continued assessment and improvement of UX design features can play a crucial role in the successful long-term adoption of new DHTTs.

Background: Floodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study population. Objective: This study aims to assess measurement equivalence properties of the Floodlight Open app across operating system (OS) platforms, OS versions, and smartphone device models. Methods: Floodlight Open enrolled adult participants with and without self-declared multiple sclerosis (MS). The study used the Floodlight Open app, a “bring-your-own-device” (BYOD) solution that remotely measured MS-related functional ability via smartphone sensor–based active tests. Measurement equivalence was assessed in all evaluable participants by comparing the performance on the 6 active tests (ie, tests requiring active input from the user) included in the app across OS platforms (iOS vs Android), OS versions (iOS versions 11-15 and separately Android versions 8-10; comparing each OS version with the other OS versions pooled together), and device models (comparing each device model with all remaining device models pooled together). The tests in scope were Information Processing Speed, Information Processing Speed Digit-Digit (measuring reaction speed), Pinching Test (PT), Static Balance Test, U-Turn Test, and 2-Minute Walk Test. Group differences were assessed by permutation test for the mean difference after adjusting for age, sex, and self-declared MS disease status. Results: Overall, 1976 participants using 206 different device models were included in the analysis. Differences in test performance between subgroups were very small or small, with percent differences generally being ≤5% on the Information Processing Speed, Information Processing Speed Digit-Digit, U-Turn Test, and 2-Minute Walk Test; <20% on the PT; and <30% on the Static Balance Test. No statistically significant differences were observed between OS platforms other than on the PT (P<.001). Similarly, differences across iOS or Android versions were nonsignificant after correcting for multiple comparisons using false discovery rate correction (all adjusted P>.05). Comparing the different device models revealed a statistically significant difference only on the PT for 4 out of 17 models (adjusted P≤.001-.03). Conclusions: Consistent with the hypothesis that smartphone sensor–based measurements obtained with different devices are equivalent, this study showed no evidence of a systematic lack of measurement equivalence across OS platforms, OS versions, and device models on 6 active tests included in the Floodlight Open app. These results are compatible with the use of smartphone-based tests in a bring-your-own-device setting, but more formal tests of equivalence would be needed.

In this work, we propose a Bluetooth low energy (BLE) beacon-based algorithm to enable remote measurement of the social behavior of the participants of an observational Autism Spectrum Disorder (ASD) clinical trial (NCT03611075). We have developed a mobile application for a smartphone and a smartwatch to collect beacon signals from BLE beacon sensors as well as to store information about the participants’ household rooms. Our goal is to collect beacon information about the time the participants spent in different rooms of their household to infer sociability information. We applied the same technology and setup in an internal experiment with healthy volunteers to evaluate the accuracy of the proposed algorithm in 10 different home setups, and we observed an average accuracy of 97.2%. Moreover, we show that it is feasible for the clinical study participants/caregivers to set up the BLE beacon sensors in their homes without any technical help, with 96% of them setting up the technology on the first day of data collection. Next, we present results from one-week location data from study participants collected through the proposed technology. Finally, we provide a list of good practice guidelines for optimally applying beacon technology for indoor location monitoring. The proposed algorithm enables us to estimate time spent in different rooms of a household that can pave the development of objective sociability features and eventually support decisions regarding drug efficacy in ASD.

Objective. Physiological sensor data (e.g. photoplethysmograph) is important for remotely monitoring patients’ vital signals, but is often affected by measurement noise. Existing feature-based models for signal cleaning can be limited as they might not capture the full signal characteristics. Approach. In this work we present a deep learning framework for sensor signal cleaning based on dilated convolutions which capture the coarse- and fine-grained structure in order to classify whether a signal is noisy or clean. However, since obtaining annotated physiological data is costly and time-consuming we propose an autoencoder-based semi-supervised model which is able to learn a representation of the sensor signal characteristics, also adding an element of interpretability. Main results. Our proposed models are over 8% more accurate than existing feature-based approaches with half the false positive/negative rates. Finally, we show that with careful tuning (that can be improved further), the semi-supervised model outperforms supervised approaches suggesting that incorporating the large amounts of available unlabeled data can be advantageous for achieving high accuracy (over 90%) and minimizing the false positive/negative rates. Significance. Our approach enables us to reliably separate clean from noisy physiological sensor signal that can pave the development of reliable features and eventually support decisions regarding drug efficacy in clinical trials.

BACKGROUND Floodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study population. OBJECTIVE This study aims to assess measurement equivalence properties of the Floodlight Open app across operating system (OS) platforms, OS versions, and smartphone device models. METHODS Floodlight Open enrolled adult participants with and without self-declared multiple sclerosis (MS). The study used the Floodlight Open app, a “bring-your-own-device” (BYOD) solution that remotely measured MS-related functional ability via smartphone sensor–based active tests. Measurement equivalence was assessed in all evaluable participants by comparing the performance on the 6 active tests (ie, tests requiring active input from the user) included in the app across OS platforms (iOS vs Android), OS versions (iOS versions 11-15 and separately Android versions 8-10; comparing each OS version with the other OS versions pooled together), and device models (comparing each device model with all remaining device models pooled together). The tests in scope were Information Processing Speed, Information Processing Speed Digit-Digit (measuring reaction speed), Pinching Test (PT), Static Balance Test, U-Turn Test, and 2-Minute Walk Test. Group differences were assessed by permutation test for the mean difference after adjusting for age, sex, and self-declared MS disease status. RESULTS Overall, 1976 participants using 206 different device models were included in the analysis. Differences in test performance between subgroups were very small or small, with percent differences generally being ≤5% on the Information Processing Speed, Information Processing Speed Digit-Digit, U-Turn Test, and 2-Minute Walk Test; <20% on the PT; and <30% on the Static Balance Test. No statistically significant differences were observed between OS platforms other than on the PT (P<.001). Similarly, differences across iOS or Android versions were nonsignificant after correcting for multiple comparisons using false discovery rate correction (all adjusted P>.05). Comparing the different device models revealed a statistically significant difference only on the PT for 4 out of 17 models (adjusted P≤.001-.03). CONCLUSIONS Consistent with the hypothesis that smartphone sensor–based measurements obtained with different devices are equivalent, this study showed no evidence of a systematic lack of measurement equivalence across OS platforms, OS versions, and device models on 6 active tests included in the Floodlight Open app. These results are compatible with the use of smartphone-based tests in a bring-your-own-device setting, but more formal tests of equivalence would be needed.