Background: Mobile health (mHealth) apps offer great opportunities to deliver large-scale, cost-efficient digital solutions for implementing lifestyle changes. Furthermore, many mHealth apps act as medical devices. Yet, there is little research on how to assess user satisfaction with an mHealth solution. Objective: This study presents the development of the mHealth Satisfaction Questionnaire and evaluates its measurement properties. Methods: Respondents who took part in the Health Integrator Study and were randomized to use the Health Integrator smartphone app for lifestyle changes (n=112), with and without additional telephone coaching, rated their satisfaction with the app using the new 14-item mHealth Satisfaction Questionnaire. The ratings were given on a 5-point Likert scale and measurement properties were evaluated using Rasch measurement theory (RMT). Results: Optimal scoring was reached when response options 2, 3, and 4 were collapsed, giving three response categories. After omitting two items that did not fit into the scale, fit residuals were within, or close to, the recommended range of ±2.5. There was no differential item functioning between intervention group, age group, or sex. The Person Separation Index was 0.79, indicating that the scale’s ability to discriminate correctly between person leniency was acceptable for group comparisons but not for individual evaluations. The scale did not meet the criterion of unidimensionality; 16.1% (18/112) of the respondents were outside the desired range of −1.96 to 1.96. In addition, several items showed local dependency and three underlying dimensions emerged: negative experiences, positive experiences, and lifestyle consequences of using the mHealth solution. Conclusions: In times where mHealth apps and digital solutions are given more attention, the mHealth Satisfaction Questionnaire provides a new possibility to measure user satisfaction to ensure usability and improve development of new apps. Our study is one of only a few cases where RMT has been used to evaluate the usability of such an instrument. There is, though, a need for further development of the mHealth Satisfaction Questionnaire, including the addition of more items and consideration of further response options. The mHealth Satisfaction Questionnaire should also be evaluated in a larger sample and with other mHealth apps and in other contexts. Trial Registration: ClinicalTrials.gov NCT03579342; http://clinicaltrials.gov/ct2/show/NCT03579342.
8 February 2018 in Measurement Science and Technology
Is it realistic to aspire to the same kind of quality-assurance of measurement in person-centred care, currently being implemented in healthcare globally, as is established in the physical sciences and engineering? Ensuring metrological comparability (‘traceability’) and reliably declaring measurement uncertainty when assessing patient ability or increased social capital are however challenging for subjective measurements often characterised by large dispersion. Drawing simple analogies between ‘instruments’ in the social sciences—questionnaires, ability tests, etc—and engineering instruments such as thermometers does not go far enough. A possible way forward, apparently equally applicable to both physical and social measurement, seems to be to model inferences in terms of performance metrics of a measurement system. Person-centred care needs person-centred measurement and a full picture of the measurement process when man acts as a measurement instrument is given in the present paper. This complements previous work by presenting the process, step by step, from the observed indication (e.g. probability of success, Psuccess , of achieving a task), through restitution with Rasch measurement theory, to the measurand (e.g. task difficulty). Rasch invariant measure theory can yield quantities—‘latent’ (or ‘explanatory’) variables such as task challenge or person ability—with characteristics akin to those of physical quantities. Metrological references for comparability via traceability and reliable estimates of uncertainty and decision risks are then in reach even for perceptive measurements (and other qualitative properties). As a case study, the person-centred measurement of cognitive ability is examined, as part of the EU project EMPIR 15HLT04 NeuroMet, for Alzheimer’s, where better analysis of correlations with brain atrophy is enabled thanks to the Rasch metrological approach.
Commonly used rating scales and tests have been found lacking reliability and validity, for example in neurodegenerative diseases studies, owing to not making recourse to the inherent ordinality of human responses, nor acknowledging the separability of person ability and item difficulty parameters according to the well-known Rasch model. Here, we adopt an information theory approach, particularly extending deployment of the classic Brillouin entropy expression when explaining the difficulty of recalling non-verbal sequences in memory tests (i.e., Corsi Block Test and Digit Span Test): a more ordered task, of less entropy, will generally be easier to perform. Construct specification equations (CSEs) as a part of a methodological development, with entropy-based variables dominating, are found experimentally to explain (r = 0.98) and predict the construct of task difficulty for short-term memory tests using data from the NeuroMET (n = 88) and Gothenburg MCI (n = 257) studies. We propose entropy-based equivalence criteria, whereby different tasks (in the form of items) from different tests can be combined, enabling new memory tests to be formed by choosing a bespoke selection of items, leading to more efficient testing, improved reliability (reduced uncertainties) and validity. This provides opportunities for more practical and accurate measurement in clinical practice, research and trials.
The justification for making a measurement can be sought in asking what decisions are based on measurement, such as in assessing the compliance of a quality characteristic of an entity in relation to a specification limit, SL. The relative performance of testing devices and classification algorithms used in assessing compliance is often evaluated using the venerable and ever popular receiver operating characteristic (ROC). However, the ROC tool has potentially all the limitations of classic test theory (CTT) such as the non-linearity, effects of ordinality and confounding task difficulty and instrument ability. These limitations, inherent and often unacknowledged when using the ROC tool, are tackled here for the first time with a modernised approach combining measurement system analysis (MSA) and item response theory (IRT), using data from pregnancy testing as an example. The new method of assessing device ability from separate Rasch IRT regressions for each axis of ROC curves is found to perform significantly better, with correlation coefficients with traditional area-under-curve metrics of at least 0.92 which exceeds that of linearised ROC plots, such as Linacre’s, and is recommended to replace other approaches for device assessment. The resulting improved measurement quality of each ROC curve achieved with this original approach should enable more reliable decision-making in conformity assessment in many scenarios, including machine learning, where its use as a metric for assessing classification algorithms has become almost indispensable.
Metrological methods for word learning list tests can be developed with an information theoretical approach extending earlier simple syntax studies. A classic Brillouin entropy expression is applied to the analysis of the Rey’s Auditory Verbal Learning Test RAVLT (immediate recall), where more ordered tasks—with less entropy—are easier to perform. The findings from three case studies are described, including 225 assessments of the NeuroMET2 cohort of persons spanning a cognitive spectrum from healthy older adults to patients with dementia. In the first study, ordinality in the raw scores is compensated for, and item and person attributes are separated with the Rasch model. In the second, the RAVLT IR task difficulty, including serial position effects (SPE), particularly Primacy and Recency, is adequately explained (Pearson’s correlation ) with construct specification equations (CSE). The third study suggests multidimensionality is introduced by SPE, as revealed through goodness-of-fit statistics of the Rasch analyses. Loading factors common to two kinds of principal component analyses (PCA) for CSE formulation and goodness-of-fit logistic regressions are identified. More consistent ways of defining and analysing memory task difficulties, including SPE, can maintain the unique metrological properties of the Rasch model and improve the estimates and understanding of a person’s memory abilities on the path towards better-targeted and more fit-for-purpose diagnostics.
BACKGROUND: Associations between longitudinal changes of plasma biomarkers and cerebral magnetic resonance (MR)‐derived measurements in Alzheimer's disease (AD) remain unclear.METHODS: In a study population (n = 127) of healthy older adults and patients within the AD continuum, we examined associations between longitudinal plasma amyloid beta 42/40 ratio, tau phosphorylated at threonine 181 (p‐tau181), glial fibrillary acidic protein (GFAP), neurofilament light chain (NfL), and 7T structural and functional MR imaging and spectroscopy using linear mixed models.RESULTS: Increases in both p‐tau181 and GFAP showed the strongest associations to 7T MR‐derived measurements, particularly with decreasing parietal cortical thickness, decreasing connectivity of the salience network, and increasing neuroinflammation as determined by MR spectroscopy (MRS) myo‐inositol.DISCUSSION: Both plasma p‐tau181 and GFAP appear to reflect disease progression, as indicated by 7T MR‐derived brain changes which are not limited to areas known to be affected by tau pathology and neuroinflammation measured by MRS myo‐inositol, respectively.Highlights: This study leverages high‐resolution 7T magnetic resonance (MR) imaging and MR spectroscopy (MRS) for Alzheimer's disease (AD) plasma biomarker insights. Tau phosphorylated at threonine 181 (p‐tau181) and glial fibrillary acidic protein (GFAP) showed the largest changes over time, particularly in the AD group. p‐tau181 and GFAP are robust in reflecting 7T MR‐based changes in AD. The strongest associations were for frontal/parietal MR changes and MRS neuroinflammation.