The accurate assessment of sleep is critical to better understand and evaluate its role in health and disease. The boom in wearable technology is part of the digital health revolution and is producing many novel, highly sophisticated and relatively inexpensive consumer devices collecting data from multiple sensors and claiming to extract information about users’ behaviors, including sleep. These devices are now able to capture different biosignals for determining, for example, HR and its variability, skin conductance, and temperature, in addition to activity. They perform 24/7, generating overwhelmingly large data sets (big data), with the potential of offering an unprecedented window on users’ health. Unfortunately, little guidance exists within and outside the scientific sleep community for their use, leading to confusion and controversy about their validity and application. The current state-of-the-art review aims to highlight use, validation and utility of consumer wearable sleep-trackers in clinical practice and research. Guidelines for a standardized assessment of device performance is deemed necessary, and several critical factors (proprietary algorithms, device malfunction, firmware updates) need to be considered before using these devices in clinical and sleep research protocols. Ultimately, wearable sleep technology holds promise for advancing understanding of sleep health; however, a careful path forward needs to be navigated, understanding the benefits and pitfalls of this technology as applied in sleep research and clinical sleep medicine.

The last 20 yrs have seen a marked increase in studies utilizing actigraphy in free-living environments. The aim of the present study is to directly compare two commercially available actigraph devices with concurrent polysomnography (PSG) during a daytime nap in healthy young adults. Thirty healthy young adults, ages 18–31 (mean 20.77 yrs, SD 3.14 yrs) simultaneously wore AW-64 and GT3X+ devices during a polysomnographically recorded nap. Mann-Whitney U (M-U) test, intraclass correlation coefficients, and Bland-Altman statistic were used to compare total sleep time (TST), sleep onset latency (SOL), wake after sleep onset (WASO), and sleep efficiency (SE) between the two actigraphs and PSG. Epoch-by-epoch (EBE) agreement was calculated to determine accuracy, sensitivity, specificity, predictive values for sleep (PVS) and wake (PVW), and kappa and prevalence- and bias-adjusted kappa (PABAK) coefficients. All frequency settings provided by the devices were examined. For both actigraphs, EBE analysis found accuracy, sensitivity, specificity, PVS, and PVW comparable to previous reports of other similar devices. Kappa and PABAK coefficients showed moderate to high agreement with PSG depending on device settings. The GT3X+ overestimated TST and SE, and underestimated SOL and WASO, whereas no significant difference was found between AW-64 and PSG. However, GT3X+ showed overall better EBE agreements to PSG than AW-64. We conclude that both actigraphs are valid and reliable devices for detecting sleep/wake diurnal patterns. The choice between devices should be based on several parameters as reliability, cost of the device, scoring algorithm, target population, experimental condition, and aims of the study (e.g., sleep and/or physical activity). (Author correspondence: smednick@ucr.edu)

Sleep-tracking devices, particularly within the consumer sleep technology (CST) space, are increasingly used in both research and clinical settings, providing new opportunities for large-scale data collection in highly ecological conditions. Due to the fast pace of the CST industry combined with the lack of a standardized framework to evaluate the performance of sleep trackers, their accuracy and reliability in measuring sleep remains largely unknown. Here, we provide a step-by-step analytical framework for evaluating the performance of sleep trackers (including standard actigraphy), as compared with gold-standard polysomnography (PSG) or other reference methods. The analytical guidelines are based on recent recommendations for evaluating and using CST from our group and others (de Zambotti and colleagues; Depner and colleagues), and include raw data organization as well as critical analytical procedures, including discrepancy analysis, Bland–Altman plots, and epoch-by-epoch analysis. Analytical steps are accompanied by open-source R functions (depicted at https://sri-human-sleep.github.io/sleep-trackers-performance/AnalyticalPipeline_v1.0.0.html). In addition, an empirical sample dataset is used to describe and discuss the main outcomes of the proposed pipeline. The guidelines and the accompanying functions are aimed at standardizing the testing of CSTs performance, to not only increase the replicability of validation studies, but also to provide ready-to-use tools to researchers and clinicians. All in all, this work can help to increase the efficiency, interpretation, and quality of validation studies, and to improve the informed adoption of CST in research and clinical settings.