10 September 2018 in IEEE Transactions on Visualization and Computer Graphics
Understanding and accounting for uncertainty is critical to effectively reasoning about visualized data. However, evaluating the impact of an uncertainty visualization is complex due to the difficulties that people have interpreting uncertainty and the challenge of defining correct behavior with uncertainty information. Currently, evaluators of uncertainty visualization must rely on general purpose visualization evaluation frameworks which can be ill-equipped to provide guidance with the unique difficulties of assessing judgments under uncertainty. To help evaluators navigate these complexities, we present a taxonomy for characterizing decisions made in designing an evaluation of an uncertainty visualization. Our taxonomy differentiates six levels of decisions that comprise an uncertainty visualization evaluation: the behavioral targets of the study, expected effects from an uncertainty visualization, evaluation goals, measures, elicitation techniques, and analysis approaches. Applying our taxonomy to 86 user studies of uncertainty visualizations, we find that existing evaluation practice, particularly in visualization research, focuses on Performance and Satisfaction-based measures that assume more predictable and statistically-driven judgment behavior than is suggested by research on human judgment and decision making. We reflect on common themes in evaluation practice concerning the interpretation and semantics of uncertainty, the use of confidence reporting, and a bias toward evaluating performance as accuracy rather than decision quality. We conclude with a concrete set of recommendations for evaluators designed to reduce the mismatch between the conceptualization of uncertainty in visualization versus other fields.
15 December 2015 in Journal of Clinical Sleep Medicine
To review sleep related consumer technologies, including mobile electronic device "apps," wearable devices, and other technologies. Validation and methodological transparency, the effect on clinical sleep medicine, and various social, legal, and ethical issues are discussed. We reviewed publications from the digital libraries of the Association for Computing Machinery, Institute of Electrical and Electronics Engineers, and PubMed; publications from consumer technology websites; and mobile device app marketplaces. Search terms included "sleep technology," "sleep app," and "sleep monitoring." Consumer sleep technologies are categorized by delivery platform including mobile device apps (integrated with a mobile operating system and utilizing mobile device functions such as the camera or microphone), wearable devices (on the body or attached to clothing), embedded devices (integrated into furniture or other fixtures in the native sleep environment), accessory appliances, and conventional desktop/website resources. Their primary goals include facilitation of sleep induction or wakening, self-guided sleep assessment, entertainment, social connection, information sharing, and sleep education. Consumer sleep technologies are changing the landscape of sleep health and clinical sleep medicine. These technologies have the potential to both improve and impair collective and individual sleep health depending on method of implementation.
26 October 2023 in IEEE Transactions on Visualization and Computer Graphics
The grammar of graphics is ubiquitous, providing the foundation for a variety of popular visualization tools and toolkits. Yet support for uncertainty visualization in the grammar graphics—beyond simple variations of error bars, uncertainty bands, and density plots—remains rudimentary. Research in uncertainty visualization has developed a rich variety of improved uncertainty visualizations, most of which are difficult to create in existing grammar of graphics implementations. ggdist, an extension to the popular ggplot2 grammar of graphics toolkit, is an attempt to rectify this situation. ggdist unifies a variety of uncertainty visualization types through the lens of distributional visualization, allowing functions of distributions to be mapped to directly to visual channels (aesthetics), making it straightforward to express a variety of (sometimes weird!) uncertainty visualization types. This distributional lens also offers a way to unify Bayesian and frequentist uncertainty visualization by formalizing the latter with the help of confidence distributions. In this paper, I offer a description of this uncertainty visualization paradigm and lessons learned from its development and adoption: ggdist has existed in some form for about six years (originally as part of the tidybayes R package for post-processing Bayesian models), and it has evolved substantially over that time, with several rewrites and API re-organizations as it changed in response to user feedback and expanded to cover increasing varieties of uncertainty visualization types. Ultimately, given the huge expressive power of the grammar of graphics and the popularity of tools built on it, I hope a catalog of my experience with ggdist will provide a catalyst for further improvements to formalizations and implementations of uncertainty visualization in grammar of graphics ecosystems. A free copy of this paper is available at https://osf.io/2gsz6. All supplemental materials are available at https://github.com/mjskay/ggdistpaper and are archived on Zenodo at doi:10.5281/zenodo.7770984.