John J. Dudley

John J. Dudley

Auto-generated profile, based on publicly available information

John J. Dudley

University of Cambridge

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 June 2018 in ACM Transactions on Interactive Intelligent Systems
Verified
Journal Article
Open access
PDF file
JDJohn J. Dudley
John J. Dudley
PKPer Ola Kristensson
Per Ola Kristensson
Interactive Machine Learning (IML) seeks to complement human perception and intelligence by tightly integrating these strengths with the computational power and speed of computers. The interactive process is designed to involve input from the user but does not require the background knowledge or experience that might be necessary to work with more traditional machine learning techniques. Under the IML process, non-experts can apply their domain knowledge and insight over otherwise unwieldy datasets to find patterns of interest or develop complex data-driven applications. This process is co-adaptive in nature and relies on careful management of the interaction between human and machine. User interface design is fundamental to the success of this approach, yet there is a lack of consolidated principles on how such an interface should be implemented. This article presents a detailed review and characterisation of Interactive Machine Learning from an interactive systems perspective. We propose and describe a structural and behavioural model of a generalised IML system and identify solution principles for building effective interfaces for IML. Where possible, these emergent solution principles are contextualised by reference to the broader human-computer interaction literature. Finally, we identify strands of user interface research key to unlocking more efficient and productive non-expert interactive machine learning applications.

12 September 2023 in Virtual Reality
Verified
Journal Article
Open access
PDF file
JDJohn Dudley
John Dudley
LYLulu Yin
Lulu Yin
VGVanja Garaj
Vanja Garaj
PKPer Ola Kristensson
Per Ola Kristensson
Virtual Reality (VR) and Augmented Reality (AR) afford new forms of work and leisure. While affordable and effective VR and AR headsets are now available, neither technology has achieved widespread user adoption. However, we predict continual technological advances and cost reductions are likely to lead to wider diffusion in society. Bridging the chasm from the early adopters to the early majority will require careful consideration of the needs of a more casual and diverse user population. In particular, it is desirable to minimise the exclusion of potential users based on their unique needs and maximise the inclusion of users in these novel immersive experiences. Ensuring equitable access to the emerging metaverse further reinforces the need to consider the diverse needs of users. We refer to this objective of maximising the accessibility and enjoyment potential of users of VR, AR and the metaverse as Inclusive Immersion. This paper reviews the research and commercial landscape seeking to address the accessibility needs of users in VR and AR. The survey provides the basis for a synthesis of the emerging strategies for maximising the inclusiveness of VR and AR applications. Finally, we identify several unaddressed accessibility challenges requiring further research attention. Our paper consolidates disparate efforts related to promoting accessible VR and AR and delivers directions for advancing research in this area.

13 December 2018 in ACM Transactions on Computer-Human Interaction
Verified
Journal Article
Open access
JDJohn J. Dudley
John J. Dudley
KVKeith Vertanen
Keith Vertanen
PKPer Ola Kristensson
Per Ola Kristensson
We present the VISAR keyboard: An augmented reality (AR) head-mounted display (HMD) system that supports text entry via a virtualised input surface. Users select keys on the virtual keyboard by imitating the process of single-hand typing on a physical touchscreen display. Our system uses a statistical decoder to infer users’ intended text and to provide error-tolerant predictions. There is also a high-precision fall-back mechanism to support users in indicating which keys should be unmodified by the auto-correction process. A unique advantage of leveraging the well-established touch input paradigm is that our system enables text entry with minimal visual clutter on the see-through display, thus preserving the user’s field-of-view. We iteratively designed and evaluated our system and show that the final iteration of the system supports a mean entry rate of 17.75wpm with a mean character error rate less than 1%. This performance represents a 19.6% improvement relative to the state-of-the-art baseline investigated: A gaze-then-gesture text entry technique derived from the system keyboard on the Microsoft HoloLens. Finally, we validate that the system is effective in supporting text entry in a fully mobile usage scenario likely to be encountered in industrial applications of AR HMDs.

4 October 2023 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
JDJohn J. Dudley
John J. Dudley
JZJingyao Zheng
Jingyao Zheng
AGAakar Gupta
Aakar Gupta
HBHrvoje Benko
Hrvoje Benko
MLMatt Longest
Matt Longest
PKPer Ola Kristensson
Per Ola Kristensson
Integrated hand-tracking on modern virtual reality (VR) headsets can be readily exploited to deliver mid-air virtual input surfaces for text entry. These virtual input surfaces can closely replicate the experience of typing on a Qwerty keyboard on a physical touchscreen, thereby allowing users to leverage their pre-existing typing skills. However, the lack of passive haptic feedback, unconstrained user motion, and potential tracking inaccuracies or observability issues encountered in this interaction setting typically degrades the accuracy of user articulations. We present a comprehensive exploration of error-tolerant probabilistic hand-based input methods to support effective text input on a mid-air virtual Qwerty keyboard. Over three user studies we examine the performance potential of hand-based text input under both gesture and touch typing paradigms. We demonstrate typical entry rates in the range of 20 to 30 wpm and average peak entry rates of 40 to 45 wpm.

2 October 2023 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
JSJunxiao Shen
Junxiao Shen
JDJohn Dudley
John Dudley
PKPer Ola Kristensson
Per Ola Kristensson
We present a fast mid-air gesture keyboard for head-mounted optical see-through augmented reality (OST AR) that supports users in articulating word patterns by merely moving their own physical index finger in relation to a virtual keyboard plane without a need to indirectly control a visual 2D cursor on a keyboard plane. To realize this, we introduce a novel decoding method that directly translates users' three-dimensional fingertip gestural trajectories into their intended text. We evaluate the efficacy of the system in three studies that investigate various design aspects, such as immediate efficacy, accelerated learning, and whether it is possible to maintain performance without providing visual feedback. We find that the new 3D trajectory decoding design results in significant improvements in entry rates while maintaining low error rates. In addition, we demonstrate that users can maintain their performance even without fingertip and gesture trace visualization.

23 January 2023 in IEEE Pervasive Computing
Verified
Journal Article
Closed access
YLYi-Chi Liao
Yi-Chi Liao
JDJohn J. Dudley
John J. Dudley
GMGeorge B. Mo
George B. Mo
CCChun-Lien Cheng
Chun-Lien Cheng
LCLiwei Chan
Liwei Chan
PKPer Ola Kristensson
Per Ola Kristensson
Interaction design typically involves challenging decision making that requires designers to consider multiple parameters and careful tradeoffs between various objectives. This article examines how AI can facilitate the process of interaction design by offloading some of the complex decision making required of designers. We study how multi-objective Bayesian optimization can be used to support designers when creating a tactile display for smart watches. We present the results of a study that explores how such human–AI collaboration afforded by multi-objective Bayesian optimization can be exploited by designers, and the advantages and disadvantages this solution offers over conventional design practice.