While user interfaces (UIs) display elements such as images and text in a grid-based layout, UI types differ significantly in the number of elements and how they are displayed. For example, webpage designs rely heavily on images and text, whereas desktop UIs tend to feature numerous small images. To examine how such differences affect the way users look at UIs, we collected and analyzed a large eye-tracking-based dataset, UEyes (62 participants and 1,980 UI screenshots), covering four major UI types: webpage, desktop UI, mobile UI, and poster. We analyze its differences in biases related to such factors as color, location, and gaze direction. We also compare state-of-the-art predictive models and propose improvements for better capturing typical tendencies across UI types. Both the dataset and the models are publicly available.

The advent of wearables (e.g., smartwatches, smartglasses, and digital jewelry) anticipates the need for text entry methods on very small devices. We conduct fundamental research on this topic using 3 qwerty-based soft keyboards for 3 different screen sizes, motivated by the extensive training that users have with qwerty keyboards. In addition to ZoomBoard (a soft keyboard for diminutive screens), we propose a callout-based soft keyboard and ZShift, a novel extension of the Shift pointing technique. We conducted a comprehensive user study followed by extensive analyses on performance, usability, and short-term learning. Our results show that different small screen sizes demand different types of assistance. In general, manufacturers can benefit from these findings by selecting an appropriate qwerty soft keyboard for their devices. Ultimately, this work provides designers, researchers, and practitioners with new understanding of qwerty soft keyboard design space and its scalability for tiny touchscreens.

Voice User Interfaces (VUIs) increasingly leverage ‘humanness’ as a foundational design metaphor, adopting roles like ‘assistants,’ ‘teachers,’ and ‘secretaries’ to foster natural interactions. Yet, this approach can sometimes misalign user trust and reinforce societal stereotypes, leading to socio-technical challenges that might impede long-term engagement. This paper explores an alternative approach to navigate these challenges—incorporating non-human metaphors in VUI design. We report on a study with 240 participants examining the effects of human versus non-human metaphors on user perceptions within health and finance domains. Results indicate a preference for the human metaphor (doctor) over the non-human (health encyclopedia) in health contexts for its perceived enjoyability and likeability. In finance, however, user perceptions do not significantly differ between human (financial advisor) and non-human (calculator) metaphors. Importantly, our research reveals that the explicit awareness of a metaphor’s use influences adoption intentions, with a marked preference for non-human metaphors when their metaphorical nature is not disclosed. These findings highlight context-specific conversation design strategies required in integrating non-human metaphors into VUI design, suggesting tradeoffs and design considerations that could enhance user engagement and adoption.

Processing data on any sort of user interaction is well known to be cumbersome and mostly time consuming. In order to assist researchers in easily inspecting fine-grained browsing data, current tools usually display user interactions as mouse cursor tracks, a video-like visualization scheme. However, to date, traditional online video inspection has not explored the full capabilities of hypermedia and interactive techniques. In response to this need, we have developed SMT2ϵ, a Web-based tracking system for analyzing browsing behavior using feature-rich hypervideo visualizations. We compare our system to related work in academia and the industry, showing that ours features unprecedented visualization capabilities. We also show that SMT2ϵ efficiently captures browsing data and is perceived by users to be both helpful and usable. A series of prediction experiments illustrate that raw cursor data are accessible and can be easily handled, providing evidence that the data can be used to construct and verify research hypotheses. Considering its limitations, it is our hope that SMT2ϵ will assist researchers, usability practitioners, and other professionals interested in understanding how users browse the Web.