Shumin Zhai

Shumin Zhai

Auto-generated profile, based on publicly available information

Shumin Zhai

Google, Mountain View, USA

Research Focus & Metrics

Academic Career Length

15Years (2012-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 December 2012 in Foundations and Trends® in Human-Computer Interaction
Verified
Journal Article
Open access
PDF file
SZShumin Zhai
Shumin Zhai
PKPer Ola Kristensson
Per Ola Kristensson
CACaroline Appert
Caroline Appert
TATue Haste Anderson
Tue Haste Anderson
XCXiang Cao
Xiang Cao
The potential for using stroke gestures to enter, retrieve and select commands and text has been recently unleashed by the popularity of touchscreen devices. This monograph provides a state-of-the-art integrative review of a body of human–computer interaction research on stroke gestures. It begins with an analysis of the design dimensions of stroke gestures as an interaction medium. The analysis classifies gestures into analogue versus abstract gestures, gestures for commands versus for symbols, gestures with different orders of complexity, visual-spatial dependent and independent gestures, and finger versus stylus drawn gestures. Gesture interfaces such as the iOS interface, the Graffiti text entry method for Palm devices, marking menus, and the SHARK/ShapeWriter word-gesture keyboard, make different choices in this multi-dimensional design space. The main body of this work consists of reviewing and synthesizing some of the foundational studies in the literature on stroke gesture interaction, particularly those done by the authors in the last decade. The human performance factors covered include motor control complexity, visual and auditory feedback, and human memory capabilities in dealing with gestures. Based on these foundational studies this review presents a set of design principles for creating stroke gesture interfaces. These include making gestures analogous to physical effects or cultural conventions, keeping gestures simple and distinct, defining stroke gestures systematically, making them self-revealing, supporting appropriate levels of chunking, and facilitating progress from visually guided performance to recall-driven performance. The overall theme is on making learning gestures easier while designing for long-term efficiency. Important system implementation issues of stroke gesture interfaces such as gesture recognition algorithms and gesture design toolkits are also covered in this review. The monograph ends with a few call-to-action research topics.

11 May 2024
Verified
Conference Paper
Open access
SLSusan Lin
Susan Lin
JWJeremy Warner
Jeremy Warner
JZJ.D. Zamfirescu-Pereira
J.D. Zamfirescu-Pereira
MLMatthew G Lee
Matthew G Lee
SJSauhard Jain
Sauhard Jain
CLCan Liu
Can Liu
Dictation enables efficient text input on mobile devices. However, writing with speech can produce disfluent, wordy, and incoherent text and thus requires heavy post-processing. This paper presents Rambler, an LLM-powered graphical user interface that supports gist-level manipulation of dictated text with two main sets of functions: gist extraction and macro revision. Gist extraction generates keywords and summaries as anchors to support the review and interaction with spoken text. LLM-assisted macro revisions allow users to respeak, split, merge, and transform dictated text without specifying precise editing locations. Together they pave the way for interactive dictation and revision that help close gaps between spontaneously spoken words and well-structured writing. In a comparative study with 12 participants performing verbal composition tasks, Rambler outperformed the baseline of a speech-to-text editor + ChatGPT, as it better facilitates iterative revisions with enhanced user control over the content while supporting surprisingly diverse user strategies.

16 October 2016
Verified
Conference Paper
Open access
PDF file
XBXiaojun Bi
Xiaojun Bi
SZShumin Zhai
Shumin Zhai
Accurately predicting the accuracy of finger-touch target acquisition is crucial for designing touchscreen UI and for modeling complex and higher level touch interaction behaviors. Despite its importance, there has been little theoretical work on creating such models. Building on the Dual Gaussian Distribution Model[3], we derived an accuracy model that predicts the success rate of target acquisition based on the target size. We evaluated the model by comparing the predicted success rates with empirical measures for three types of targets including 1-dimensional vertical and horizontal, and 2-dimensional circular targets. The predictions matched the empirical data very well: the differences between predicted and observed success rates were under 5% for 4.8 mm and 7.2 mm targets, and under 10% for 2.4 mm targets. The evaluation results suggest that our simple model can reliably predict touch accuracy.

26 April 2014
Verified
Conference Paper
Open access
PDF file
XBXiaojun Bi
Xiaojun Bi
TOTom Ouyang
Tom Ouyang
SZShumin Zhai
Shumin Zhai
Correcting erroneous input (i.e., correction) and completing a word based on partial input (i.e., completion) are two important "smart" capabilities of a modern intelligent touchscreen keyboard. However little is known whether these two capabilities are conflicting or compatible with each other in the keyboard parameter tuning. Applying computational optimization methods, this work explores the optimality issues related to them. The work demonstrates that it is possible to simultaneously optimize a keyboard algorithm for both correction and completion. The keyboard simultaneously optimized for both introduces no compromise to correction and only a slight compromise to completion when compared to the keyboards exclusively optimized for one objective. Our research also demonstrates the effectiveness of the proposed optimization method in keyboard algorithm design, which is based on the Pareto multi-objective optimization and the Metropolis algorithm. For the development and test datasets used in our experiments, computational optimization improved the correction accuracy rate by 8.3% and completion power by 17.7%.

11 May 2024
Verified
Conference Paper
Open access
DSDanqing Shi
Danqing Shi
YZYujun Zhu
Yujun Zhu
JJJussi P. P. Jokinen
Jussi P. P. Jokinen
AAAditya Acharya
Aditya Acharya
APAini Putkonen
Aini Putkonen
AOAntti Oulasvirta
Antti Oulasvirta
Touchscreen typing requires coordinating the fingers and visual attention for button-pressing, proofreading, and error correction. Computational models need to account for the associated fast pace, coordination issues, and closed-loop nature of this control problem, which is further complicated by the immense variety of keyboards and users. The paper introduces CRTypist, which generates human-like typing behavior. Its key feature is a reformulation of the supervisory control problem, with the visual attention and motor system being controlled with reference to a working memory representation tracking the text typed thus far. Movement policy is assumed to asymptotically approach optimal performance in line with cognitive and design-related bounds. This flexible model works directly from pixels, without requiring hand-crafted feature engineering for keyboards. It aligns with human data in terms of movements and performance, covers individual differences, and can generalize to diverse keyboard designs. Though limited to skilled typists, the model generates useful estimates of the typing performance achievable under various conditions.

21 April 2018
Verified
Conference Paper
Open access
PDF file
JZJingjie Zheng
Jingjie Zheng
XBXiaojun Bi
Xiaojun Bi
KLKun Li
Kun Li
YLYang Li
Yang Li
SZShumin Zhai
Shumin Zhai
Despite their learning advantages in theory, marking menus have faced adoption challenges in practice, even on today's touchscreen-based mobile devices. We address these challenges by designing, implementing, and evaluating multiple versions of M3 Gesture Menu (M3), a reimagination of marking menus targeted at mobile interfaces. M3 is defined on a grid rather than in a radial space, relies on gestural shapes rather than directional marks, and has constant and stationary space use. Our first controlled experiment on expert performance showed M3 was faster and less error-prone by a factor of two than traditional marking menus. A second experiment on learning demonstrated for the first time that users could successfully transition to recall-based execution of a dozen commands after three ten-minute practice sessions with both M3 and Multi-Stroke Marking Menu. Together, M3, with its demonstrated resolution, learning, and space use benefits, contributes to the design and understanding of menu selection in the mobile-first era of end-user computing.