Yan Ma

Yan Ma

Auto-generated profile, based on publicly available information

Yan Ma

Computer Science Department, Stony Brook University, Stony Brook, New York, USA

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 April 2025
Verified
Conference Paper
Open access
HLHyunchul Lim
Hyunchul Lim
NDNam Anh Dang
Nam Anh Dang
DLDylan Lee
Dylan Lee
TYTianhong Catherine Yu
Tianhong Catherine Yu
JLJane Lu
Jane Lu
CZCheng Zhang
Cheng Zhang

19 April 2023
Verified
Conference Paper
Open access
JCJeremy Chu
Jeremy Chu
DADongsheng An
Dongsheng An
YMYan Ma
Yan Ma
WCWenzhe Cui
Wenzhe Cui
SZShumin Zhai
Shumin Zhai
XBXiaojun Bi
Xiaojun Bi
Word-gesture production models that can synthesize word-gestures are critical to the training and evaluation of word-gesture keyboard decoders. We propose WordGesture-GAN, a conditional generative adversarial network that takes arbitrary text as input to generate realistic word-gesture movements in both spatial (i.e., (x, y) coordinates of touch points) and temporal (i.e., timestamps of touch points) dimensions. WordGesture-GAN introduces a Variational Auto-Encoder to extract and embed variations of user-drawn gestures into a Gaussian distribution which can be sampled to control variation in generated gestures. Our experiments on a dataset with 38k gesture samples show that WordGesture-GAN outperforms existing gesture production models including the minimum jerk model [37] and the style-transfer GAN [31, 32] in generating realistic gestures. Overall, our research demonstrates that the proposed GAN structure can learn variations in user-drawn gestures, and the resulting WordGesture-GAN can generate word-gesture movement and predict the distribution of gestures. WordGesture-GAN can serve as a valuable tool for designing and evaluating gestural input systems.


29 October 2023
Verified
Conference Paper
Open access
JCJeremy Chu
Jeremy Chu
YMYan Ma
Yan Ma
SZShumin Zhai
Shumin Zhai
DGDavid Xianfeng Gu
David Xianfeng Gu
XBXiaojun Bi
Xiaojun Bi
Models that can generate touch typing tasks are important to the development of touch typing keyboards. We propose TouchType-GAN, a Conditional Generative Adversarial Network that can simulate locations and time stamps of touch points in touch typing. TouchType-GAN takes arbitrary text as input to generate realistic touch typing both spatially (i.e., (x, y) coordinates of touch points) and temporally (i.e., timestamps of touch points). TouchType-GAN introduces a variational generator that estimates Gaussian Distributions for every target letter to prevent mode collapse. Our experiments on a dataset with 3k typed sentences show that TouchType-GAN outperforms existing touch typing models, including the Rotational Dual Gaussian model [36] for simulating the distribution of touch points, and the Finger-Fitts Euclidean Model [30] for simulating typing time. Overall, our research demonstrates that the proposed GAN structure can learn the distribution of user typed touch points, and the resulting TouchType-GAN can also estimate typing movements. TouchType-GAN can serve as a valuable tool for designing and evaluating touch typing input systems.

25 April 2025
Verified
Conference Paper
Open access
YMYan Ma
Yan Ma
DZDan Zhang
Dan Zhang
IRIv Ramakrishnan
Iv Ramakrishnan
XBXiaojun Bi
Xiaojun Bi
Large language models (LLMs) have shown exceptional performance in various language-related tasks. However, their application in keyboard decoding, which involves converting input signals (e.g. taps and gestures) into text, remains underexplored. This paper presents a fine-tuned FLAN-T5 model for decoding. It achieves 93.1% top-1 accuracy on user-drawn gestures, outperforming the widely adopted SHARK2 decoder, and 95.4% on real-word tap typing data. In particular, our decoder supports Flexible Typing, allowing users to enter a word with taps, gestures, multi-stroke gestures, and tap-gesture combinations. User study results show that Flexible Typing is beneficial and well-received by participants, where 35.9% of words were entered using word gestures, 29.0% with taps, 6.1% with multi-stroke gestures, and the remaining 29.0% using tap-gestures. Our investigation suggests that the LLM-based decoder improves decoding accuracy over existing word gesture decoders while enabling the Flexible Typing method, which enhances the overall typing experience and accommodates diverse user preferences.

27 September 2025
Verified
Conference Paper
Open access
DZDan Zhang
Dan Zhang
YMYan Ma
Yan Ma
GDGlenn Dausch
Glenn Dausch
WSWilliam H Seiple
William H Seiple
DGDavid Xianfeng Gu
David Xianfeng Gu
XBXiaojun Bi
Xiaojun Bi
A soft Braille keyboard is a graphical representation of the Braille writing system on smartphones. It provides an essential text input method for visually impaired individuals, but accuracy and efficiency remain significant challenges. We present an intelligent Braille keyboard with auto-correction ability, which uses optimal transportation theory to estimate the distances between touch input and Braille patterns, and combines it with a language model to estimate the probability of entering words. The proposed system was evaluated through both simulations and user studies. In a touch interaction simulation on an Android phone and an iPhone, our intelligent Braille keyboard demonstrated superior error correction performance compared to the Android Braille keyboard with proofreading suggestions and the iPhone Braille keyboard with spelling suggestions. It reduced the error rate from 55.81% on Android and 57.13% on iPhone to 19.80% under high typing noise. Furthermore, in a user study of 12 participants who are legally blind, the intelligent Braille keyboard reduced word error rate (WER) by 59.5% (42.53% to 17.28%) with a slight drop of 0.74 words per minute (WPM), compared to a conventional Braille keyboard without auto-correction. These findings suggest that our approach has the potential to greatly improve the typing experience for Braille users on touchscreen devices.