Yincheng Jin

Yincheng Jin

Auto-generated profile, based on publicly available information

Yincheng Jin

Computer Science, Binghamton University, Binghamton, New York, USA

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 September 2023 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
JLJiyang Li
Jiyang Li
LHLin Huang
Lin Huang
SSSiddharth Shah
Siddharth Shah
SJSean J. Jones
Sean J. Jones
YJYincheng Jin
Yincheng Jin
ZJZhanpeng Jin
Zhanpeng Jin
Sign language is a natural language widely used by Deaf and hard of hearing (DHH) individuals. Advanced wearables are developed to recognize sign language automatically. However, they are limited by the lack of labeled data, which leads to a small vocabulary and unsatisfactory performance even though laborious efforts are put into data collection. Here we propose SignRing, an IMU-based system that breaks through the traditional data augmentation method, makes use of online videos to generate the virtual IMU (v-IMU) data, and pushes the boundary of wearable-based systems by reaching the vocabulary size of 934 with sentences up to 16 glosses. The v-IMU data is generated by reconstructing 3D hand movements from two-view videos and calculating 3-axis acceleration data, by which we are able to achieve a word error rate (WER) of 6.3% with a mix of half v-IMU and half IMU training data (2339 samples for each), and a WER of 14.7% with 100% v-IMU training data (6048 samples), compared with the baseline performance of the 8.3% WER (trained with 2339 samples of IMU data). We have conducted comparisons between v-IMU and IMU data to demonstrate the reliability and generalizability of the v-IMU data. This interdisciplinary work covers various areas such as wearable sensor development, computer vision techniques, deep learning, and linguistics, which can provide valuable insights to researchers with similar research objectives.

27 March 2023
Verified
Conference Paper
Open access
SCSeokmin Choi
Seokmin Choi
JYJunghwan Yim
Junghwan Yim
YJYincheng Jin
Yincheng Jin
YGYang Gao
Yang Gao
JLJiyang Li
Jiyang Li
ZJZhanpeng Jin
Zhanpeng Jin
Wearable devices have become indispensable gadgets in people’s daily lives nowadays; especially wireless earphones have experienced unprecedented growth in recent years, which lead to increasing interest and explorations of user authentication techniques. Conventional user authentication methods embedded in wireless earphones that use microphones or other modalities are vulnerable to environmental factors, such as loud noises or occlusions. To address this limitation, we introduce EarPPG, a new biometric modality that takes advantage of the unique in-ear photoplethysmography (PPG) signals, altered by a user’s unique speaking behaviors. When the user is speaking, muscle movements cause changes in the blood vessel geometry, inducing unique PPG signal variations. As speaking behaviors and PPG signals are unique, the EarPPG combines both biometric traits and presents a secure and obscure authentication solution. The system first detects and segments EarPPG signals and proceeds to extract effective features to construct a user authentication model with the 1D ReGRU network. We conducted comprehensive real-world evaluations with 25 human participants and achieved 94.84% accuracy, 0.95 precision, recall, and f1-score, respectively. Moreover, considering the practical implications, we conducted several extensive in-the-wild experiments, including body motions, occlusions, lighting, and permanence. Overall outcomes of this study possess the potential to be embedded in future smart earable devices.

27 March 2023
Verified
Conference Paper
Open access
YJYincheng Jin
Yincheng Jin
SCSeokmin Choi
Seokmin Choi
YGYang Gao
Yang Gao
JLJiyang Li
Jiyang Li
ZLZhengxiong Li
Zhengxiong Li
ZJZhanpeng Jin
Zhanpeng Jin
Sign language is a primary language used by deaf and hard-of-hearing (DHH) communities. However, existing sign language translation solutions primarily focus on recognizing manual markers. The non-manual markers, such as negative head shaking, question markers, and mouthing, are critical grammatical and semantic components of sign language for better usability and generalizability. Considering the significant role of non-manual markers, we propose the TransASL, a real-time, end-to-end system for sign language recognition and translation. TransASL extracts feature from both manual markers and non-manual markers via a customized eyeglasses-style wearable device with two parallel sensing modalities. Manual marker information is collected by two pairs of outward-facing microphones and speakers mounted to the legs of the eyeglasses. In contrast, non-manual marker information is acquired from a pair of inward-facing microphones and speakers connected to the eyeglasses. Both manual and non-manual marker features undergo a multi-modal, multi-channel fusion network and are eventually recognized as comprehensible ASL content. We evaluate the recognition performance of various sign language expressions at both the word and sentence levels. Given 80 frequently used ASL words and 40 meaningful sentences consisting of manual and non-manual markers, TransASL can achieve the WER of 8.3% and 7.1%, respectively. Our proposed work reveals a great potential for convenient ASL recognition in daily communications between ASL signers and hearing people.

12 June 2023 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
YJYincheng Jin
Yincheng Jin
Shibo ZhangShibo Zhang
Shibo Zhang
YGYang Gao
Yang Gao
XXXuhai Xu
Xuhai Xu
SCSeokmin Choi
Seokmin Choi
ZJZhanpeng Jin
Zhanpeng Jin
Sign language builds up an important bridge between the d/Deaf and hard-of-hearing (DHH) and hearing people. Regrettably, most hearing people face challenges in comprehending sign language, necessitating sign language translation. However, state-of-the-art wearable-based techniques mainly concentrate on recognizing manual markers (e.g., hand gestures), while frequently overlooking non-manual markers, such as negative head shaking, question markers, and mouthing. This oversight results in the loss of substantial grammatical and semantic information in sign language. To address this limitation, we introduce SmartASL, a novel proof-of-concept system that can 1) recognize both manual and non-manual markers simultaneously using a combination of earbuds and a wrist-worn IMU, and 2) translate the recognized American Sign Language (ASL) glosses into spoken language. Our experiments demonstrate the SmartASL system's significant potential to accurately recognize the manual and non-manual markers in ASL, effectively bridging the communication gaps between ASL signers and hearing people using commercially available devices.

21 November 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
YGYang Gao
Yang Gao
WZWenbo Zhang
Wenbo Zhang
JRJunbin Ren
Junbin Ren
RZRuihao Zheng
Ruihao Zheng
YJYingcheng Jin
Yingcheng Jin
ZJZhanpeng Jin
Zhanpeng Jin
The accurate assessment of human body posture through wearable technology has significant implications for sports science, clinical diagnostics, rehabilitation, and VR interaction. Traditional methods often require complex setups or are limited by the environment's constraints. In response to these challenges, this paper presents an innovative approach to human posture estimation under complex motion scenarios through the development of an advanced shoe insole embedded with pressure sensors and an Inertial Measurement Unit (IMU). Coupled with a single wrist-mounted IMU, our system facilitates a comprehensive analysis of human biomechanics by integrating physical kinematics modeling based on pressure data with a multi-region human posture estimation network. To enhance the robustness of our system model, we employed large language models to generate virtual human motion sequences. These sequences were utilized to create synthetic IMU data for data augmentation purposes, addressing the challenge of limited real-world data availability and variability. Our approach uniquely combines physical modeling with data-driven techniques to improve the accuracy and reliability of posture estimation. Experimental results demonstrate that our integrated system significantly advances wearable technology for motion analysis. The Mean Per Joint Position Error (MPJPE) was reduced to 7.75 cm, highlighting the effectiveness of our multi-modal modeling and virtual data augmentation in refining posture estimation.

27 September 2023 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
SCSeokmin Choi
Seokmin Choi
JYJunghwan Yim
Junghwan Yim
SKSe Jun Kim
Se Jun Kim
YJYincheng Jin
Yincheng Jin
DWDi Wu
Di Wu
ZJZhanpeng Jin
Zhanpeng Jin
Technical advances in the smart device market have fixated smartphones at the heart of our lives, warranting an ever more secure means of authentication. Although most smartphones have adopted biometrics-based authentication, after a couple of failed attempts, most users are given the option to quickly bypass the system with passcodes. To add a layer of security, two-factor authentication (2FA) has been implemented but has proven to be vulnerable to various attacks. In this paper, we introduce VibPath, a simultaneous 2FA scheme that can understand the user's hand neuromuscular system through touch behavior. VibPath captures the individual's vibration path responses between the hand and the wrist with the attention-based encoder-decoder network, authenticating the genuine users from the imposters unobtrusively. In a user study with 30 participants, VibPath achieved an average performance of 0.98 accuracy, 0.99 precision, 0.98 recall, 0.98 f1-score for user verification, and 94.3% accuracy for user identification across five passcodes. Furthermore, we also conducted several extensive studies, including in-the-wile, permanence, vulnerability, usability, and system overhead studies, to assess the practicability and viability of the VibPath from multiple aspects.