Cheng Zhang

Cheng Zhang

Auto-generated profile, based on publicly available information

Cheng Zhang

Computing and Information Science, Cornell University, Ithaca, New York, USA

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 April 2023
Verified
Conference Paper
Open access
RZRuidong Zhang
Ruidong Zhang
KLKe Li
Ke Li
YHYihong Hao
Yihong Hao
YWYufan Wang
Yufan Wang
ZLZhengnan Lai
Zhengnan Lai
CZCheng Zhang
Cheng Zhang
We present EchoSpeech, a minimally-obtrusive silent speech interface (SSI) powered by low-power active acoustic sensing. EchoSpeech uses speakers and microphones mounted on a glass-frame and emits inaudible sound waves towards the skin. By analyzing echos from multiple paths, EchoSpeech captures subtle skin deformations caused by silent utterances and uses them to infer silent speech. With a user study of 12 participants, we demonstrate that EchoSpeech can recognize 31 isolated commands and 3-6 figure connected digits with 4.5% (std 3.5%) and 6.1% (std 4.2%) Word Error Rate (WER), respectively. We further evaluated EchoSpeech under scenarios including walking and noise injection to test its robustness. We then demonstrated using EchoSpeech in demo applications in real-time operating at 73.3mW, where the real-time pipeline was implemented on a smartphone with only 1-6 minutes of training data. We believe that EchoSpeech takes a solid step towards minimally-obtrusive wearable SSI for real-life deployment.

11 May 2024
Verified
Conference Paper
Open access
CLChi-Jung Lee
Chi-Jung Lee
RZRuidong Zhang
Ruidong Zhang
DADevansh Agarwal
Devansh Agarwal
TYTianhong Catherine Yu
Tianhong Catherine Yu
VGVipin Gunda
Vipin Gunda
CZCheng Zhang
Cheng Zhang
Our hands serve as a fundamental means of interaction with the world around us. Therefore, understanding hand poses and interaction contexts is critical for human-computer interaction (HCI). We present EchoWrist, a low-power wristband that continuously estimates 3D hand poses and recognizes hand-object interactions using active acoustic sensing. EchoWrist is equipped with two speakers emitting inaudible sound waves toward the hand. These sound waves interact with the hand and its surroundings through reflections and diffractions, carrying rich information about the hand’s shape and the objects it interacts with. The information captured by the two microphones goes through a deep learning inference system that recovers hand poses and identifies various everyday hand activities. Results from the two 12-participant user studies show that EchoWrist is effective and efficient at tracking 3D hand poses and recognizing hand-object interactions. Operating at 57.9 mW, EchoWrist can continuously reconstruct 20 3D hand joints with MJEDE of 4.81 mm and recognize 12 naturalistic hand-object interactions with 97.6% accuracy.

27 September 2023 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
SMSaif Mahmud
Saif Mahmud
KLKe Li
Ke Li
GHGuilin Hu
Guilin Hu
HCHao Chen
Hao Chen
RJRichard Jin
Richard Jin
CZCheng Zhang
Cheng Zhang
In this paper, we introduce PoseSonic, an intelligent acoustic sensing solution for smartglasses that estimates upper body poses. Our system only requires two pairs of microphones and speakers on the hinges of the eyeglasses to emit FMCW-encoded inaudible acoustic signals and receive reflected signals for body pose estimation. Using a customized deep learning model, PoseSonic estimates the 3D positions of 9 body joints including the shoulders, elbows, wrists, hips, and nose. We adopt a cross-modal supervision strategy to train our model using synchronized RGB video frames as ground truth. We conducted in-lab and semi-in-the-wild user studies with 22 participants to evaluate PoseSonic, and our user-independent model achieved a mean per joint position error of 6.17 cm in the lab setting and 14.12 cm in semi-in-the-wild setting when predicting the 9 body joint positions in 3D. Our further studies show that the performance was not significantly impacted by different surroundings or when the devices were remounted or by real-world environmental noise. Finally, we discuss the opportunities, challenges, and limitations of deploying PoseSonic in real-world applications.

29 May 2024
Verified
Conference Paper
Open access
KLKe Li
Ke Li
RZRuidong Zhang
Ruidong Zhang
BCBoao Chen
Boao Chen
SCSiyuan Chen
Siyuan Chen
SYSicheng Yin
Sicheng Yin
CZCheng Zhang
Cheng Zhang

11 May 2024
Verified
Conference Paper
Open access
KLKe Li
Ke Li
RZRuidong Zhang
Ruidong Zhang
SCSiyuan Chen
Siyuan Chen
BCBoao Chen
Boao Chen
MSMose Sakashita
Mose Sakashita
CZCheng Zhang
Cheng Zhang
In this paper, we introduce EyeEcho, a minimally-obtrusive acoustic sensing system designed to enable glasses to continuously monitor facial expressions. It utilizes two pairs of speakers and microphones mounted on glasses, to emit encoded inaudible acoustic signals directed towards the face, capturing subtle skin deformations associated with facial expressions. The reflected signals are processed through a customized machine-learning pipeline to estimate full facial movements. EyeEcho samples at 83.3 Hz with a relatively low power consumption of 167mW. Our user study involving 12 participants demonstrates that, with just four minutes of training data, EyeEcho achieves highly accurate tracking performance across different real-world scenarios, including sitting, walking, and after remounting the devices. Additionally, a semi-in-the-wild study involving 10 participants further validates EyeEcho’s performance in naturalistic scenarios while participants engage in various daily activities. Finally, we showcase EyeEcho’s potential to be deployed on a commercial-off-the-shelf (COTS) smartphone, offering real-time facial expression tracking.

21 November 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
SMSaif Mahmud
Saif Mahmud
VPVineet Parikh
Vineet Parikh
QLQikang Liang
Qikang Liang
KLKe Li
Ke Li
RZRuidong Zhang
Ruidong Zhang
CZCheng Zhang
Cheng Zhang
We present ActSonic, an intelligent, low-power active acoustic sensing system integrated into eyeglasses that can recognize 27 different everyday activities (e.g., eating, drinking, toothbrushing) from inaudible acoustic waves around the body. It requires only a pair of miniature speakers and microphones mounted on each hinge of the eyeglasses to emit ultrasonic waves, creating an acoustic aura around the body. The acoustic signals are reflected based on the position and motion of various body parts, captured by the microphones, and analyzed by a customized self-supervised deep learning framework to infer the performed activities on a remote device such as a mobile phone or cloud server. ActSonic was evaluated in user studies with 19 participants across 19 households to track its efficacy in everyday activity recognition. Without requiring any training data from new users (leave-one-participant-out evaluation), ActSonic detected 27 activities, achieving an average F1-score of 86.6% in fully unconstrained scenarios and 93.4% in prompted settings at participants' homes.