Yongjian Fu

Yongjian Fu

Auto-generated profile, based on publicly available information

Yongjian Fu

Tsinghua University, Beijing, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 March 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
SWShuning Wang
Shuning Wang
LZLinghui Zhong
Linghui Zhong
YFYongjian Fu
Yongjian Fu
LCLili Chen
Lili Chen
JRJu Ren
Ju Ren
YZYaoxue Zhang
Yaoxue Zhang
Facial expression recognition (FER) is a crucial task for human-computer interaction and a multitude of multimedia applications that typically call for friendly, unobtrusive, ubiquitous, and even long-term monitoring. Achieving such a FER system meeting these multi-requirements faces critical challenges, mainly including the tiny irregular non-periodic deformation of emotion movements, high variability in facial positions and severe self-interference caused by users' own other behavior. In this work, we present UFace, a long-term, unobtrusive and reliable FER system for daily life using acoustic signals generated by a portable smartphone. We design an innovative network model with dual-stream input based on the attention mechanism, which can leverage distance-time profile features from various viewpoints to extract fine-grained emotion-related signal changes, thus enabling accurate identification of many kinds of expressions. Meanwhile, we propose effective mechanisms to deal with a series of interference issues during actual use. We implement UFace prototype with a daily-used smartphone and conduct extensive experiments in various real-world environments. The results demonstrate that UFace can successfully recognize 7 typical facial expressions with an average accuracy of 87.8% across 20 participants. Besides, the evaluation of different distances, angles, and interferences proves the great potential of the proposed system to be employed in practical scenarios.

3 June 2024
Verified
Conference Paper
Open access
YFYongjian Fu
Yongjian Fu
YZYongzhao Zhang
Yongzhao Zhang
YLYu Lu
Yu Lu
LQLili Qiu
Lili Qiu
YCYi-Chao Chen
Yi-Chao Chen
YZYaoxue Zhang
Yaoxue Zhang
Acoustic imaging is attractive due to its ability to work under occlusion, different lighting conditions, and privacy-sensitive environments. Existing acoustic imaging methods require large transceiver arrays or device movement, which makes it challenging to use in many scenarios. In this paper, we develop a novel acoustic imaging system for low-cost devices with few speakers and microphones without any device movement. To achieve this goal, we leverage a 3D-printed passive acoustic metasurface to significantly enhance the diversity of the measurement data, thereby improving the imaging quality. Specifically, we jointly design the transmission signal, transceivers' beamforming weights, metasurface, and imaging algorithm to minimize the imaging reconstruction error in an end-to-end manner. We further develop a scheme to dynamically adapt the imaging resolution based on the distance to the target. We implement a system prototype. Using extensive experiments, we show that our system yields high-quality images across a wide range of scenarios.

26 June 2024 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
YFYongjian Fu
Yongjian Fu
SWShuning Wang
Shuning Wang
LZLinghui Zhong
Linghui Zhong
LCLili Chen
Lili Chen
JRJu Ren
Ju Ren
YZYaoxue Zhang
Yaoxue Zhang
Silent Speech Interfaces (SSI) have been developed to convert silent articulatory gestures into speech, aiding communication in public spaces and assisting individuals with aphasia. Previous SSIs, which rely on wearable devices or cameras, often pose issues like prolonged contact or privacy risks. Recent advancements in acoustic sensing present new opportunities for gesture sensing, but they typically focus on content classification rather than reconstructing audible speech. This results in the loss of crucial speech characteristics such as rate, intonation, and emotion.In this paper, we propose UltraSR, a novel sensing system designed for accurate audible speech reconstruction by analyzing the disturbance of tiny articulatory gestures on reflected ultrasound signals. UltraSR employs a multi-scale feature extraction scheme to aggregate information from multiple views and introduces a new model that maps ultrasound to speech signals, enabling the reconstruction of audible speech from silent gestures.Instead of the laborious collection of massive training data, UltraSR constructs an inverse task to generate virtual gestures from widely available audio (e.g., phone calls) for efficient model training. Additionally, it incorporates a finetuning mechanism using unlabeled data for user adaptation.We implemented UltraSR on a portable smartphone and evaluated it in various environments. Results show that UltraSR can achieve a Character Error Rate (CER) as low as 5.22% and reduce the CER from 80.13% to 6.31% for new users with only 1 hour of ultrasound data, outperforming state-of-the-art acoustic-based approaches while preserving rich speech information.

11 November 2024 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
YFYongjian Fu
Yongjian Fu
LYLanqing Yang
Lanqing Yang
HPHao Pan
Hao Pan
YCYi-Chao Chen
Yi-Chao Chen
GXGuangtao Xue
Guangtao Xue
JRJu Ren
Ju Ren
Various characteristics of mobile applications (apps) and associated in-app services can reveal potentially-sensitive user information; however, privacy concerns have prompted third-party apps to restrict access to data related to mobile app usage. This paper outlines a novel approach to extracting detailed app usage information by analyzing electromagnetic (EM) signals emitted from mobile devices during app-related tasks. The proposed system, MagSpy, recovers user privacy information from magnetometer readings that do not require access permissions. This EM leakage becomes complex when multiple apps are used simultaneously and is subject to interference from geomagnetic signals generated by device movement. To address these challenges, MagSpy employs multiple techniques to extract and identify signals related to app usage. Specifically, the geomagnetic offset signal is canceled using accelerometer and gyroscope sensor data, and a Cascade-LSTM algorithm is used to classify apps and in-app services. MagSpy also uses CWT-based peak detection and a Random Forest classifier to detect PIN inputs. A prototype system was evaluated on over 50 popular mobile apps with 30 devices. Extensive evaluation results demonstrate the efficacy of MagSpy in identifying in-app services (96% accuracy), apps (93.5% accuracy), and extracting PIN input information (96% top-3 accuracy).

29 May 2024
Verified
Conference Paper
Open access
LCLili Chen
Lili Chen
BYBozhong Yu
Bozhong Yu
YFYongjian Fu
Yongjian Fu
JRJu Ren
Ju Ren
HPHao Pan
Hao Pan
YZYaoxue Zhang
Yaoxue Zhang

13 May 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
YFYongjian Fu
Yongjian Fu
YZYongzhao Zhang
Yongzhao Zhang
HPHao Pan
Hao Pan
YLYu Lu
Yu Lu
XLXinyi Li
Xinyi Li
YZYaoxue Zhang
Yaoxue Zhang
With the growing popularity of smart speakers, numerous novel acoustic sensing applications have been proposed for low-frequency human speech and high-frequency inaudible sounds. Spatial information plays a crucial role in these acoustic applications, enabling various location-based services. However, typically commercial microphone arrays face limitations in spatial perception of inaudible sounds due to their sparse array geometries optimized for low-frequency speech. In this paper, we introduce MetaAng, a system designed to augment microphone arrays by enabling wideband spatial perception across both speech signals and inaudible sounds by leveraging the spatial encoding capabilities of acoustic metasurfaces. Our design is grounded in the fact that, while sensitive to high-frequency signals, acoustic metasurfaces are almost non-responsive to low-frequency speech due to significant wavelength discrepancy. This observation allows us to integrate acoustic metasurfaces with sparse array geometry, simultaneously enhancing the spatial perception of high-frequency and low-frequency acoustic signals. To achieve this, we first utilize acoustic metasurfaces and a configuration optimization algorithm to encode the unique features for each incident angle. Then, we propose an unrolling soft thresholding network that employs neural-enhanced priors and compressive sensing for high-accuracy, high-resolution multi-source angle estimation. We implement a prototype, and experimental results demonstrate that MetaAng maintains robustness across various scenarios, facilitating multiple applications, including localization and tracking.