Head-mounted wearables are rapidly growing in popularity. However, a gap exists in providing robust voice-related applications like conversation or command control in complex environments, such as competing speakers and strong noises. The compact design of HMWs introduces non-trivial challenges to existing speech enhancement systems that use microphone recording only. In this paper, we handle this problem by using bone vibration conducted through the head skull. The principle is that the accelerometer is widely installed on head-mounted wearables and can capture the clean user's voice. Hence, we develop VibVoice, a lightweight multi-modal speech enhancement system for head-mounted wearables. We design a two-branch encoder-decoder deep neural network to fuse the high-level features of the two modalities and reconstruct clean speech. To address the issue of insufficient paired data for training, we extensively measure the bone conduction effect from a limited dataset to extract the physical impulse function for cross-modal data augmentation. We evaluate VibVoice on a dataset collected in real world and compare it with two state-of-the-art baselines. Results show that VibVoice yields up to 21% better performance in PESQ and up to 26% better performance in SNR compared with the baseline with 72 times less paired data required. We also conduct a user study with 35 participants, in which 87% participants prefer VibVoice compared with the baseline. In addition, VibVoice requires 4 to 31 times less execution time compared with baselines on mobile devices. The demo audio of VibVoice is available at https://www.youtube.com/watch?v=8_-s_C_NGRI.
Drowning poses a significant threat to human life, making underwater human activity monitoring necessary for pool management. However, it is challenging for existing methods to recognize the aquatic activities of diverse users continuously without privacy concerns. In this paper, we propose a novel sonar-based pool monitoring system that localizes swimmers and recognizes pool activities. Our system is deployed and evaluated in a real-world pool, outperforming baselines in terms of activity recognition.
23 April 2021 in IEEE Photonics Technology Letters
The whispering gallery modes (WGMs) are studied theoretically in a proposed magneto-optic (MO) silicon micro-ring resonator with a radial magnetic field. The dispersion relations between both the frequency and polarization of the resonant mode and the gyration coefficient are achieved through Finite-Element Method (FEM). Nonreciprocal polarization coupling is observed between the intrinsic quasi-TM and quasi-TE modes, due to their different frequency shift coefficients originating from the magneto-optic effect. This polarization coupling results in the rapid changes of linear polarization degrees of the supermodes. The nonreciprocal mode coupling is universal and will be of practical value for the MO optical integration.