Many people can read and listen to English well but may need help to speak it well. This paper aims to implement an AI English-speaking interactive training (AIESIT) system based on AI for special-purpose English-speaking training research, enabling students to express and communicate in professional English naturally. We will provide new tools for solving the software design in English-speaking training. The proposed AIESIT system integrates generative AI, speech recognition, and body recognition. The AIESIT system uses generative AI to generate an AI agent with mouth shapes that match the English speech, which enhances the user’s feeling of being in the real world. The speech recognition system recognizes the voice content of the user’s response for passing the evaluation. Since the AIESIT system does not have a natural person online, users can speak English boldly to improve their oral skills. During the learning process, OpenPoseNet is used to recognize whether the user has poor posture or leaves the seat during the learning process. Eye CNN is used to recognize whether the user falls asleep during the learning process and as a reference for continuing the lesson. Finally, the learning trajectory, including the recognition rate and response time, is output to score the performance of the English dialogues. The results of multiple user tests have shown that increasing the number of practice sessions can improve the English speaking, encouraging users to keep practicing, and this system helps improve their English speaking.
The accuracy of noise estimation is important for the performance of a speech denoising system. Most noise estimators suffer from either overestimation or underestimation on the noise level. An overestimate on noise magnitude will cause serious speech distortion for speech denoising. Conversely, a great quantity of residual noise will occur when the noise magnitude is underestimated. Accurately estimating noise magnitude is important for speech denoising. This study proposes employing variable segment length for noise tracking and variable thresholds for the determination of speech presence probability, resulting in the performance improvement for a minima-controlled-recursive-averaging (MCRA) algorithm in noise estimation. Initially, the fundamental frequency was estimated to determine whether a frame is a vowel. In the case of a vowel frame, the increment of segment lengths and the decrement of threshold for speech presence were performed which resulted in underestimating the level of noise magnitude. Accordingly, the speech distortion is reduced in denoised speech. On the contrary, the segment length decreases rapidly in noise-dominant regions. This enables the noise estimate to update quickly and the noise variation to track well, yielding interference noise being removed effectively through the process of speech denoising. Experimental results show that the proposed approach has been effective in improving the performance of the MCRA algorithm by preserving the weak vowels and consonants. The denoising performance is therefore improved.
17 February 2021 in Signal, Image and Video Processing