Aims: To develop Spoken Dialogue System in Indian language with Voice response and voice based biometric feature. Backgrounds: Most of research works in spoken dialogue system are carried out in U.S. and Europe and currently, few government funding projects on spoken dialogue system (SDS) are carried out in Indian academic institutes. Objective: We have tried to use our developed spoken language system to eliminate the desktop clutter. It is very normal tendency of computer users to place the most frequently used files, folders, applications shortcuts on their computer’s desktop. Cluttering of desktop not only slows down the productivity of computer but also looks very messy and very difficult to find files as well. Therefore, we tried to use the spoken dialogue system to eliminate the desktop clutters in painless manner and the services are provided to the computer users by opening the files, folders and frequently used application of users in spoken command mode with voice response. Methods: In this research article, we have attempted to utilize an Indian spoken language for communication with spoken dialogue system. We have adopted a statistical machine learning algorithm called Hidden Markov Model for development of speech recognition engine. The speaker verification module is developed using fuzzy c-means algorithm. Speech synthesis is carried out using diphone corpus. Results: The speaker verification module has yielded satisfactory results with average accuracy of 66.2% using FCM algorithm. It is also seen that fundamental frequency and formant frequency carry the distinctive characteristics of speaker verification over Indian spoken language. The vital module of SDS i.e. speech recognition engine is developed by using HMM, a statistical algorithm. It is observed that word accuracy of ASR engine is 78.22 % and 62.31 % for seen and unseen users respectively. The voice response is given to the user in terms of synthesized speech. The audio quality of synthesized speech is measured using the MOS test. The MOS test value is found as 3.8 and 3.6 over two distinct groups of listeners. Conclusion: In this research paper, we have developed a spoken dialogue system based on Odia language phone set. We have integrated speaker verification module in order to provide additional biometric based security.

Introduction: A strong student support system plays a crucial role in providing quality education for students. Facilitating the requisite support to students in the academic environment by human facilitators is still an uphill task, as facilitators are inherently subject to changes in knowledge levels and emotional states, which can affect the quality of service expected. Methods: In this research article, the authors have designed a retrieval-based multimodal and multilingual chatbot interface using artificial intelligence and machine learning algorithms. The authors have considered both continuous speech and text inputs to interact with the chatbot in the domain of educational institutions. The chatbot can be interacted with by the students in three languages, namely Hindi, Odia, and English. Results: The authors have rigorously validated the proposed artificial intelligence and machine learning-driven chatbot models using various statistical metrics, including F1 score, Precision, Recall, Sensitivity, Specificity, Confusion matrix, BLEU score, Wilcoxon signed-rank test, and 95% Confidence Intervals (CI), etc. Our proposed DNN-based model has yielded promising results compared to other algorithms, including SVM, Naïve Bayes, KNN, Random Forest, and Gradient Boost. BLEU scores for a multimodal, multilingual chatbot in Hindi, Odia, and English are 0.89, 0.92, and 0.91, respectively. Discussion: Our proposed architecture for the MM-chatbot can accept queries in both voice and text modes. A pronunciation dictionary has been integrated to address pronunciation variations, accents, and speaking styles among stakeholders. MM-chatbot can generate voice responses in accordance with the user’s query language. We have significantly improved the voice output of our proposed MM-chatbot by manipulating pitch in the synthesized speech. However, users are required to explicitly select the query language at the beginning of their conversation with the MM-chatbot. Conclusion: This research aims to design a multimodal and multilingual chatbot that can be useful for retrieving information from student support systems in educational institutions. Further the authors have proposed a chatbot that can operate in both voice and text modes.