Background: Artificial intelligence (AI) chatbots have recently gained use in medical practice by health care practitioners. Interestingly, the output of these AI chatbots was found to have varying degrees of hallucination in content and references. Such hallucinations generate doubts about their output and their implementation. Objective: The aim of our study was to propose a reference hallucination score (RHS) to evaluate the authenticity of AI chatbots’ citations. Methods: Six AI chatbots were challenged with the same 10 medical prompts, requesting 10 references per prompt. The RHS is composed of 6 bibliographic items and the reference’s relevance to prompts’ keywords. RHS was calculated for each reference, prompt, and type of prompt (basic vs complex). The average RHS was calculated for each AI chatbot and compared across the different types of prompts and AI chatbots. Results: Bard failed to generate any references. ChatGPT 3.5 and Bing generated the highest RHS (score=11), while Elicit and SciSpace generated the lowest RHS (score=1), and Perplexity generated a middle RHS (score=7). The highest degree of hallucination was observed for reference relevancy to the prompt keywords (308/500, 61.6%), while the lowest was for reference titles (169/500, 33.8%). ChatGPT and Bing had comparable RHS (β coefficient=–0.069; P=.32), while Perplexity had significantly lower RHS than ChatGPT (β coefficient=–0.345; P<.001). AI chatbots generally had significantly higher RHS when prompted with scenarios or complex format prompts (β coefficient=0.486; P<.001). Conclusions: The variation in RHS underscores the necessity for a robust reference evaluation tool to improve the authenticity of AI chatbots. Further, the variations highlight the importance of verifying their output and citations. Elicit and SciSpace had negligible hallucination, while ChatGPT and Bing had critical hallucination levels. The proposed AI chatbots’ RHS could contribute to ongoing efforts to enhance AI’s general reliability in medical research.

Automatic voice pathology detection and classification systems effectively contribute to the assessment of voice disorders, enabling the early detection of voice pathologies and the diagnosis of the type of pathology from which patients suffer. This paper concentrates on developing an accurate and robust feature extraction for detecting and classifying voice pathologies by investigating different frequency bands using autocorrelation and entropy. We extracted maximum peak values and their corresponding lag values from each frame of a voiced signal by using autocorrelation as features to detect and classify pathological samples. We also extracted the entropy for each frame of the voice signal after we normalized its values to be used as the features. These features were investigated in distinct frequency bands to assess the contribution of each band to the detection and classification processes. Various samples of the sustained vowel /a/ for both normal and pathological voices were extracted from three different databases in English, German, and Arabic. A support vector machine was used as a classifier. We also performed u-tests to investigate if there is a significant difference between the means of the normal and pathological samples. The best achieved accuracies in both detection and classification varied depending on the used band, method, and database. The most contributive bands in both detection and classification were between 1000 and 8000 Hz. The highest obtained accuracies in the case of detection were 99.69%, 92.79%, and 99.79% for Massachusetts eye and ear infirmary (MEEI), Saarbrücken voice database (SVD), and Arabic voice pathology database (AVPD), respectively. However, the highest achieved accuracies for classification were 99.54%, 99.53%, and 96.02% for MEEI, SVD, and AVPD, correspondingly, using the combined feature.

A voice disorder database is an essential element in doing research on automatic voice disorder detection and classification. Ethnicity affects the voice characteristics of a person, and so it is necessary to develop a database by collecting the voice samples of the targeted ethnic group. This will enhance the chances of arriving at a global solution for the accurate and reliable diagnosis of voice disorders by understanding the characteristics of a local group. Motivated by such idea, an Arabic voice pathology database (AVPD) is designed and developed in this study by recording three vowels, running speech, and isolated words. For each recorded samples, the perceptual severity is also provided which is a unique aspect of the AVPD. During the development of the AVPD, the shortcomings of different voice disorder databases were identified so that they could be avoided in the AVPD. In addition, the AVPD is evaluated by using six different types of speech features and four types of machine learning algorithms. The results of detection and classification of voice disorders obtained with the sustained vowel and the running speech are also compared with the results of an English-language disorder database, the Massachusetts Eye and Ear Infirmary (MEEI) database.