Deaf and mute people are an integral part of society, and it is particularly important to provide them with a platform to be able to communicate without the need for any training or learning. These people rely on sign language, but for effective communication, it is expected that others can understand sign language. Learning sign language is a challenge for those with no impairment. Another challenge is to have a system in which hand gestures of different languages are supported. In this manuscript, a system is presented that provides communication between deaf and mute (DnM) and non-deaf and mute (NDnM). The hand gestures of DnM people are acquired and processed using deep learning, and multiple language support is achieved using supervised machine learning. The NDnM people are provided with an audio interface where the hand gestures are converted into speech and generated through the sound card interface of the computer. Speech from NDnM people is acquired using microphone input and converted into text. The system is easy to use and low cost. The system is modular and can be enhanced by adding data to support more languages in the future. A supervised machine learning dataset is defined and created that provides automated multi-language communication between the DnM and NDnM people. It is expected that this system will support DnM people in communicating effectively with others and restoring a feeling of normalcy in their daily lives. The hand gesture detection accuracy of the system is more than 90% for most, while for certain scenarios, this is between 80% and 90% due to variations in hand gestures between DnM people. The system is validated and evaluated using a series of experiments.
This manuscript presents a full duplex communication system for the Deaf and Mute (D-M) based on Machine Learning (ML). These individuals, who generally communicate through sign language, are an integral part of our society, and their contribution is vital. They face communication difficulties mainly because others, who generally do not know sign language, are unable to communicate with them. The work presents a solution to this problem through a system enabling the non-deaf and mute (ND-M) to communicate with the D-M individuals without the need to learn sign language. The system is low-cost, reliable, easy to use, and based on a commercial-off-the-shelf (COTS) Leap Motion Device (LMD). The hand gesture data of D-M individuals is acquired using an LMD device and processed using a Convolutional Neural Network (CNN) algorithm. A supervised ML algorithm completes the processing and converts the hand gesture data into speech. A new dataset for the ML-based algorithm is created and presented in this manuscript. This dataset includes three sign language datasets, i.e., American Sign Language (ASL), Pakistani Sign Language (PSL), and Spanish Sign Language (SSL). The proposed system automatically detects the sign language and converts it into an audio message for the ND-M. Similarities between the three sign languages are also explored, and further research can be carried out in order to help create more datasets, which can be a combination of multiple sign languages. The ND-M can communicate by recording their speech, which is then converted into text and hand gesture images. The system can be upgraded in the future to support more sign language datasets. The system also provides a training mode that can help D-M individuals improve their hand gestures and also understand how accurately the system is detecting these gestures. The proposed system has been validated through a series of experiments resulting in hand gesture detection accuracy exceeding 95%.
On 11 February 2020, the prevalent outbreak of COVID-19, a coronavirus illness, was declared a global pandemic. Since then, nearly seven million people have died and over 765 million confirmed cases of COVID-19 have been reported. The goal of this study is to develop a diagnostic tool for detecting COVID-19 infections more efficiently. Currently, the most widely used method is Reverse Transcription Polymerase Chain Reaction (RT-PCR), a clinical technique for infection identification. However, RT-PCR is expensive, has limited sensitivity, and requires specialized medical expertise. One of the major challenges in the rapid diagnosis of COVID-19 is the need for reliable imaging, particularly X-ray imaging. This work takes advantage of artificial intelligence (AI) techniques to enhance diagnostic accuracy by automating the detection of COVID-19 infections from chest X-ray (CXR) images. We obtained and analyzed CXR images from the Kaggle public database (4035 images in total), including cases of COVID-19, viral pneumonia, pulmonary opacity, and healthy controls. By integrating advanced techniques with transfer learning from pre-trained convolutional neural networks (CNNs), specifically InceptionV3, ResNet50, and Xception, we achieved an accuracy of 95%, significantly higher than the 85.5% achieved with ResNet50 alone. Additionally, our proposed method, CXR-DNNs, can accurately distinguish between three different types of chest X-ray images for the first time. This computer-assisted diagnostic tool has the potential to significantly enhance the speed and accuracy of COVID-19 diagnoses.
16 May 2024 in VFAST Transactions on Software Engineering
The world is experiencing a transformation shift from manual labor to digital solutions, making work simpler and more efficient while enhancing the quality of life globally. A prime example of this shift is the Object Picking Robotic Arm (OPRA). Designed to operate with minimal human intervention, the OPRA reduces the risk of physical injuries among workers by replacing human labor with robotic precision. This technology finds applications in both industrial and domestic settings, including the automotive industry, metalworking, chemical processing, and various pick-and-place tasks. In this research, we develop a robotic system for automated waste picking and sorting. This system utilizes the YOLOv5 object detection algorithm to achieve high accuracy (95\%) and precision (90\%) in classifying five common waste categories: cardboard, metal, paper, plastic, and trash.
6G is one of the key cornerstone elements of the futuristic smart system setup – the others being cloud computing, big data, wearable devices and Artificial Intelligence. Also, smart offices and homes have become even more popular than before, because of the advancement in computer vision and Machine Learning (ML) technologies. Recognition of human actions and situations are fundamental components of such systems, especially in complex environments like healthcare, for example at the dentist clinic, where we need cues such as eye movement to distinguish procedures being undertaken. In this work, we compare models based on hierarchical modelling and machine learning to identify the dental procedure. We used the objects seen while following the eye trajectories and focussed on elements including material used for treatment, equipment involved and the teeth conditions i.e. symptoms. Our experiments showed that using Artificial Neural Network (ANN) increased the accuracy of prediction compared to hierarchical modelling. Our experiments show an improvement in accuracy for each of the constituent parameters i.e., symptom (ANN: 95.58% vs. Hierarchical: 45.68%), material (ANN: 86.32% vs. Hierarchical: 45.18%) and equipment (ANN: 92.65% vs. Hierarchical: 59.39%).
16 May 2024 in Foundation University Journal of Engineering and Applied Sciences (HEC Recognized Y Category , ISSN 2706-7351)
The popularity of smart clinics has increased significantly as a result of technical developments in fields like computer vision. At the heart of such systems is the ability to recognize objects and activities as well as perceive the environment as a whole. This is crucial for both eco-independent systems and human-machine interaction, especially in settings with constrained workspaces, like dental care. Our study delves into an extensive analysis of multiple machine learning models designed to robustly predict dental treatments. These models encompass Lazy Instance-based Learning, Sequential Minimal Optimization, Hoeffding Tree, and Random Tree. Leveraging object-oriented input sourced from gaze-guided wearable cameras, we scrutinize intricate attributes such as material properties, patients' dental conditions, and the array of instruments in use. Notably, we exploit the insight that identifying visual cues during an activity holds the potential to address the specific therapy identification challenge. Utilizing a dental data set that we gathered in the real world, we conducted our experiments and discovered that combining multiple criteria enhances accuracy in comparison to using each one alone. We did see, nevertheless, that in certain circumstances employing the symptoms alone produced superior outcomes. Additionally, symptoms demonstrated to have lesser error than combination in terms of RMS error convergence. Finally, we observed that the machine learning models' build and test durations increased as a result of the combined method. This demonstrates that adding additional parameters does not necessarily result in better outcomes in machine learning applications generally and in medical/dental applications in particular. Instead, it relies on the machine learning tool used, the settings taken into account and the input data provided. The versatility of our approach extends beyond dental contexts. It has been systematically validated across diverse domains, including the recognition of kitchen activities within smart home environments. This methodology holds relevance for various outdoor scenarios where the focal point of attention guides ongoing activities.