Explainable artificial intelligence (XAI) techniques are used to understand the rationale behind the decision-making of machine learning models. In addition to the need for model explainability, the demand for an ever-growing number of multimodal features has dramatically increased model complexity. This underscores the importance of precise feature selection to ensure high model accuracy. Using our Approximate Inverse Model Explanations (AIME) technique, which currently presents the best XAI capability in the field, this study incorporated a novel backward and forward deletion process. This pre-assesses global feature importance by calculating and ordering their AIME-reported global importance. Through the backward deletion process, it assesses model accuracy by progressively eliminating less important features, resulting in a feature set configuration that guarantees the highest model accuracy. Then, the forward deletion process further refines the feature set by discarding the least important features until the he model’s accuracy declines, which reduces the computational burden and ensures optimal performance. We applied our method to the detailed and expansive Multimodal Emotion Line dataset and leveraged 4,870 facial, voice, and spoken language features in the Google Colab Pro+ environment to demonstrate AIME’s efficacy in enabling researchers to maximize both model explainability and performance: the holy grail of XAI.

Wearing a facial mask is indispensable in the COVID-19 pandemic; however, it has tremendous effects on the performance of existing facial emotion recognition approaches. In this paper, we propose a feature vector technique comprising three main steps to recognize emotions from facial mask images. First, a synthetic mask is used to cover the facial input image. With only the upper part of the image showing, and including only the eyes, eyebrows, a portion of the bridge of the nose, and the forehead, the boundary and regional representation technique is applied. Second, a feature extraction technique based on our proposed rapid landmark detection method employing the infinity shape is utilized to flexibly extract a set of feature vectors that can effectively indicate the characteristics of the partially occluded masked face. Finally, those features, including the location of the detected landmarks and the Histograms of the Oriented Gradients, are brought into the classification process by adopting CNN and LSTM; the experimental results are then evaluated using images from the CK+ and RAF-DB data sets. As the result, our proposed method outperforms existing cutting-edge approaches and demonstrates better performance, achieving 99.30% and 95.58% accuracy on CK+ and RAF-DB, respectively.

Visually impaired individuals are at a high risk of accidents due to sudden changes in walking surfaces and surrounding obstacles. Existing smart cane systems lack the capability to detect pathway surface transition points with accurate distance estimation and danger-level assessment. This study proposes a low-cost smart cane that integrates a novel Pathway Surface Transition Point Detection (PSTPD) method with enhanced obstacle detection. The system employs dual RGB cameras, an ultrasonic sensor, and YOLO-based models to deliver real-time alerts based on object type, surface class, distance, and severity. It comprises three modules: (1) obstacle detection and classification into mild, moderate, or severe levels; (2) pathway surface detection across eight surface types with distance estimation using weighted bounding boxes and depth mapping; and (3) auditory notifications. Experimental results show a mean Average Precision (mAP@50) of 0.70 for obstacle detection and 0.92 for surface classification. The average distance estimation error was 0.3 cm for obstacles and 4.22 cm for pathway surface transition points. Additionally, the PSTPD method also demonstrated efficient processing with an average runtime of 0.6 s per instance.

The explainability of AI and machine learning models has become crucial, drawing significant attention to explainable AI (XAI) technologies. Global feature importance extraction is key to understanding the behavior of these models. However, existing XAI techniques often overlook the effects of multicollinearity in data, limiting their accuracy. We propose a method that combines approximate inverse model explanations (AIME) with hidden Markov models (HMMs). The HMMs are used to extract hidden states, and the local feature importance for each state is calculated. These values are then used to generate heuristics that adjust the importance of global features. We evaluated our method using the Wine Quality, Census Income, Gas Sensor Array Drift at Different Concentrations, and Breast Cancer Wisconsin datasets. The results showed that our method effectively handles strong multicollinearity and provides approximately 20% more accurate feature importance assessments than previous methods depending on the dataset. Therefore, our approach can better represent XAI contributions using algorithmic or human-supplied heuristics. Our study confirms that this approach is valid for XAI and enables the integration of algorithmic or human-supplied heuristics to realize XAI, which can significantly benefit various fields by enhancing model interpretability and decision-making processes.

The effectiveness of the existing approaches of facial expression recognition is significantly impacted by the necessity of wearing a facial mask during the Covid-19 outbreak. A large number of significant features on the lower face such as mouth, nose, and cheek, are completely eliminated. In this study, we propose an approach to extract features for recognizing emotions in a partial facial image that is based on Star-Like Particle Polygon Estimation (SLPPE). There are 3 primary steps in the proposed SLPPE. First, the lower part of the original facial input image is obscured by a synthetic mask. The process of representing regions is then used in the upper face image. Secondly, SLPPE is adopted as a feature extraction procedure. It is used to extract probability-based vectors of features that efficiently manifest the occluded facial image’s properties. Unlike other CNN based methods that bring the input image known as a raster form into several Conv modules in order to perform feature extraction process, we extract features from the original input image using our proposed SLPPE feature extraction technique in order to eliminate the issues of extracting unexpected features caused by the facial mask region provided by the Conv modules. Our proposed SLPPE feature extraction module produce the potential transformed data in the form of feature vectors without applying Conv modules. Finally, by utilizing LSTM and ANN networks, the SLPPE is adapted to the datasets of CK+, FER2013, and RAF-DB in order to evaluate the proposed methodology. With the accuracy of 99.01%, 98.7%, and 94.62% on CK+, FER2013, and RAF-DB, respectively, our proposed method outperforms the common CNN approaches which are currently in use and yields better results.