The lack of transparency of powerful Machine Learning systems paired with their growth in popularity over the last decade led to the emergence of the eXplainable Artificial Intelligence (XAI) field. Instead of focusing solely on obtaining highly performing models, researchers also develop explanation techniques that help better understand the system’s reasoning for a particular output. An explainable system can be designed, developed, and evaluated from different perspectives, which enables researchers from different disciplines to work together on this topic. However, the multidisciplinary nature of XAI systems creates new challenges for condensing and structuring adequate methodologies to design and evaluate such systems. This paper presents a survey of Human-centred and Computer-centred methods to evaluate XAI systems. We propose a new taxonomy to categorize XAI evaluation methods more clearly and intuitively. This categorization gathers knowledge from different disciplines and organizes the evaluation methods according to a set of categories that represent key properties of XAI systems. Possible ways to use the proposed taxonomy in the design and evaluation of XAI systems are also discussed, alongside with some concluding remarks and future directions of research.

The direct effect of global warming on viticulture is already apparent, with unexpected pests and diseases as one of the most concerning consequences. Deploying sticky traps on grape plantations to attract key insects has been the backbone of conventional pest management programs. However, they are time-consuming processes for winegrowers, conducted through visual inspection via the manual identification and counting of key insects. Additionally, winegrowers usually lack taxonomy expertise for accurate species identification. This paper explores the usage of deep learning on the edge to identify and quantify pest counts automatically. Different mobile devices were used to acquire a dataset of yellow sticky and delta traps, consisting of 168 images with 8966 key insects manually annotated by experienced taxonomy specialists. Five different deep learning models suitable to run locally on mobile devices were selected, trained, and benchmarked to detect five different insect species. Model-centric, data-centric, and deployment-centric strategies were explored to improve and fine-tune the considered models, where they were tested on low-end and high-end mobile devices. The SSD ResNet50 model proved to be the most suitable architecture for deployment on edge devices, with accuracies per class ranging from 82% to 99%, the F1 score ranging from 58% to 84%, and inference speeds per trap image of 19.4 s and 62.7 s for high-end and low-end smartphones, respectively. These results demonstrate the potential of the approach proposed to be integrated into a mobile-based solution for vineyard pest monitoring by providing automated detection and the counting of key vector insects to winegrowers and taxonomy specialists.

The high incidence of cervical cancer in women has prompted the research of automatic screening methods. This work focuses on two of the steps present in such systems, more precisely, the identification of cervical lesions and their respective classification. The development of automatic methods for these tasks is associated with some shortcomings, such as acquiring sufficient and representative clinical data. These limitations are addressed through a hybrid pipeline based on a deep learning model (RetinaNet) for the detection of abnormal regions, combined with random forest and SVM classifiers for their categorization, and complemented by the use of domain knowledge in its design. Additionally, the nuclei in each detected region are segmented, providing a set of nuclei-specific features whose impact on the classification result is also studied. Each module is individually assessed in addition to the complete system, with the latter achieving a precision, recall and F1 score of 0.04, 0.20 and 0.07, respectively. Despite the low precision, the system demonstrates potential as an analysis support tool with the capability of increasing the overall sensitivity of the human examination process.

Due to the increasingly alarming consequences of climate change, pests are becoming a growing threat to grape quality and viticulture yields. Estimating the quantity and type of treatments to control these diseases is particularly challenging due to the unpredictability of insects’ dynamics and intrinsic difficulties in performing pest monitoring. Conventional pest monitoring programs consist of deploying sticky traps on vineyards, which attract key insects and allow human operators to identify and count them manually. However, this is a time-consuming process that usually requires in-depth taxonomic knowledge. This scenario motivated the development of EyesOnTraps, a novel AI-powered mobile solution for pest monitoring in viticulture. The methodology behind the development of the proposed system merges multidisciplinary research efforts by specialists from different fields, including informatics, electronics, machine learning, computer vision, human-centered design, agronomy and viticulture. This research work resulted in a decision support tool that allows winegrowers and taxonomy specialists to: (i) ensure the adequacy and quality of mobile-acquired sticky trap images; (ii) provide automated detection and counting of key insects; (iii) register local temperature near traps; and (iv) improve and anticipate treatment recommendations for the detected pests. By merging mobile computing and AI, we believe that broader technology acceptance for pest management in viticulture can be achieved via solutions that work on regular sticky traps and avoid the need for proprietary instrumented traps.