The Multi-Armed Bandit (MAB) problem has been extensively studied in order to address real-world challenges related to sequential decision making. In this setting, an agent selects the best action to be performed at time-step t, based on the past rewards received by the environment. This formulation implicitly assumes that the expected payoff for each action is kept stationary by the environment through time. Nevertheless, in many real-world applications this assumption does not hold and the agent has to face a non-stationary environment, that is, with a changing reward distribution. Thus, we present a new MAB algorithm, named f-Discounted-Sliding-Window Thompson Sampling (f-dsw TS), for non-stationary environments, that is, when the data streaming is affected by concept drift. The f-dsw TS algorithm is based on Thompson Sampling (TS) and exploits a discount factor on the reward history and an arm-related sliding window to contrast concept drift in non-stationary environments. We investigate how to combine these two sources of information, namely the discount factor and the sliding window, by means of an aggregation function $f(.)$. In particular, we proposed a pessimistic ($f=min$), an optimistic ($f=max$), as well as an averaged ($f=mean$) version of the f-dsw TS algorithm. A rich set of numerical experiments is performed to evaluate the f-dsw TS algorithm compared to both stationary and non-stationary state-of-the-art TS baselines. We exploited synthetic environments (both randomly-generated and controlled) to test the MAB algorithms under different types of drift, that is, sudden/abrupt, incremental, gradual and increasing/decreasing drift. Furthermore, we adapt four real-world active learning tasks to our framework—a prediction task on crimes in the city of Baltimore, a classification task on insects species, a recommendation task on local web-news, and a time-series analysis on microbial organisms in the tropical air ecosystem. The f-dsw TS approach emerges as the best performing MAB algorithm. At least one of the versions of f-dsw TS performs better than the baselines in synthetic environments, proving the robustness of f-dsw TS under different concept drift types. Moreover, the pessimistic version ($f=min$) results as the most effective in all real-world tasks.

The classification of coralline algae commonly relies on the morphology of cells and reproductive structures, along with thallus organization, observed through Scanning Electron Microscopy (SEM). Nevertheless, species identification based on morphology often leads to uncertainty, due to their general plasticity. Evolutionary and environmental studies featured coralline algae for their ecological significance in both recent and past Oceans and need to rely on robust taxonomy. Research efforts towards new putative diagnostic tools have recently been focused on cell wall ultrastructure. In this work, we explored a new classification tool for coralline algae, using fine-tuning pretrained Convolutional Neural Networks (CNNs) on SEM images paired to morphological categories, including cell wall ultrastructure. We considered four common Mediterranean species, classified at genus and at the species level (Lithothamnion corallioides, Mesophyllum philippii, Lithophyllum racemus, Lithophyllum pseudoracemus). Our model produced promising results in terms of image classification accuracy given the constraint of a limited dataset and was tested for the identification of two ambiguous samples referred to as L. cf. racemus. Overall, explanatory image analyses suggest a high diagnostic value of calcification patterns, which significantly contributed to class predictions. Thus, CNNs proved to be a valid support to the morphological approach to taxonomy in coralline algae.

Reproducibility is a main principle in science and fundamental to ensure scientific progress. However, many recent works point out that there are widespread deficiencies for this aspect in the AI field, making the reproducibility of results impractical or even impossible. We therefore studied the state of reproducibility support on the topic of Reinforcement Learning & Recommender Systems to analyse the situation in this context. We collected a total of 60 papers and analysed them by defining a set of variables to inspect the most important aspects that enable reproducibility, such as dataset, pre-processing code, hardware specifications, software dependencies, algorithm implementation, algorithm hyperparameters, and experiment code. Furthermore, we used the ACM Badges definitions assigning them to the selected papers. We discovered that, like in many other AI domains, the Reinforcement Learning & Recommender Systems field is grappling with a reproducibility crisis, as none of the selected papers were reproducible when strictly applying the ACM Badges definitions according to our analysis.

Post-harvest diseases are one of the main causes of economical losses in the apple fruit production sector. Therefore, this paper presents an application of a knowledge-based expert system to diagnose post-harvest diseases of apple. Specifically, we detail the process of domain knowledge elicitation for constructing a Bayesian network reasoning system. We describe the developed expert system, dubbed BN-DSSApple, and the diagnostic mechanism given the evidence provided by the user, as well as a likelihood evidence method, learned from the estimated consensus of users’ and expert’s interactions, to effectively transfer the performance of the model to different cohorts of users. Finally, we detail a novel technique for explaining the provided diagnosis, thus increasing the trust in the system. We evaluate BN-DSSApple with three different types of user studies, involving real diseased apples, where the ground truth of the target instances was established by microbiological and DNA analysis. The experiments demonstrate the performance differences in the knowledge-based reasoning mechanism due to heterogeneous users interacting with the system under various conditions and the capability of the likelihood-based method to improve the diagnostic performance in different environments.