Hate speech is an important problem in the management of user-generated content. To remove offensive content or ban misbehaving users, content moderators need reliable hate speech detectors. Recently, deep neural networks based on the transformer architecture, such as the (multilingual) BERT model, have achieved superior performance in many natural language classification tasks, including hate speech detection. So far, these methods have not been able to quantify their output in terms of reliability. We propose a Bayesian method using Monte Carlo dropout within the attention layers of the transformer models to provide well-calibrated reliability estimates. We evaluate and visualize the results of the proposed approach on hate speech detection problems in several languages. Additionally, we test whether affective dimensions can enhance the information extracted by the BERT model in hate speech classification. Our experiments show that Monte Carlo dropout provides a viable mechanism for reliability estimation in transformer networks. Used within the BERT model, it offers state-of-the-art classification performance and can detect less trusted predictions.

This study explores the geospatial relationship between the invasive crayfish species Faxonius limosus and the native Austropotamobius bihariensis and A. torrentium crayfish populations in Eastern Europe, identifying the environmental factors which influence the invasion. We used species distribution modelling based on several climatic, geophysical and water quality variables and crayfish distributional data to predict sectors suitable for each species within the river network. Thus, we identified the sectors potentially connecting invasive and native population clusters and quantified the degree of proximity between competing species. These sectors were then extensively surveyed with trapping and hand searching, doubled by eDNA methods, in order to assess whether any crayfish or the crayfish plague pathogen Aphanomyces astaci are present. The predictive models exhibited excellent performance and successfully distinguished between the analysed crayfish species. The expansion of F. limosus in streams was found to be limited by flash-flood potential, resulting in a range that is constrained to lowland rivers. Field surveys found neither crayfish nor pathogen presence in the connective sectors. Another interesting finding derived from the screening efforts, which are among the most extensive carried out across native, apparently healthy crayfish populations, was the existence of a latent infection with an A. astaci strain identified as A-haplogroup. Our results provide realistic insights for the long-term conservation of native Austropotamobius species, which appear to be naturally protected from F. limosus expansion. Conservation efforts can thus focus on other relevant aspects, such as ark-sites establishment for preventing the spread of more dangerous invasive crayfish species and of virulent crayfish plague pathogen strains, even in locations without direct contact between crayfish hosts.

The idle crayfish (Austropotamobius bihariensis Pârvulescu, 2019), endemic to Romania's Apuseni Mountains, urgently requires a specific conservation plan. Due to its recent description, conservation efforts have been limited, highlighting the need for immediate and practical recommendations to ensure its protection. Over 13 years, field observations were conducted to evaluate population trends and identify threats following IUCN standards. Additionally, geospatial assessments and predictive modelling were employed to estimate both the optimal habitat and current population size under three distinct scenarios. The primary threats identified include poor forest management, extreme drought, anthropogenic development and riverbed alterations, all contributing to declines in crayfish abundance. The most severe impacts arise when these pressures converge at a single site, compounded by a chronic, low‐virulence crayfish plague infection (A‐haplogroup). The total population is estimated at 31,150 (± 449.9 SE) individuals, with 1,163,754 m² of suitable habitat, of which only 37.9% lies within 13 protected areas under the most realistic scenario. Poor water quality was found to significantly reduce the modelled population size. In light of these findings, we propose a series of targeted conservation actions tailored for each protected area and highlight the importance of extending measures beyond their current boundaries. Additionally, we recommend implementing the ‘ark sites’ concept in regions with optimal ecological conditions, stable populations and genetic diversity, to reduce pressures and safeguard the species through effective field management.

Clinical text often includes numbers of various types and formats. However, most current text classification approaches do not take advantage of these numbers. This study aims to demonstrate that using numbers as features can significantly improve the performance of text classification models. This study also demonstrates the feasibility of extracting such features from clinical text. Unsupervised learning was used to identify patterns of number usage in clinical text. These patterns were analyzed manually and converted into pattern-matching rules. Information extraction was used to incorporate numbers as features into a document representation model. We evaluated text classification models trained on such representation. Our experiments were performed with two document representation models (vector space model and word embedding model) and two classification models (support vector machines and neural networks). The results showed that even a handful of numerical features can significantly improve text classification performance. We conclude that commonly used document representations do not represent numbers in a way that machine learning algorithms can effectively utilize them as features. Although we demonstrated that traditional information extraction can be effective in converting numbers into features, further community-wide research is required to systematically incorporate number representation into the word embedding process.

Human annotations are an important source of information in the development of natural language understanding approaches. As under the pressure of productivity annotators can assign different labels to a given text, the quality of produced annotations frequently varies. This is especially the case if decisions are difficult, with high cognitive load, requires awareness of broader context, or careful consideration of background knowledge. To alleviate the problem, we propose two semi-supervised methods to guide the annotation process: a Bayesian deep learning model and a Bayesian ensemble method. Using a Bayesian deep learning method, we can discover annotations that cannot be trusted and might require reannotation. A recently proposed Bayesian ensemble method helps us to combine the annotators' labels with predictions of trained models. According to the results obtained from three hate speech detection experiments, the proposed Bayesian methods can improve the annotations and prediction performance of BERT models.