The growth of hateful online content, or hate speech, has been associated with a global increase in violent crimes against minorities [23]. Harmful online content can be produced easily, automatically and anonymously. Even though, some form of auto-detection is already achieved through text classifiers in NLP, they can be fooled by adversarial attacks. To strengthen existing systems and stay ahead of attackers, we need better adversarial attacks. In this paper, we show that up to 70% of adversarial examples generated by adversarial attacks should be discarded because they do not preserve semantics. We address this core weakness and propose a new, fully supervised sentence embedding technique called Semantics-Preserving-Encoder (SPE). Our method outperforms existing sentence encoders used in adversarial attacks by achieving 1.2× ∼ 5.1× better real attack success rate. We release our code as a plugin that can be used in any existing adversarial attack to improve its quality and speed up its execution. (The code, datasets and test examples are available at https://github.com/DavidHerel/semantics-preserving-encoder.)

This article introduces Alquist 5.0, our SocialBot that was designed for the Alexa Prize SocialBot Grand Challenge 5. Building upon previous iterations, we present the integration of our novel neural response generator (NRG) Barista within a hybrid architecture that combines traditional predefined dialogues with advanced neural response generation. We provide a comprehensive analysis of the current state-of-the-art NRGs and large language models (LLMs), leveraging these insights to enhance Barista’s capabilities. A key focus of our development was in ensuring the safety of our chatbot and implementing robust measures to prevent profanity and inappropriate content. Additionally, we incorporated a new search engine to improve information retrieval and response accuracy. Expanding the capabilities of our system, we designed Alquist 5.0 to accommodate multimodal devices, utilizing APL templates enriched with custom features to deliver an outstanding conversational experience complemented by an excellent user interface. This paper offers detailed insights into the development of Alquist 5.0, which effectively addresses evolving user demands while preserving its empathetic and knowledgeable conversational prowess across a wide range of topics.

How much is 56 times 37? Language models often make mistakes in these types of difficult calculations. This is usually explained by their inability to perform complex reasoning. Since language models rely on large training sets and great memorization capability, naturally they are not equipped to run complex calculations. However, one can argue that humans also cannot perform this calculation immediately and require a considerable amount of time to construct the solution. In order to enhance the generalization capability of language models, and as a parallel to human behavior, we propose to use special 'thinking tokens' which allow the model to perform much more calculations whenever a complex problem is encountered.

This paper provides a pioneering examination and enhancement of generative chat models, with a specific focus on the BlenderBot 3 model. Through meticulous interaction with a diverse set of human participants, we dissected the fundamental components of these models, unveiling several deficiencies, including long-term memory and entity recognition. Leveraging these insights, we engineered refined, streamlined iterations, culminating in a chatbot that transcends the capabilities of all existing models. Our work follows Occam’s razor principle and proves that, for tasks with relatively low complexity, using large overparameterized models instead of smaller ones does not bring significant benefits but increases latency, which may result in a lowered overall user experience. In upholding our commitment to transparency and the progression of shared knowledge, we have made our improved model universally accessible through open-source distribution.

The growth of hateful online content, or hate speech, has been associated with a global increase in violent crimes against minorities [23]. Harmful online content can be produced easily, automatically and anonymously. Even though, some form of auto-detection is already achieved through text classifiers in NLP, they can be fooled by adversarial attacks. To strengthen existing systems and stay ahead of attackers, we need better adversarial attacks. In this paper, we show that up to 70% of adversarial examples generated by adversarial attacks should be discarded because they do not preserve semantics. We address this core weakness and propose a new, fully supervised sentence embedding technique called Semantics-Preserving-Encoder (SPE). Our method outperforms existing sentence encoders used in adversarial attacks by achieving 1.2x - 5.1x better real attack success rate. We release our code as a plugin that can be used in any existing adversarial attack to improve its quality and speed up its execution.

Generalization is arguably the most important goal of statistical language modeling research. Publicly available benchmarks and papers published with an open-source code have been critical to advancing the field. However, it is often very difficult, and sometimes even impossible, to reproduce the results fully as reported in publications. In this paper, we propose a simple framework that should help advance the state of the art in language modeling in terms of generalization. We propose to publish not just the code, but also probabilities on dev and test sets with future publications so that one can easily add the new model into an ensemble. This has crucial advantages: it is much easier to determine whether a newly proposed model is actually complementary to the current baseline. Therefore, instead of inventing new names for the old tricks, the scientific community can advance faster. Finally, this approach promotes diversity of ideas: one does not need to create an individual model that is the new state of the art to attract attention; it will be sufficient to develop a new model that learns patterns which other models do not. Thus, even a suboptimal model can be found to have value. Remarkably, our approach has yielded new state-of-the-art results across various language modeling benchmarks up to 10%.