This article provides a comprehensive survey of contemporary language modeling approaches within the realm of natural language processing (NLP) tasks. This paper conducts an analytical exploration of diverse methodologies employed in the creation of language models. This exploration encompasses the architecture, training processes, and optimization strategies inherent in these models. The detailed discussion covers various models ranging from traditional n-gram and hidden Markov models to state-of-the-art neural network approaches such as BERT, GPT, LLAMA, and Bard. This article delves into different modifications and enhancements applied to both standard and neural network architectures for constructing language models. Special attention is given to addressing challenges specific to agglutinative languages within the context of developing language models for various NLP tasks, particularly for Arabic and Turkish. The research highlights that contemporary transformer-based methods demonstrate results comparable to those achieved by traditional methods employing Hidden Markov Models. These transformer-based approaches boast simpler configurations and exhibit faster performance during both training and analysis. An integral component of the article is the examination of popular and actively evolving libraries and tools essential for constructing language models. Notable tools such as NLTK, TensorFlow, PyTorch, and Gensim are reviewed, with a comparative analysis considering their simplicity and accessibility for implementing diverse language models. The aim is to provide readers with insights into the landscape of contemporary language modeling methodologies and the tools available for their implementation.
31 March 2024 in Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska
The paper presents an information technology for assessing the degree of engraftment of dental implants in the event of a pathology violation through the use of fuzzy sets, which allows using this method for medical diagnostic tasks. Main scientific results: developed algorithms and mathematical models that formalize the process supporting diagnostic decisions based on fuzzy logic; developed mathematical models of membership functions that formalize the presentation of qualitative and qualitative informational features based on the rules of fuzzy logic, which can be used in information expert systems when assessing the degree of engraftment of dental implants in case of disease with pathological diseases.
30 December 2022 in Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska
The article presents the results of research on the implementation of the principles of inclusive education in Ukraine, substantiates its relevance in the development of a democratic society and European integration policy. The principles of the state policy in the field of inclusive education, its normative-legal support and international practice of providing educational services to citizens with special needs are described. Advanced experience of inclusion introduction in the educational process of preschool, general secondary, vocational and higher education institutions is analyzed. Official statistics on the total number of people with disabilities in Ukraine; availability of special secondary education institutions (boarding schools) and their contingent; the number of children with disabilities in preschool education institutions, students with disabilities in full-time secondary education institutions, persons with disabilities in vocational education institutions, persons with disabilities among students of higher education institutions etc., have been handled. The importance and expediency of the scientific development of the principles of inclusive education, its practical implementation and study of socio-economic effect are substantiated. Graphic visualization of the correlation between the employed persons with disabilities and persons without disabilities, number of children, pupils, students with special educational needs in education establishments of Ukraine is presented.
Similar text fragments extraction from weakly formalized data is the task of natural language processing and intelligent data analysis and is used for solving the problem of automatic identification of connected knowledge fields. In order to search such common communities in Wikipedia, we propose to use as an additional stage a logical-algebraic model for similar collocations extraction. With Stanford Part-Of-Speech tagger and Stanford Universal Dependencies parser, we identify the grammatical characteristics of collocation words. With WordNet synsets, we choose their synonyms. Our dataset includes Wikipedia articles from different portals and projects. The experimental results show the frequencies of synonymous text fragments in Wikipedia articles that form common information spaces. The number of highly frequented synonymous collocations can obtain an indication of key common up-to-date Wikipedia communities.
Open Information Extraction (OIE) is a modern strategy to extract the triplet of facts from Web-document collections. However, most part of the current OIE approaches is based on NLP techniques such as POS tagging and dependency parsing, which tools are accessible not to all languages. In this paper, we suggest the logical-linguistic model, which basic mathematical means are logical-algebraic equations of finite predicates algebra. These equations allow expressing a semantic role of the participant of a triplet of the fact (Subject-Predicate-Object) due to the relations of grammatical characteristics of words in the sentence. We propose the model that extracts the unlimited domain-independent number of facts from sentences of different languages. The use of our model allows extracting the facts from unstructured texts without requiring a pre-specified vocabulary, by identifying relations in phrases and associated arguments in arbitrary sentences of English, Kazakh, and Russian languages. We evaluate our approach on corpora of three languages based on English and Kazakh bilingual news websites. We achieve the precision of facts extraction over 87% for English corpus, over 82% for Russian corpus and 71% for Kazakh corpus.