Riste Stojanov

Dr. Riste Stojanov

orcid

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 August 2021 in Journal of Medical Internet Research
Verified
Journal Article
Open access
Riste StojanovRiste Stojanov
Riste Stojanov
GPGorjan Popovski
Gorjan Popovski
GCGjorgjina Cenikj
Gjorgjina Cenikj
BSBarbara Koroušić Seljak
Barbara Koroušić Seljak
Tome EftimovTome Eftimov
Tome Eftimov
Background: Recently, food science has been garnering a lot of attention. There are many open research questions on food interactions, as one of the main environmental factors, with other health-related entities such as diseases, treatments, and drugs. In the last 2 decades, a large amount of work has been done in natural language processing and machine learning to enable biomedical information extraction. However, machine learning in food science domains remains inadequately resourced, which brings to attention the problem of developing methods for food information extraction. There are only few food semantic resources and few rule-based methods for food information extraction, which often depend on some external resources. However, an annotated corpus with food entities along with their normalization was published in 2019 by using several food semantic resources. Objective: In this study, we investigated how the recently published bidirectional encoder representations from transformers (BERT) model, which provides state-of-the-art results in information extraction, can be fine-tuned for food information extraction. Methods: We introduce FoodNER, which is a collection of corpus-based food named-entity recognition methods. It consists of 15 different models obtained by fine-tuning 3 pretrained BERT models on 5 groups of semantic resources: food versus nonfood entity, 2 subsets of Hansard food semantic tags, FoodOn semantic tags, and Systematized Nomenclature of Medicine Clinical Terms food semantic tags. Results: All BERT models provided very promising results with 93.30% to 94.31% macro F1 scores in the task of distinguishing food versus nonfood entity, which represents the new state-of-the-art technology in food information extraction. Considering the tasks where semantic tags are predicted, all BERT models obtained very promising results once again, with their macro F1 scores ranging from 73.39% to 78.96%. Conclusions: FoodNER can be used to extract and annotate food entities in 5 different tasks: food versus nonfood entities and distinguishing food entities on the level of food groups by using the closest Hansard semantic tags, the parent Hansard semantic tags, the FoodOn semantic tags, or the Systematized Nomenclature of Medicine Clinical Terms semantic tags.

2 September 2022 in Foods
Verified
Journal Article
Open access
PDF file
GIGordana Ispirova
Gordana Ispirova
GCGjorgjina Cenikj
Gjorgjina Cenikj
MOMatevž Ogrinc
Matevž Ogrinc
EVEva Valencic
Eva Valencic
Riste StojanovRiste Stojanov
Riste Stojanov
Tome EftimovTome Eftimov
Tome Eftimov
Besides the numerous studies in the last decade involving food and nutrition data, this domain remains low resourced. Annotated corpuses are very useful tools for researchers and experts of the domain in question, as well as for data scientists for analysis. In this paper, we present the annotation process of food consumption data (recipes) with semantic tags from different semantic resources—Hansard taxonomy, FoodOn ontology, SNOMED CT terminology and the FoodEx2 classification system. FoodBase is an annotated corpus of food entities—recipes—which includes a curated version of 1000 instances, considered a gold standard. In this study, we use the curated version of FoodBase and two different approaches for annotating—the NCBO annotator (for the FoodOn and SNOMED CT annotations) and the semi-automatic StandFood method (for the FoodEx2 annotations). The end result is a new version of the golden standard of the FoodBase corpus, called the CafeteriaFCD (Cafeteria Food Consumption Data) corpus. This corpus contains food consumption data—recipes—annotated with semantic tags from the aforementioned four different external semantic resources. With these annotations, data interoperability is achieved between five semantic resources from different domains. This resource can be further utilized for developing and training different information extraction pipelines using state-of-the-art NLP approaches for tracing knowledge about food safety applications.

9 January 2023 in Computers
Verified
Journal Article
Open access
PDF file
NJNasi Jofche
Nasi Jofche
KMKostadin Mishev
Kostadin Mishev
Riste StojanovRiste Stojanov
Riste Stojanov
Milos JovanovikMilos Jovanovik
Milos Jovanovik
Eftim ZdravevskiEftim Zdravevski
Eftim Zdravevski
Dimitar TrajanovDimitar Trajanov
Dimitar Trajanov
Even though named entity recognition (NER) has seen tremendous development in recent years, some domain-specific use-cases still require tagging of unique entities, which is not well handled by pre-trained models. Solutions based on enhancing pre-trained models or creating new ones are efficient, but creating reliable labeled training for them to learn on is still challenging. In this paper, we introduce PharmKE, a text analysis platform tailored to the pharmaceutical industry that uses deep learning at several stages to perform an in-depth semantic analysis of relevant publications. The proposed methodology is used to produce reliably labeled datasets leveraging cutting-edge transfer learning, which are later used to train models for specific entity labeling tasks. By building models for the well-known text-processing libraries spaCy and AllenNLP, this technique is used to find Pharmaceutical Organizations and Drugs in texts from the pharmaceutical domain. The PharmKE platform also incorporates the NER findings to resolve co-references of entities and examine the semantic linkages in each phrase, creating a foundation for further text analysis tasks, such as fact extraction and question answering. Additionally, the knowledge graph created by DBpedia Spotlight for a specific pharmaceutical text is expanded using the identified entities. The obtained results with the proposed methodology result in about a 96% F1-score on the NER tasks, which is up to 2% better than those of the fine-tuned BERT and BioBERT models developed using the same dataset. The ultimate benefits of the platform are that pharmaceutical domain specialists may more easily identify the knowledge extracted from the input texts thanks to the platform’s visualization of the model findings. Likewise, the proposed techniques can be integrated into mobile and pervasive systems to give patients more relevant and comprehensive information from scanned medication guides. Similarly, it can provide preliminary insights to patients and even medical personnel on whether a drug from a different vendor is compatible with the patient’s prescription medication.

8 May 2023 in arXiv
Verified
Preprint
Open access
PDF file
MTMilena Trajanoska
Milena Trajanoska
Riste StojanovRiste Stojanov
Riste Stojanov
Dimitar TrajanovDimitar Trajanov
Dimitar Trajanov
The growing trend of Large Language Models (LLM) development has attracted significant attention, with models for various applications emerging consistently. However, the combined application of Large Language Models with semantic technologies for reasoning and inference is still a challenging task. This paper analyzes how the current advances in foundational LLM, like ChatGPT, can be compared with the specialized pretrained models, like REBEL, for joint entity and relation extraction. To evaluate this approach, we conducted several experiments using sustainability-related text as our use case. We created pipelines for the automatic creation of Knowledge Graphs from raw texts, and our findings indicate that using advanced LLM models can improve the accuracy of the process of creating these graphs from unstructured text. Furthermore, we explored the potential of automatic ontology creation using foundation LLM models, which resulted in even more relevant and accurate knowledge graphs.

16 September 2023 in Information
Verified
Journal Article
Open access
PDF file
Aleksandar IvanovskiAleksandar Ivanovski
Aleksandar Ivanovski
Milos JovanovikMilos Jovanovik
Milos Jovanovik
Riste StojanovRiste Stojanov
Riste Stojanov
Dimitar TrajanovDimitar Trajanov
Dimitar Trajanov
In this work, we present a state-of-the-art solution for automatic playlist continuation through a knowledge graph-based recommender system. By integrating representational learning with graph neural networks and fusing multiple data streams, the system effectively models user behavior, leading to accurate and personalized recommendations. We provide a systematic and thorough comparison of our results with existing solutions and approaches, demonstrating the remarkable potential of graph-based representation in improving recommender systems. Our experiments reveal substantial enhancements over existing approaches, further validating the efficacy of this novel approach. Additionally, through comprehensive evaluation, we highlight the robustness of our solution in handling dynamic user interactions and streaming data scenarios, showcasing its practical viability and promising prospects for next-generation recommender systems.

27 November 2017 in IEEE Access
Verified
Journal Article
Open access
Riste StojanovRiste Stojanov
Riste Stojanov
SGSasho Gramatikov
Sasho Gramatikov
IMIgor Mishkovski
Igor Mishkovski
Dimitar TrajanovDimitar Trajanov
Dimitar Trajanov
The expansion of the smart devices, the growing popularity of the social networks and the wide spread of the corporate services impose huge amounts of heterogeneous data to be generated and stored in separate silos on a daily basis. Parts of this data are private and highly sensitive as they reflect owner’s behavior, obligations, habits, and preferences. On the other hand, the emerging crowd services challenge the owners to expose this data in return to the convenience they offer. Therefore, it is imperative not only to protect the interaction with sensitive data, but also to selectively open it in an unharmful manner for the owner’s personal integrity. One of the main enablers of the crowd services is the emerging Linked Data, which is all about opening heterogeneous knowledge from separate data silos. Its growing popularity encourages the data owners to publish their personal data in linked data format. The fusion of sensor, social and corporate data opens new security challenges which extend the standard security considerations towards more flexible and context aware authorization platforms. In this paper, we propose a Linked Data Authorization (LDA) platform atop a policy language flexible enough to cover all newly emerged requirements, including context awareness. The proposed policy language extends the widely accepted W3C’s SPARQL query language and leverages its expressiveness to protect every part of the data. The novelty of our LDA platform is its unique capability of design time policy validation through stand alone testing, conflict detection and overall protection coverage extraction.