Named entity recognition (NER) constitutes an important step in the processing of unstructured text content for the extraction of information as well as for the computer-supported analysis of large amounts of digital data via machine learning methods. However, NER often relies on domain-specific knowledge, being conducted manually in a time- and human-resource-intensive process. These can be reduced with statistical models performing NER automatically. The current work investigates whether Conditional Random Fields (CRF) can be efficiently trained for NER in German texts, by means of an iterative procedure combining self-learning with a manual annotation–active learning–component. The training dataset increases continuously with the iterative procedure. Whilst self-learning did not markedly improve the performance of the CRF for NER, the manual annotation of sentences with the lowest probability of correct prediction clearly improved the model F1-score and simultaneously reduced the amount of manual annotation required to train the model. A model with an F1-score of 0.885 was able to be trained in 11.4 h.

Zusammenfassung: Der Umgang mit Hatespeech ist bereits seit mehreren Jahren ein Problem im Internet, insbesondere in sozialen Netzwerken. Da die enorme Menge an Kommentaren nicht mehr manuell moderiert werden kann, ist es essenziell, automatische Methoden zur Detektion offensiver Kommentare unterstützend einzusetzen. Doch speziell in Bezug auf die deutsche Sprache bringt die Erforschung von Methoden zur Hatespeech-Erkennung einige Schwierigkeiten mit sich: zum einen sprachliche Besonderheiten und zum anderen die Knappheit geeigneter Datensätze. Deshalb soll mit diesem Kapitel ein Überblick über die Forschungsentwicklung gegeben werden, die wir insbesondere anhand von Shared Tasks darstellen. Außerdem werden geeignete Datensätze, Methoden und Ergebnisse zusammenfassend dargestellt und diskutiert.

Introduction: The integration of Retrieval-Augmented Generation (RAG) into domain-specific systems enables context-aware and traceable information retrieval. This study explores chunking and embedding strategies for a RAG-based question-answering system tailored to administrative documents at University Hospital Halle, focusing on model selection, parameter tuning, and retrieval performance. The insights gained from this study should serve as the foundation for the future development of a Retrieval-Augmented Generation (RAG) based chatbot system that aims to facilitate access to document pool contents for hospital staff. Methods: A corpus of 1,219 documents was preprocessed and chunked using varied parameters, including soft/hard character limits and overlaps. Eight embedding models were evaluated using Similarity Score and Maximum Marginal Relevance (MMR) retrievers. Top models Jinaai-v3 and Aari1995 were further analyzed across eight parameter configurations and ensemble retrievers using weight (w) and context (c) parameters. Results: Aari1995 reached the highest Top10 score (92.3%) with stable performance across chunk sizes and retriever configurations. Jinaai-v3 showed slightly stronger Top5 (84.6%) and Top3 (76.9%) scores but with greater sensitivity to parameter variations. Ensemble retrievers improved retrieval quality for both models, particularly when tuned via w-values. The c-parameter showed negligible influence. Runtime evaluation revealed that Jinaai-v3 generated vector stores more than four times faster than Aari1995. Overall, the similarity score retriever consistently outperformed MMR, both standalone and in ensemble configurations. Conclusion: Chunking and embedding choices significantly affect retrieval in domain-specific RAG systems. While both Jinaai-v3 and Aari1995 were effective, they differed in stability, accuracy, and efficiency. Findings support deploying a locally executable RAG system for administrative use, guiding future optimization of chunking and parameter robustness.

Introduction: Locally deployed speech-to-text systems such as Whisper enable privacy-preserving transcription of medical encounters. However, the resulting transcripts are often lengthy, noisy, and insufficiently structured for direct integration into clinical documentation workflows. Large language models (LLMs) can transform such transcripts into concise clinical summaries, yet evaluating their quality and reliability remains challenging, particularly in the absence of gold-standard reference summaries. Materials and Methods: We propose a reference-free evaluation framework for benchmarking LLM-based post-processing of clinical transcripts generated by a locally deployed Whisper system. The framework combines embedding-based transcript–summary semantic similarity, stability analysis across repeated generations, and structured human evaluation. It was applied to 26 German physician–patient conversations. Four LLMs generated four summaries per transcript using standardized prompts and decoding parameters, resulting in 416 summaries. A subset of summaries was additionally assessed by three human raters using six predefined quality criteria. Results: GPT-OSS-120B and MedGemma-27B achieved the highest transcript–summary similarity scores across most embedding models. Although absolute similarity values varied across embedding models, relative model rankings remained largely consistent, indicating robustness of the evaluation framework. Stability analysis showed high consistency across repeated runs, with cosine similarities typically exceeding 0.90, while higher sampling temperatures reduced semantic similarity. Human evaluation showed partial agreement between embedding-based similarity signals and human judgments of summary quality. Conclusion: The proposed framework enables scalable, reference-free evaluation of LLM-based clinical summarization in privacy-sensitive settings. By combining semantic similarity, stability analysis, and human evaluation, it supports systematic model benchmarking, relative model comparison, and optimization without requiring reference summaries. These findings suggest that embedding-based metrics can provide useful signals for selecting and optimizing LLM-based post-processing models in local speech-to-text pipelines.