Marek Herde

Marek Herde


Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 December 2021 in IEEE Access
Verified
Journal Article
Open access
MHMarek Herde
Marek Herde
DHDenis Huseljic
Denis Huseljic
BSBernhard Sick
Bernhard Sick
ACAdrian Calma
Adrian Calma
Pool-based active learning (AL) aims to optimize the annotation process (i.e., labeling) as the acquisition of annotations is often time-consuming and therefore expensive. For this purpose, an AL strategy queries annotations intelligently from annotators to train a high-performance classification model at a low annotation cost. Traditional AL strategies operate in an idealized framework. They assume a single, omniscient annotator who never gets tired and charges uniformly regardless of query difficulty. However, in real-world applications, we often face human annotators, e.g., crowd or in-house workers, who make annotation mistakes and can be reluctant to respond if tired or faced with complex queries. Recently, many novel AL strategies have been proposed to address these issues. They differ in at least one of the following three central aspects from traditional AL: 1) modeling of (multiple) human annotators whose performances can be affected by various factors, such as missing expertise; 2) generalization of the interaction with human annotators through different query and annotation types, such as asking an annotator for feedback on an inferred classification rule; 3) consideration of complex cost schemes regarding annotations and misclassifications. This survey provides an overview of these AL strategies and refers to them as real-world AL. Therefore, we introduce a general real-world AL strategy as part of a learning cycle and use its elements, e.g., the query and annotator selection algorithm, to categorize about 60 real-world AL strategies. Finally, we outline possible directions for future research in the field of AL.

31 August 2024 in International Journal of Computer Vision
Verified
Journal Article
Open access
PDF file
DHDenis Huseljic
Denis Huseljic
MHMarek Herde
Marek Herde
PHPaul Hahn
Paul Hahn
MMMehmet Müjde
Mehmet Müjde
BSBernhard Sick
Bernhard Sick
In the field of deep learning based computer vision, the development of deep object detection has led to unique paradigms (e.g., two-stage or set-based) and architectures (e.g., Faster-RCNN or DETR) which enable outstanding performance on challenging benchmark datasets. Despite this, the trained object detectors typically do not reliably assess uncertainty regarding their own knowledge, and the quality of their probabilistic predictions is usually poor. As these are often used to make subsequent decisions, such inaccurate probabilistic predictions must be avoided. In this work, we investigate the uncertainty calibration properties of different pretrained object detection architectures in a multi-class setting. We propose a framework to ensure a fair, unbiased, and repeatable evaluation and conduct detailed analyses assessing the calibration under distributional changes (e.g., distributional shift and application to out-of-distribution data). Furthermore, by investigating the influence of different detector paradigms, post-processing steps, and suitable choices of metrics, we deliver novel insights into why poor detector calibration emerges. Based on these insights, we are able to improve the calibration of a detector by simply finetuning its last layer.

4 May 2021 in Machine Learning
Verified
Journal Article
Open access
PDF file
DKDaniel Kottke
Daniel Kottke
MHMarek Herde
Marek Herde
CSChristoph Sandrock
Christoph Sandrock
DHDenis Huseljic
Denis Huseljic
GKGeorg Krempl
Georg Krempl
BSBernhard Sick
Bernhard Sick
Gathering labeled data to train well-performing machine learning models is one of the critical challenges in many applications. Active learning aims at reducing the labeling costs by an efficient and effective allocation of costly labeling resources. In this article, we propose a decision-theoretic selection strategy that (1) directly optimizes the gain in misclassification error, and (2) uses a Bayesian approach by introducing a conjugate prior distribution to determine the class posterior to deal with uncertainties. By reformulating existing selection strategies within our proposed model, we can explain which aspects are not covered in current state-of-the-art and why this leads to the superior performance of our approach. Extensive experiments on a large variety of datasets and different kernels validate our claims.

5 March 2021 in Preprints.org
Verified
Preprint
Open access
PDF file
DKDaniel Kottke
Daniel Kottke
MHMarek Herde
Marek Herde
MPMinh Tuan Pham
Minh Tuan Pham
ABAlexander Benz
Alexander Benz
PMPascal Mergard
Pascal Mergard
BSBernhard Sick
Bernhard Sick
Machine learning applications often need large amounts of training data to perform well. Whereas unlabeled data can be easily gathered, the labeling process is difficult, time-consuming, or expensive in most applications. Active learning can help solve this problem by querying labels for those data points that will improve the performance the most. Thereby, the goal is that the learning algorithm performs sufficiently well with fewer labels. We provide a library called scikit-activeml that covers the most relevant query strategies and implements tools to work with partially labeled data. It is programmed in Python and builds on top of scikit-learn.

16 October 2024
Verified
Book Chapter
Open access
PDF file
MHMarek Herde
Marek Herde
LLLukas Lührs
Lukas Lührs
DHDenis Huseljic
Denis Huseljic
BSBernhard Sick
Bernhard Sick
Training with noisy class labels impairs neural networks’ generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that multiple annotators, e.g., crowdworkers, typically provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eleven (mostly state-of-the-art) approaches in an evaluation study with eleven datasets comprising noisy class labels from either human or simulated annotators. Our code is publicly available through our GitHub repository at https://github.com/ies-research/multi-annotator-machine-learning/tree/annot-mix.

4 July 2025 in Preprints.org
Verified
Preprint
Open access
PDF file
MHMarek Herde
Marek Herde
MPMinh Tuan Pham
Minh Tuan Pham
DKDaniel Kottke
Daniel Kottke
ABAlexander Benz
Alexander Benz
LLLukas Lührs
Lukas Lührs
BSBernhard Sick
Bernhard Sick
Scikit-activeml is a user-friendly open-source Python library for active learning on top of scikit-learn. Included are implementations of a large collection of query strategies, models, and visualization tools in pool- and stream-based active learning for classification or regression tasks with single or multiple annotators. The flexible design of the active learning cycle enables individual adaptations to a variety of learning scenarios. Our source code with comprehensive documentation is available at https://scikit-activeml.github.io.