Rainer Stiefelhagen

Prof. Dr. Rainer Stiefelhagen


Research Focus & Metrics

Academic Career Length

23Years (2004-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 October 2012
Verified
Book Chapter
Closed access
PDF file
HKHilde Kuehne
Hilde Kuehne
HJHueihan Jhuang
Hueihan Jhuang
RSRainer Stiefelhagen
Rainer Stiefelhagen
TSThomas Serre
Thomas Serre

3 June 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
JZJiaming Zhang
Jiaming Zhang
KYKailun Yang
Kailun Yang
HSHao Shi
Hao Shi
SRSimon Reiß
Simon Reiß
KPKunyu Peng
Kunyu Peng
RSRainer Stiefelhagen
Rainer Stiefelhagen
In this paper, we address panoramic semantic segmentation which is under-explored due to two critical challenges: (1) image distortions and object deformations on panoramas; (2) lack of semantic annotations in the $360^\circ$ imagery. To tackle these problems, first, we propose the upgraded Transformer for Panoramic Semantic Segmentation, i.e., Trans4PASS+, equipped with Deformable Patch Embedding (DPE) and Deformable MLP (DMLPv2) modules for handling object deformations and image distortions whenever (before or after adaptation) and wherever (shallow or deep levels). Second, we enhance the Mutual Prototypical Adaptation (MPA) strategy via pseudo-label rectification for unsupervised domain adaptive panoramic segmentation. Third, aside from Pinhole-to-Panoramic ( Pin2Pan ) adaptation, we create a new dataset (SynPASS) with 9,080 panoramic images, facilitating Synthetic-to-Real ( Syn2Real ) adaptation scheme in $360^\circ$ imagery. Extensive experiments are conducted, which cover indoor and outdoor scenarios, and each of them is investigated with Pin2Pan and Syn2Real regimens. Trans4PASS+ achieves state-of-the-art performances on four domain adaptive panoramic semantic segmentation benchmarks.

19 February 2023
Verified
Book Chapter
Closed access
PDF file
PWPing-Cheng Wei
Ping-Cheng Wei
KPKunyu Peng
Kunyu Peng
ARAlina Roitberg
Alina Roitberg
KYKailun Yang
Kailun Yang
JZJiaming Zhang
Jiaming Zhang
RSRainer Stiefelhagen
Rainer Stiefelhagen

30 August 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
ZMZdravko Marinov
Zdravko Marinov
PJPaul F. Jäger
Paul F. Jäger
JEJan Egger
Jan Egger
JKJens Kleesiek
Jens Kleesiek
RSRainer Stiefelhagen
Rainer Stiefelhagen
Interactive segmentation is a crucial research area in medical image analysis aiming to boost the efficiency of costly annotations by incorporating human feedback. This feedback takes the form of clicks, scribbles, or masks and allows for iterative refinement of the model output so as to efficiently guide the system towards the desired behavior. In recent years, deep learning-based approaches have propelled results to a new level causing a rapid growth in the field with 121 methods proposed in the medical imaging domain alone. In this review, we provide a structured overview of this emerging field featuring a comprehensive taxonomy, a systematic review of existing methods, and an in-depth analysis of current practices. Based on these contributions, we discuss the challenges and opportunities in the field. For instance, we find that there is a severe lack of comparison across methods which needs to be tackled by standardized baselines and benchmarks.

23 September 2024 in IEEE Transactions on Intelligent Transportation Systems
Verified
Journal Article
Closed access
RLRuiping Liu
Ruiping Liu
KYKailun Yang
Kailun Yang
ARAlina Roitberg
Alina Roitberg
JZJiaming Zhang
Jiaming Zhang
KPKunyu Peng
Kunyu Peng
RSRainer Stiefelhagen
Rainer Stiefelhagen
Semantic segmentation benchmarks in the realm of autonomous driving are dominated by large pre-trained transformers, yet their widespread adoption is impeded by substantial computational costs and prolonged training durations. To lift this constraint, we look at efficient semantic segmentation from a perspective of comprehensive knowledge distillation and aim to bridge the gap between multi-source knowledge extractions and transformer-specific patch embeddings. We put forward the Transformer-based Knowledge Distillation (TransKD) framework which learns compact student transformers by distilling both feature maps and patch embeddings of large teacher transformers, bypassing the long pre-training process and reducing the FLOPs by ${>}85.0\%$ . Specifically, we propose two fundamental modules to realize feature map distillation and patch embedding distillation, respectively: 1) Cross Selective Fusion (CSF) enables knowledge transfer between cross-stage features via channel attention and feature map distillation within hierarchical transformers; 2) Patch Embedding Alignment (PEA) performs dimensional transformation within the patchifying process to facilitate the patch embedding distillation. Furthermore, we introduce two optimization modules to enhance the patch embedding distillation from different perspectives: 1) Global-Local Context Mixer (GL-Mixer) extracts both global and local information of a representative embedding; 2) Embedding Assistant (EA) acts as an embedding method to seamlessly bridge teacher and student models with the teacher’s number of channels. Experiments on Cityscapes, ACDC, NYUv2, and Pascal VOC2012 datasets show that TransKD outperforms state-of-the-art distillation frameworks and rivals the time-consuming pre-training method. The source code is publicly available at https://github.com/RuipingL/TransKD.

12 August 2024 in IEEE Transactions on Intelligent Transportation Systems
Verified
Journal Article
Closed access
LJLin jiacheng
Lin jiacheng
JCJiajun Chen
Jiajun Chen
KPKunyu Peng
Kunyu Peng
XHXuan He
Xuan He
LZLi Zhiyong
Li Zhiyong
KYKailun Yang
Kailun Yang
This paper introduces the task of Auditory Referring Multi-Object Tracking (AR-MOT), which dynamically tracks specific objects in a video sequence based on audio expressions and appears as a challenging problem in autonomous driving. Due to the lack of semantic modeling capacity in audio and video, existing works have mainly focused on text-based multi-object tracking, which often comes at the cost of tracking quality, interaction efficiency, and even the safety of assistance systems, limiting the application of such methods in autonomous driving. In this paper, we delve into the problem of AR-MOT from the perspective of audio-video fusion and audio-video tracking. We put forward EchoTrack, an end-to-end AR-MOT framework with dual-stream vision transformers. The dual streams are intertwined with our Bidirectional Frequency-domain Cross-attention Fusion Module (Bi-FCFM), which bidirectionally fuses audio and video features from both frequency-and spatiotemporal domains. Moreover, we propose the Audio-visual Contrastive Tracking Learning (ACTL) regime to extract homogeneous semantic features between expressions and visual objects by learning homogeneous features between different audio and video objects effectively. Aside from the architectural design, we establish the first set of large-scale AR-MOT benchmarks, including Echo-KITTI, Echo-KITTI $+$ , and Echo-BDD. Extensive experiments on the established benchmarks demonstrate the effectiveness of the proposed EchoTrack and its components. The source code and datasets are available at https://github.com/lab206/EchoTrack.