Helge Rhodin

Prof. Helge Rhodin

orcid

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 October 2019 in eLife
Verified
Journal Article
Open access
SGSemih Günel
Semih Günel
HRHelge Rhodin
Helge Rhodin
DMDaniel Morales
Daniel Morales
JCJoão Campagnolo
João Campagnolo
PRPavan Ramdya
Pavan Ramdya
PFPascal Fua
Pascal Fua
Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in three-dimensional (3D) space. Deep neural networks can estimate two-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here, we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.


15 February 2019 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
WXWeipeng Xu
Weipeng Xu
ACAvishek Chatterjee
Avishek Chatterjee
MZMichael Zollhofer
Michael Zollhofer
HRHelge Rhodin
Helge Rhodin
PFPascal Fua
Pascal Fua
CTChristian Theobalt
Christian Theobalt
We propose the first real-time system for the egocentric estimation of 3D human body pose in a wide range of unconstrained everyday activities. This setting has a unique set of challenges, such as mobility of the hardware setup, and robustness to long capture sessions with fast recovery from tracking failures. We tackle these challenges based on a novel lightweight setup that converts a standard baseball cap to a device for high-quality pose estimation based on a single cap-mounted fisheye camera. From the captured egocentric live stream, our CNN based 3D pose estimation approach runs at 60 Hz on a consumer-level GPU. In addition to the lightweight hardware setup, our other main contributions are: 1) a large ground truth training corpus of top-down fisheye images and 2) a disentangled 3D pose estimation approach that takes the unique properties of the egocentric viewpoint into account. As shown by our evaluation, we achieve lower 3D joint error as well as better 2D overlay than the existing baselines.

5 August 2021 in Nature Methods
Verified
Journal Article
Closed access
PDF file
AGAdam Gosztolai
Adam Gosztolai
SGSemih Günel
Semih Günel
VLVictor Lobato-Ríos
Victor Lobato-Ríos
MAMarco Pietro Abrate
Marco Pietro Abrate
DMDaniel Morales
Daniel Morales
PRPavan Ramdya
Pavan Ramdya

5 April 2021 in Nature Methods
Verified
Journal Article
Closed access
PDF file
LBLuis A. Bolaños
Luis A. Bolaños
DXDongsheng Xiao
Dongsheng Xiao
NFNancy L. Ford
Nancy L. Ford
JLJeff M. LeDue
Jeff M. LeDue
PGPankaj K. Gupta
Pankaj K. Gupta
TMTimothy H. Murphy
Timothy H. Murphy

6 October 2019 in Sensors
Verified
Journal Article
Open access
PDF file
MOMirela Ostrek
Mirela Ostrek
HRHelge Rhodin
Helge Rhodin
PFPascal Fua
Pascal Fua
Erich MüllerErich Müller
Erich Müller
Jörg SpörriJörg Spörri
Jörg Spörri
In this study, we compared a monocular computer vision (MCV)-based approach with the golden standard for collecting kinematic data on ski tracks (i.e., video-based stereophotogrammetry) and assessed its deployment readiness for answering applied research questions in the context of alpine skiing. The investigated MCV-based approach predicted the three-dimensional human pose and ski orientation based on the image data from a single camera. The data set used for training and testing the underlying deep nets originated from a field experiment with six competitive alpine skiers. The normalized mean per joint position error of the MVC-based approach was found to be 0.08 ± 0.01 m. Knee flexion showed an accuracy and precision (in parenthesis) of 0.4 ± 7.1° (7.2 ± 1.5°) for the outside leg, and −0.2 ± 5.0° (6.7 ± 1.1°) for the inside leg. For hip flexion, the corresponding values were −0.4 ± 6.1° (4.4° ± 1.5°) and −0.7 ± 4.7° (3.7 ± 1.0°), respectively. The accuracy and precision of skiing-related metrics were revealed to be 0.03 ± 0.01 m (0.01 ± 0.00 m) for relative center of mass position, −0.1 ± 3.8° (3.4 ± 0.9) for lean angle, 0.01 ± 0.03 m (0.02 ± 0.01 m) for center of mass to outside ankle distance, 0.01 ± 0.05 m (0.03 ± 0.01 m) for fore/aft position, and 0.00 ± 0.01 m2 (0.01 ± 0.00 m2) for drag area. Such magnitudes can be considered acceptable for detecting relevant differences in the context of alpine skiing.