Rene Ranftl

Rene Ranftl

Auto-generated profile, based on publicly available information

Rene Ranftl

Intel Labs, Intel, Munich Germany 85579

Research Focus & Metrics

Academic Career Length

14Years (2013-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 August 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
RRRene Ranftl
Rene Ranftl
KLKatrin Lasinger
Katrin Lasinger
DHDavid Hafner
David Hafner
KSKonrad Schindler
Konrad Schindler
VKVladlen Koltun
Vladlen Koltun
The success of monocular depth estimation relies on large and diverse training sets. Due to the challenges associated with acquiring dense ground-truth depth across different environments at scale, a number of datasets with distinct characteristics and biases have emerged. We develop tools that enable mixing multiple datasets during training, even if their annotations are incompatible. In particular, we propose a robust training objective that is invariant to changes in depth range and scale, advocate the use of principled multi-objective learning to combine data from different sources, and highlight the importance of pretraining encoders on auxiliary tasks. Armed with these tools, we experiment with six diverse training datasets, including a new, massive data source: 3D films. To demonstrate the generalization power of our approach we use zero-shot cross-dataset transfer, i.e. we evaluate on datasets that were not seen during training. The experiments confirm that mixing data from complementary sources greatly improves monocular depth estimation. Our approach clearly outperforms competing methods across diverse datasets, setting a new state of the art for monocular depth estimation.

31 December 2019 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
HRHenri Rebecq
Henri Rebecq
RRRene Ranftl
Rene Ranftl
VKVladlen Koltun
Vladlen Koltun
DSDavide Scaramuzza
Davide Scaramuzza
Event cameras are novel sensors that report brightness changes in the form of a stream of asynchronous "events" instead of intensity frames. They offer significant advantages with respect to conventional cameras: high temporal resolution, high dynamic range, and no motion blur. While the stream of events encodes in principle the complete visual signal, the reconstruction of an intensity image from a stream of events is an ill-posed problem in practice. Existing reconstruction approaches are based on hand-crafted priors and strong assumptions about the imaging process as well as the statistics of natural images. In this work we propose to learn to reconstruct intensity images from event streams directly from data instead of relying on any hand-crafted priors. We propose a novel recurrent network to reconstruct videos from a stream of events, and train it on a large amount of simulated event data. During training we propose to use a perceptual loss to encourage reconstructions to follow natural image statistics. We further extend our approach to synthesize color images from color event streams. Our quantitative experiments show that our network surpasses state-of-the-art reconstruction methods by a large margin in terms of image quality (>20%), while comfortably running in real-time. We show that the network is able to synthesize high framerate videos (> 5,000 frames per second) of high-speed phenomena (e.g. a bullet hitting an object) and is able to provide high dynamic range reconstructions in challenging lighting conditions. As an additional contribution, we demonstrate the effectiveness of our reconstructions as an intermediate representation for event data. We show that off-the-shelf computer vision algorithms can be applied to our reconstructions for tasks such as object classification and visual-inertial odometry and that this strategy consistently outperforms algorithms that were specifically designed for event data. We release the reconstruction code, a pre-trained model and the datasets to enable further research.

13 October 2021 in Science Robotics
Verified
Journal Article
Closed access
ALAntonio Loquercio
Antonio Loquercio
EKElia Kaufmann
Elia Kaufmann
RRRené Ranftl
René Ranftl
MMMatthias Müller
Matthias Müller
VKVladlen Koltun
Vladlen Koltun
DSDavide Scaramuzza
Davide Scaramuzza
Deep Learning enables agile flight in challenging environments with onboard sensing and computation.

21 October 2019 in IEEE Transactions on Robotics
Verified
Journal Article
Closed access
ALAntonio Loquercio
Antonio Loquercio
EKElia Kaufmann
Elia Kaufmann
RRRene Ranftl
Rene Ranftl
ADAlexey Dosovitskiy
Alexey Dosovitskiy
VKVladlen Koltun
Vladlen Koltun
DSDavide Scaramuzza
Davide Scaramuzza
Dynamically changing environments, unreliable state estimation, and operation under severe resource constraints are fundamental challenges that limit the deployment of small autonomous drones. We address these challenges in the context of autonomous, vision-based drone racing in dynamic environments. A racing drone must traverse a track with possibly moving gates at high speed. We enable this functionality by combining the performance of a state-of-the-art planning and control system with the perceptual awareness of a convolutional neural network. The resulting modular system is both platform independent and domain independent: it is trained in simulation and deployed on a physical quadrotor without any fine-tuning. The abundance of simulated data, generated via domain randomization, makes our system robust to changes of illumination and gate appearance. To the best of our knowledge, our approach is the first to demonstrate zero-shot sim-to-real transfer on the task of agile drone flight. We extensively test the precision and robustness of our system, both in simulation and on a physical platform, and show significant improvements over the state of the art.

28 January 2019 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
LWLorenz Wellhausen
Lorenz Wellhausen
ADAlexey Dosovitskiy
Alexey Dosovitskiy
RRRene Ranftl
Rene Ranftl
KWKrzysztof Walas
Krzysztof Walas
CCCesar Cadena
Cesar Cadena
MHMarco Hutter
Marco Hutter
Legged robots have the potential to traverse diverse and rugged terrain. To find a safe and efficient navigation path and to carefully select individual footholds, it is useful to be able to predict properties of the terrain ahead of the robot. In this work, we propose a method to collect data from robot-terrain interaction and associate it to images. Using sparse data acquired in teleoperation experiments with a quadrupedal robot, we train a neural network to generate a dense prediction of the terrain properties in front of the robot. To generate training data, we project the foothold positions from the robot trajectory into on-board camera images. We then attach labels to these footholds by identifying the dominant features of the force-torque signal measured with sensorized feet. We show that data collected in this fashion can be used to train a convolutional network for terrain property prediction as well as weakly supervised semantic segmentation. Finally, we show that the predicted terrain properties can be used for autonomous navigation of the ANYmal quadruped robot.

20 January 2020 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
LWLorenz Wellhausen
Lorenz Wellhausen
RRRene Ranftl
Rene Ranftl
MHMarco Hutter
Marco Hutter
Navigation in natural outdoor environments requires a robust and reliable traversability classification method to handle the plethora of situations a robot can encounter. Binary classification algorithms perform well in their native domain but tend to provide overconfident predictions when presented with out-of-distribution samples, which can lead to catastrophic failure when navigating unknown environments. We propose to overcome this issue by using anomaly detection on multi-modal images for traversability classification, which is easily scalable by training in a self-supervised fashion from robot experience. In this work, we evaluate multiple anomaly detection methods with a combination of uni- and multi-modal images in their performance on data from different environmental conditions. Our results show that an approach using a feature extractor and normalizing flow with an input of RGB, depth and surface normals performs best. It achieves over 95% area under the ROC curve and is robust to out-of-distribution samples.