Luis Falconeri Coelho

Luis Falconeri Coelho

Auto-generated profile, based on publicly available information

Luis Falconeri Coelho

Porsche AG, CODE University, Berlin, Germany

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 October 2021
Verified
Conference Paper
Closed access
MKMohamed Kari
Mohamed Kari
TGTobias Grosse-Puppendahl
Tobias Grosse-Puppendahl
LCLuis Falconeri Coelho
Luis Falconeri Coelho
AFAndreas Rene Fender
Andreas Rene Fender
DBDavid Bethge
David Bethge
CHChristian Holz
Christian Holz
Despite the advances in machine perception, semantic scene understanding is still a limiting factor in mixed reality scene composition. In this paper, we present TransforMR, a video see-through mixed reality system for mobile devices that performs 3D-pose-aware object substitution to create meaningful mixed reality scenes. In real-time and for previously unseen and unprepared real-world environments, TransforMR composes mixed reality scenes so that virtual objects assume behavioral and environment-contextual properties of replaced real-world objects. This yields meaningful, coherent, and humaninterpretable scenes, not yet demonstrated by today’s augmentation techniques. TransforMR creates these experiences through our novel pose-aware object substitution method building on different 3D object pose estimators, instance segmentation, video inpainting, and pose-aware object rendering. TransforMR is designed for use in the real-world, supporting the substitution of humans and vehicles in everyday scenes, and runs on mobile devices using just their monocular RGB camera feed as input. We evaluated TransforMR with eight participants in an uncontrolled city environment employing different transformation themes. Applications of TransforMR include real-time character animation analogous to motion capturing in professional film making, however without the need for preparation of either the scene or the actor, as well as narrative-driven experiences that allow users to explore fictional parallel universes in mixed reality. We make all of our source code and assets available 1 . 1 TransforMR code release: https://github.com/MohamedKari/transformr

21 December 2022 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
DBDavid Bethge
David Bethge
LCLuis Falconeri Coelho
Luis Falconeri Coelho
Thomas KoschThomas Kosch
Thomas Kosch
SMSatiyabooshan Murugaboopathy
Satiyabooshan Murugaboopathy
UZUlrich von Zadow
Ulrich von Zadow
TGTobias Grosse-Puppendahl
Tobias Grosse-Puppendahl
Driver emotions play a vital role in driving safety and performance. Consequently, regulating driver emotions through empathic interfaces have been investigated thoroughly. However, the prerequisite - driver emotion sensing - is a challenging endeavor: Body-worn physiological sensors are intrusive, while facial and speech recognition only capture overt emotions. In a user study (N=27), we investigate how emotions can be unobtrusively predicted by analyzing a rich set of contextual features captured by a smartphone, including road and traffic conditions, visual scene analysis, audio, weather information, and car speed. We derive a technical design space to inform practitioners and researchers about the most indicative sensing modalities, the corresponding impact on users' privacy, and the computational cost associated with processing this data. Our analysis shows that contextual emotion recognition is significantly more robust than facial recognition, leading to an overall improvement of 7% using a leave-one-participant-out cross-validation.