Mustafa Doga Dogan

Mustafa Doga Dogan

Auto-generated profile, based on publicly available information

Mustafa Doga Dogan

Adobe Research, Basel, Switzerland

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 October 2024
Verified
Conference Paper
Open access
MDMustafa Doga Dogan
Mustafa Doga Dogan
EGEric J Gonzalez
Eric J Gonzalez
KAKaran Ahuja
Karan Ahuja
RDRuofei Du
Ruofei Du
ACAndrea Colaço
Andrea Colaço
DKDavid Kim
David Kim
Seamless integration of physical objects as interactive digital entities remains a challenge for spatial computing. This paper explores Augmented Object Intelligence (AOI) in the context of XR, an interaction paradigm that aims to blur the lines between digital and physical by equipping real-world objects with the ability to interact as if they were digital, where every object has the potential to serve as a portal to digital functionalities. Our approach utilizes real-time object segmentation and classification, combined with the power of Multimodal Large Language Models (MLLMs), to facilitate these interactions without the need for object pre-registration. We implement the AOI concept in the form of XR-Objects, an open-source prototype system that provides a platform for users to engage with their physical environment in contextually relevant ways using object-based context menus. This system enables analog objects to not only convey information but also to initiate digital actions, such as querying for details or executing tasks. Our contributions are threefold: (1) we define the AOI concept and detail its advantages over traditional AI assistants, (2) detail the XR-Objects system’s open-source design and implementation, and (3) show its versatility through various use cases and a user study.

29 October 2023
Verified
Conference Paper
Open access
MDMustafa Doga Dogan
Mustafa Doga Dogan
Raul Garcia-MartinRaul Garcia-Martin
Raul Garcia-Martin
PHPatrick William Haertel
Patrick William Haertel
JOJamison John O'KEefe
Jamison John O'KEefe
ATAhmad Taka
Ahmad Taka
SMStefanie Mueller
Stefanie Mueller
Existing invisible object tagging methods are prone to low resolution, which impedes tracking performance. We present BrightMarker, a fabrication method that uses fluorescent filaments to embed easily trackable markers in 3D printed color objects. By using an infrared-fluorescent filament that "shifts" the wavelength of the incident light, our optical detection setup filters out all the noise to only have the markers present in the infrared camera image. The high contrast of the markers allows us to track them robustly regardless of the moving objects’ surface color. We built a software interface for automatically embedding these markers for the input object geometry, and hardware modules that can be attached to existing mobile devices and AR/VR headsets. Our image processing pipeline robustly localizes the markers in real time from the captured images. BrightMarker can be used in a variety of applications, such as custom fabricated wearables for motion capture, tangible interfaces for AR/VR, rapid product tracking, and privacy-preserving night vision. BrightMarker exceeds the detection rate of state-of-the-art invisible marking, and even small markers (1"x1") can be tracked at distances exceeding 2m.

8 October 2023
Verified
Conference Paper
Open access
MDMustafa Doga Dogan
Mustafa Doga Dogan
VCVivian Hsinyueh Chan
Vivian Hsinyueh Chan
RQRichard Qi
Richard Qi
GTGrace Tang
Grace Tang
TRThijs Roumen
Thijs Roumen
SMStefanie Mueller
Stefanie Mueller
We introduce StructCode, a technique to store machine-readable data in laser-cut objects using their fabrication artifacts. StructCode modifies the lengths of laser-cut finger joints and/or living hinges to represent bits of information without introducing additional parts or materials. We demonstrate StructCode through use cases for augmenting laser-cut objects with data such as labels, instructions, and narration. We present and evaluate a tag decoding pipeline that is robust to various backgrounds, viewing angles, and wood types. In our mechanical evaluation, we show that StructCodes preserve the structural integrity of laser-cut objects.

26 February 2025 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
FDFatih Doğangün
Fatih Doğangün
SBSerdar Bahar
Serdar Bahar
YYYigit YILDIRIM
Yigit YILDIRIM
BTBora Toprak Temir
Bora Toprak Temir
EUEmre Ugur
Emre Ugur
MDMustafa Doga Dogan
Mustafa Doga Dogan
This paper introduces Robotic Augmented Reality for Machine Programming by Demonstration (Rampa), the first ML-integrated, XR-driven end-to-end robotic system, allowing training and deployment of ML models such as ProMPs on the fly, and utilizing the capabilities of state-of-the-art and commercially available AR headsets, e.g., Meta Quest 3, to facilitate the application of Programming by Demonstration (PbD) approaches on industrial robotic arms, e.g., Universal Robots UR10. Our approach enables in-situ data recording, visualization, and fine-tuning of skill demonstrations directly within the user's physical environment. Rampa addresses critical challenges of PbD, such as safety concerns, programming barriers, and the inefficiency of collecting demonstrations on the actual hardware. The performance of our system is evaluated against the traditional method of kinesthetic control in teaching three different robotic manipulation tasks and analyzed with quantitative metrics, measuring task performance and completion time, trajectory smoothness, system usability, user experience, and task load using standardized surveys. Our findings indicate a substantial advancement in how robotic tasks are taught and refined, promising improvements in operational safety, efficiency, and user engagement in robotic programming.


11 May 2024
Verified
Conference Paper
Open access
DZDaniel Campos Zamora
Daniel Campos Zamora
MDMustafa Doga Dogan
Mustafa Doga Dogan
ASAlexa F Siu
Alexa F Siu
EKEunyee Koh
Eunyee Koh
CXChang Xiao
Chang Xiao