Maximilian Gilles

Maximilian Gilles

Auto-generated profile, based on publicly available information

Maximilian Gilles

Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 November 2023 in IEEE Transactions on Automation Science and Engineering
Verified
Journal Article
Open access
MGMaximilian Gilles
Maximilian Gilles
YCYuhao Chen
Yuhao Chen
EZE. Zhixuan Zeng
E. Zhixuan Zeng
YWYifan Wu
Yifan Wu
KFKai Furmans
Kai Furmans
RRRania Rayyes
Rania Rayyes
Grasping unknown objects in unstructured environments is one of the most challenging and demanding tasks for robotic bin picking systems. Developing a holistic approach is crucial to building such dexterous bin picking systems to meet practical requirements on speed, cost and reliability. Proposed datasets so far focus only on challenging sub-problems and are therefore limited in their ability to leverage the complementary relationship between individual tasks. In this paper, we tackle this holistic data challenge and design MetaGraspNetV2, an all-in-one bin picking dataset consisting of (i) a photo-realistic dataset with over 296k images, which has been created through physics-based metaverse synthesis; and (ii) a real-world test dataset with 3.2k images featuring task-specific difficulty levels. Both datasets provide full annotations for amodal panoptic segmentation, object relationship detection, occlusion reasoning, 6-DoF pose estimation, and grasp detection for a parallel-jaw as well as a vacuum gripper. Extensive experiments demonstrate that our dataset outperforms state-of-the-art datasets in object detection, instance segmentation, amodal detection, parallel-jaw grasping, and vacuum grasping. Furthermore, leveraging the potential of our data for building holistic perception systems, we propose a single-shot-multi-pick (SSMP) grasping policy for scene understanding accelerated fast picking in high clutter. SSMP reasons about suitable manipulation orders for blindly picking multiple items given a single image acquisition. Physical robot experiments demonstrate that SSMP effectively speeds up cycle times through reducing image acquisitions by more than 47% while providing better grasp performance compared to state-of-the-art bin picking methods. Note to Practitioners —In robotic bin picking, most proposed methods and datasets focus on solving only one aspect of the grasping task, such as grasp point detection, object detection, or relationship reasoning. They do not address practical aspects such as the widespread use of vacuum grasp technology or the need for short cycle times. In practice, however, efficient bin picking solutions often rely on multiple task-specific methods. Hence, having one dataset for a large variety of vision-related tasks in robotic picking reduces data redundancy and enables the development of holistic methods. While deep learning has been proven highly effective for bin picking vision systems, it demands large, high-quality training datasets. Collecting such datasets in the real-world, while assuring label quality and consistency, is prohibitively expensive and time-consuming. To overcome these challenges, we set up a photo-realistic metaverse data generation pipeline and create a large-scale synthetic training dataset. Furthermore, we design a comprehensive real-world dataset for testing. Unlike previously proposed datasets, our datasets provide difficulty levels and annotations in simulation and real-world for a comprehensive list of high-level tasks, including amodal object detection, scene layout reasoning, and grasp detection. In real-world applications, cycle time is a critical factor affecting the productivity and profitability of a robotic system. We tackle time-efficiency through scene understanding and demonstrate the capability of our data regarding holistic system development by proposing a single-shot-multi-pick (SSMP) policy. Our SSMP algorithm, trained exclusively on our synthetic data, distinguishes between uncovered and occluded items, and infers specific manipulation orders to perform multiple blind picks in a single shot. Physical robot experiments show that SSMP was able to reduce image acquisitions by more than 47% without compromising grasp performance. This clearly demonstrates that SSMP, together with our dataset, paves the way for application-oriented research in time-critical bin picking.

20 August 2022
Verified
Conference Paper
Open access
MGMaximilian Gilles
Maximilian Gilles
YCYuhao Chen
Yuhao Chen
TWTim Robin Winter
Tim Robin Winter
EZE. Zhixuan Zeng
E. Zhixuan Zeng
Alexander WongAlexander Wong
Alexander Wong
Autonomous bin picking poses significant challenges to vision-driven robotic systems given the complexity of the problem, ranging from various sensor modalities, to highly entangled object layouts, to diverse item properties and gripper types. Existing methods often address the problem from one perspective. Diverse items and complex bin scenes require diverse picking strategies together with advanced reasoning. As such, to build robust and effective machine-learning algorithms for solving this complex task requires significant amounts of comprehensive and high quality data. Collecting such data in real world would be too expensive and time prohibitive and therefore intractable from a scalability perspective. To tackle this big, diverse data problem, we take inspiration from the recent rise in the concept of metaverses, and introduce MetaGraspNet, a large-scale photo-realistic bin picking dataset constructed via physics-based metaverse synthesis. The pro-posed dataset contains 217k RGBD images across 82 different article types, with full annotations for object detection, amodal perception, keypoint detection, manipulation order and ambidextrous grasp labels for a parallel-jaw and vacuum gripper. We also provide a real dataset consisting of over 2.3k fully annotated high-quality RGBD images, divided into 5 levels of difficulties and an unseen object set to evaluate different object and layout properties. Finally, we conduct extensive experiments showing that our proposed vacuum seal model and synthetic dataset achieves state-of-the-art performance and generalizes to real world use-cases.

19 February 2025 in IEEE Robotics and Automation Letters
Verified
Journal Article
Open access
MGMaximilian Gilles
Maximilian Gilles
KFKai Furmans
Kai Furmans
RRRania Rayyes
Rania Rayyes
Learning-based robotic grasping systems typically rely on large-scale datasets for training. However, collecting such datasets in the real-world is both costly and time-consuming. Synthetic data generation data is a cost-effective alternative, but models trained solely on synthetic data often struggle with zero-shot real-world performance due to the large domain gap between synthetic and real-world data. To address this challenge of dataset costs against model performance, we propose an active learning framework designed for fast and sample-efficient sim-to-real domain adaptation. Our proposed learning framework uses synthetic data as initial knowledge base and incrementally adapts to the target data domain by selecting the most informative real-world data samples for further model training. For this purpose, we propose a novel, hybrid query strategy, MetaMVUC, which leverages multi-view uncertainty and metadata diversity. MetaMVUC assesses model uncertainty by comparing model predictions across multiple viewpoints, identifying samples with the highest uncertainty. Additionally, since robots in industry or logistics often operate in environments rich in metadata, MetaMVUC leverages this information to ensure diverse and well-distributed sample selection. Experimental results demonstrate the effectiveness of our proposed learning framework. With a limited annotation budget of 16 samples, a robot trained using MetaMVUC achieves a successful grasp rate of $87.7\%$. Increasing the budget to 40 samples improves grasp performance to $96.7\%$, outperforming the zero-shot sim-to-real by $17.4\%$ and $26.4\%$, respectively.

23 September 2022 in Advanced Robotics
Verified
Journal Article
Closed access
BZBenchun Zhou
Benchun Zhou
MGMaximilian Gilles
Maximilian Gilles
YMYongqi Meng
Yongqi Meng
Simultaneous localization and mapping (SLAM) is a fundamental problem for indoor mobile robots operating in unknown environments. While visual SLAM systems often use geometry features, the reconstructed maps lack semantic information. On the other hand, current object detection methods provide rich information about the scene from the image. In this paper, we present a structure SLAM system with feature points, geometry planes, and semantic objects. Unlike other systems modeling planes and objects as collections of points, we choose a parametric representation for these landmarks. For every single frame, we start by generating cuboid candidates of detected objects with varying dimensions and orientations, then use 2D-3D fitting constraints to calculate the cuboid's translation, and finally introduce 3D spatial and 2D image constraints to select the best cuboid candidate. For SLAM optimization, the detected planes and objects can provide geometry constraints to improve the localization result, and act as landmarks to reconstruct a semantic map. Experiments on the ICL NUIM RGB-D dataset show that the proposed point-plane-object SLAM system can slightly improve localization accuracy, and is able to build a semantic map of the environment. GRAPHICAL ABSTRACT

16 July 2024 in IEEE Access
Verified
Journal Article
Open access
DMDayou Mao
Dayou Mao
YCYuhao Chen
Yuhao Chen
YWYifan Wu
Yifan Wu
MGMaximilian Gilles
Maximilian Gilles
Alexander WongAlexander Wong
Alexander Wong
The core idea of Multi-Task Learning (MTL) is to develop neural networks with a shared feature extraction backbone and multiple prediction heads, each capable of inferring a different task simultaneously. Parameters in the backbone contribute to all tasks while those in the prediction heads contribute to only one or fewer tasks. Challenges arise when multiple tasks compete for resource. Existing methods focus on resource competition in shared parameters and proposed explanatory factors of task conflicts, task dominance, and gradient stability. However the fundamental nature of MTL is still understudied. In this paper, instead of following the existing methodology research directions, we carry out large-scale empirical study and provide deeper insight on understanding MTL. In particular, instead of focusing on resource competition in the shared parameters in the backbone, we shift our attention to resource competition in the backbone output, which is the embedded representation that is shared by all prediction heads. We show that the existing explanatory factors display weak causal relationship with model performance. We propose a novel measurement, which we term Feature Disentanglement, and show that understanding MTL problems from the perspective of how the shared representation is leveraged by different prediction heads, is a more faithful and reliable way than that from the perspective of how supervision signals from different tasks are interfering in the shared parameters. Additionally, it has been a commonly employed technique to replace gradients w.r.t. shared parameters with gradients w.r.t. shared representation for reduced computation.We conduct a comprehensive study and show that unless a theoretical analysis could be developed, there is not general guarantee that this fast approximation technique would work in practice.

27 November 2021 in The Visual Computer
Verified
Journal Article
Open access
PDF file
MGMaximilian Gilles
Maximilian Gilles
SISascha Ibrahimpasic
Sascha Ibrahimpasic
Knowing the robot's pose is a crucial prerequisite for mobile robot tasks such as collision avoidance or autonomous navigation. Using powerful predictive models to estimate transformations for visual odometry via downward facing cameras is an understudied area of research. This work proposes a novel approach based on deep learning for estimating ego motion with a downward looking camera. The network can be trained completely unsupervised and is not restricted to a specific motion model. We propose two neural network architectures based on the Early Fusion and Slow Fusion design principle: “EarlyBird” and “SlowBird”. Both networks share a Spatial Transformer layer for image warping and are trained with a modified structural similarity index (SSIM) loss function. Experiments carried out in simulation and for a real world differential drive robot show similar and partially better results of our proposed deep learning based approaches compared to a state-of-the-art method based on fast Fourier transformation.