Traditional mapping modules in Visual Simultaneous Localization and Mapping (i.e. Visual SLAM) systems can only estimate 3D information of isolated sparse or semi-dense feature points. But there are lots of object instances in the environments which geometric information can be utilized to enhance the quality of mapping and localization. Hence, it is required for the Visual SLAM system to utilize high-dimensional features like object instances or structural lines in mapping and localization. To meet the gap between the above requirements and the traditional implementation of Visual SLAM systems, we present in this paper a novel Visual SLAM method that can effectively utilize texture-less object instances for mapping and localization. The proposed Visual SLAM method includes newly designed feature extraction, matching, localization and mapping modules, which jointly use object features and point features to estimate camera 6-DOF poses and do richer map construction. A group of organized raster points is used to represent objects during feature matching and pose estimation process in the proposed Visual SLAM pipeline. Owing to the object feature fusion in the co-visibility graph it could conduct scale aware bundle adjustments to reduce accumulated error. The advantages of proposed Visual SLAM method are demonstrated through experiments conducted both on synthetic datasets and real-world datasets.

Service robots across industrial, medical, and domestic environments face significant challenges in flattening garments, due to their nearly infinite degrees of freedom and complex dynamic properties. Traditional quasi-static methods for handling fabrics require extensive interactions and are inefficient. While high-velocity dynamic actions enable efficient unfolding, they are unable to perform fine-grained adjustments. To address this, we propose a self-supervised learning framework combining dynamic (Fling) and static (pick and place and pick and drag) manipulation primitives using a dual-arm robot. This method unfolds crumpled garments efficiently without expert demonstrations or hand-labeled data. We introduce a novel factorized reward function based on garment coverage, shape matching, and smoothness. In addition, we employ a spatial action maps network, leveraging a Vision Transformer, to accurately determine grasping points. Our approach significantly reduces interaction steps and improves fabric coverage. We conducted simulations and real-world experiments on 14 garments across six categories. Our approach quickly achieved over 80% coverage within just two actions on long sleeves, surpassing the state-of-the-art by about 9%, and reached 88% –92.9% coverage across all categories within five actions. Our zero-shot sim-to-real transfer has successfully been demonstrated on UR5 robots, proving the effectiveness of the model on various fabrics.

A reliable and accurate visual object 6DoF pose tracking system for texture-less objects plays an important role in various fields of modern industry hence it has been a hot research topic for decades. Traditional feature-based pose tracking methods require rich feature points on objects, and it cannot handle texture-less objects. To tackle this problem, the paper proposes a novel edge-based method for continuous 6DOF pose tracking of texture-less objects. The pose of the object is estimated by minimizing the matching error between extracted image edges and re-projected CAD model edges. The matching error is represented using the Directional Chamfer Matching (DCM) Tensor. Compared with previous methods, the proposed system improves the overall performance of pose tracking system in two ways. Firstly, the method proposes an analytical mathematic model in the optimization process; Secondly, the method proposes an adaptive edge point weighting algorithm to tackle the occlusion or edge weakness problem. Both methods help improve the accuracy and robustness of the pose estimation system. With the benefit of GPU acceleration on DCM Tensor calculation the proposed method could run in real time. Extensive experiments are conducted both on public datasets and CG-rendered synthetic datasets to validate the accuracy and robustness of the proposed algorithm against other state-of-the-art methods.

Visual simultaneous localization and mapping (VSLAM) is an appropriate method for positioning and navigation of intelligent unmanned systems under Global Navigation Satellite Systems (GNSS)-denied environment, but it is still facing some dilemmas in repetitive large-scale environments. In this article, a VSLAM method based on bifocal-binocular vision is proposed. By introducing the binocular camera with different focal lengths, the perception ability of the system in vast space is improved as the designed cameras could complement each other at different working distances. Meanwhile, considering the inherent structure of the scene, additional optimization is proposed to reduce the accumulated error based on the markers distribution knowledge obtained from online placement inference. The algorithm proposed in this article significantly improves the stability and accuracy of the VSLAM system in repetitive large-scale scenes, and is validated in both virtual datasets and real-world environments.

Robotic cloth manipulation poses significant challenges due to the fabric’s complex dynamics and the high dimensionality of configuration spaces. Previous approaches have focused on isolated smoothing or folding tasks and relied heavily on simulations, often struggling to bridge the sim-to-real gap. This gap arises as simulated cloth dynamics fail to capture real-world properties such as elasticity, friction, and occlusions, causing accuracy loss and limited generalization. To tackle these challenges, we propose a two-stream architecture with sequential and spatial pathways, unifying smoothing and folding tasks into a single adaptable policy model. The sequential stream determines pick-and-place positions, while the spatial stream, using a connectivity dynamics model, constructs a visibility graph from partial point cloud data, enabling the model to infer the cloth’s full configuration despite occlusions. To address the sim-to-real gap, we integrate real-world human demonstration data via a hand-tracking detection algorithm, enhancing real-world performance across diverse cloth configurations. Our method, validated on a UR5 robot across six distinct cloth folding tasks, consistently achieves desired folded states from arbitrary crumpled initial configurations, with success rates of 100.0%, 100.0%, 83.3%, 66.7%, 83.3%, and 66.7%. It outperforms state-of-the-art cloth manipulation techniques and generalizes to unseen fabrics with diverse colors, shapes, and stiffness. Project page: https://zcswdt.github.io/SSFold/.