Multiple mobile robots have gradually played a key role in many industrial systems, such as factory freight logistics system, patrol security systems and multi-robots collaborative service and work. As a key issue in industrial environment perception, the accurate robot localization can enhance their autonomous ability and is an important branch of robotics studies in artificial intelligence. In this paper, we propose a new method for cooperative autonomous localization among air-ground robots in a wide-ranging outdoor industrial environment. The aerial robot first maps an area of interest and achieves self-localization. Then a simplified orthogonal perspective 2.5D map from the aerial robot is transferred to the ground robots for collaboration. Within the collaboration, the ground robot achieves pose estimation with respect to the UAV pose by instantaneously registering a single panorama with respect to the 2.5D map. The 2.5D map is used as the spatial association among air-ground robots. Our method effectively overcomes the dramatic differences between the air-level view and the ground-level view. A set of experiments are performed in the outdoor industrial environment to demonstrate the applicability of our localization method. The proposed robotic collaborative localization outperforms most consumer sensor in accuracy and also has an outstanding running time.

Object-based simultaneous localization and mapping (SLAM) is a more natural and robust way for agents to interact with their surrounding environment. However, it introduces a problem of semantic objects association. Correct object association is the key factor to achieve a successful object SLAM system because object association and SLAM are inherently coupled and have not been well tackled yet. A novel formulation of the object association problem based on a hierarchical Dirichlet process (HDP) is proposed. Through the HDP, we can hierarchically associate the grouped object measurements. This can improve the object association accuracy and computation efficiency. Thanks to the novel formulation, the proposed method is also able to correct failure object associations according to its sampling inference algorithm. Furthermore, we introduce object poses to the processing of pose optimization. The object association and pose optimization are then solved in a tightly coupled way, by which both aspects can promote each other. The proposed method is evaluated on indoor and outdoor datasets and the experimental results show a very impressive improvement with respect to the traditional SLAM.

Obtaining 3-D data by LIDAR from unmanned aerial vehicles (UAVs) is vital for the field of remote sensing; however, the highly dynamic movement of UAVs and narrow viewpoint of LIDAR pose a great challenge to the self-localization for UAVs based on solely LIDAR sensor. To this end, we propose a robust simultaneous localization and mapping (SLAM) system, which combines the image data obtained by vision sensor and point clouds obtained by LIDAR. In the front-end of the proposed system, the more stable line and plane features are extracted from point clouds through clustering. Then the relative pose between two consecutive frames is computed by the least squares iterative closest point algorithm. Afterward, a novel direct odometry algorithm is developed by combining the image frames and sparse point clouds, where the relative pose is used as a prior. In the back-end, the pose estimation is refined and the 3-D map with texture information is built at a lower frequency. Extensive experiments show that our method can achieve robust and highly precise localization and mapping for UAVs.

Eye gaze is an important natural behavior in social interaction as it delivers complex exchanges between observer and observed, by building up the geometric constraints and relation of the exchanges. These interperson exchanges can be modeled based on gaze direction estimated using computer vision. Despite significant progresses in vision-based gaze estimation in last 10 years, it is still nontrivial since the accuracy of gaze estimation is significantly affected by such intrinsic factors as head pose variance, individual bias between optical axis and visual axis, eye blink, occlusion and image blur, degrade gaze features, lead to inaccurate gaze-involved human social interaction analysis. This article aims to review and discuss existing methods addressing above-mentioned problems, gaze involved applications and data sets against the state of the arts in vision-based gaze estimation. It also points out future research directions and challenges of gaze estimation in terms of metalearning, causal inference, disentangled representation, and social gaze behavior for unconstrained gaze estimation.

Visual Simultaneous Localization and Mapping (vSLAM) is a cornerstone technology in computer vision and robotics, underpinning applications such as autonomous vehicles and robot navigation. While traditional vSLAM systems have shown significant progress in indoor or outdoor environments, their performance often degrades in complex scenes, limiting their adaptability and robustness. Semantic vSLAM, which integrates high-level semantic information into vSLAM systems, has emerged as a promising solution to address these limitations by enabling a richer understanding of the environment. In this paper, we provide a comprehensive review of semantic vSLAM, offering a critical analysis of its evolution, methods, and challenges. We begin by revisiting the development of traditional vSLAM, emphasizing its limitations and the motivation for incorporating semantic information. Subsequently, we delve into the core modules of semantic vSLAM, including semantic extraction, object association, semantic loop closing, back-end optimization, and semantic mapping. Then, we present a performance comparison of semantic vSLAM systems under two different datasets, indoor and outdoor, respectively. Furthermore, we also provide a comparative analysis of widely used SLAM datasets to provide guidance for performance testing and validation. To further enrich the discussion, we identify unresolved challenges in semantic vSLAM, such as long-term semantic perception and association, open and unstructured environments. We propose future research directions, including balancing computational resources and quantifying system risk, large model-based navigation and mapping, and embodied AI SLAM. By providing key insights and forward-looking perspectives, this work aims to stimulate future research and improve the capabilities of semantic vSLAM in real-world applications.

The map recovery and fusion is a key issue in the application of large scale and long-term AR scenarios. However, they are still not addressed well in an efficient and precise way, especially for complex industrial environments. In this study, we propose a map recovery and fusion strategy based on VI SLAM. We first develop a heuristic strategy that can fast search and match map points among multiple maps, and can be used for efficient map fusion. For map recovery, we leverage the inertial sensors for short time motion estimation, and transform the previous lost map to the current map. Based on this, a novel framework for collaborative AR is implemented and parallelly runs in multiple mobile devices in real time. Extensive experiments have been carried out on a public dataset. The results show that the proposed method can recovery and fuse multiple maps with high completeness and precision.