3D visual recognition is a prerequisite for most autonomous robotic systems operating in the real world. It empowers robots to perform a variety of tasks, such as tracking, understanding the environment, and human–robot interaction. Autonomous robots equipped with 3D recognition capability can better perform their social roles through supportive task assistance in professional jobs and effective domestic services. For active assistance, social robots must recognize their surroundings, including objects and places to perform the task more efficiently. This article first highlights the value-centric role of social robots in society by presenting recently developed robots and describes their main features. Instigated by the recognition capability of social robots, we present the analysis of data representation methods based on sensor modalities for 3D object and place recognition using deep learning models. In this direction, we delineate the research gaps that need to be addressed, summarize 3D recognition datasets, and present performance comparisons. Finally, a discussion of future research directions concludes the article. This survey is intended to show how recent developments in 3D visual recognition based on sensor modalities using deep-learning-based approaches can lay the groundwork to inspire further research and serves as a guide to those who are interested in vision-based robotics applications.

Multi-robot systems have been used in many fields by utilizing parallel working robots to perform missions by allocating tasks and cooperating. For task planning, multi-robot systems need to solve complex problems that simultaneously consider the movement of the robots and the influence of each robot. For this purpose, researchers have proposed various methods for modeling and planning multi-robot missions. In particular, some approaches have been presented for high-level task planning by introducing semantic knowledge, such as relationships and domain rules, for environmental factors. This paper proposes a semantic knowledge-based hierarchical planning approach for multi-robot systems. We extend the semantic knowledge by considering the influence and interaction between environmental elements in multi-robot systems. Relationship knowledge represents the space occupancy of each environmental element and the possession of objects. Additionally, the knowledge property is defined to express the hierarchical information of each space. Based on the suggested semantic knowledge, the task planner utilizes spatial hierarchy knowledge to group the robots and generate optimal task plans for each group. With this approach, our method efficiently plans complex missions while handling overlap and deadlock problems among the robots. The experiments verified the feasibility of the suggested semantic knowledge and demonstrated that the task planner could reduce the planning time in simulation environments.

Autonomous driving and its real-world implementation have been among the most actively studied topics in the past few years. In recent years, this growth has been accelerated by the development of advanced deep learning-based data processing technologies. Moreover, large automakers manufacture vehicles that can achieve partially or fully autonomous driving for driving on real roads. However, self-driving cars are limited to some areas with multi-lane roads, such as highways, and self-driving cars that drive in urban areas or residential complexes are still in the development stage. Among autonomous vehicles for various purposes, this paper focused on the development of autonomous vehicles for garbage collection in residential areas. Since we set the target environment of the vehicle as a residential complex, there is a difference from the target environment of a general autonomous vehicle. Therefore, in this paper, we defined ODD, including vehicle length, speed, and driving conditions for the development vehicle to drive in a residential area. In addition, to recognize the vehicle’s surroundings and respond to various situations, it is equipped with various sensors and additional devices that can notify the outside of the vehicle’s state or operate it in an emergency. In addition, an autonomous driving system capable of object recognition, lane recognition, route planning, vehicle manipulation, and abnormal situation detection was configured to suit the vehicle hardware and driving environment configured in this way. Finally, by performing autonomous driving in the actual experimental section with the developed vehicle, it was confirmed that the function of autonomous driving in the residential area works appropriately. Moreover, we confirmed that this vehicle would support garbage collection works through the experiment of work efficiency.

To achieve Level 4 and above autonomous driving, a robust and stable autonomous driving system is essential to adapt to various environmental changes. This paper aims to perform vehicle pose estimation, a crucial element in forming autonomous driving systems, more universally and robustly. The prevalent method for vehicle pose estimation in autonomous driving systems relies on Real-Time Kinematic (RTK) sensor data, ensuring accurate location acquisition. However, due to the characteristics of RTK sensors, precise positioning is challenging or impossible in indoor spaces or areas with signal interference, leading to inaccurate pose estimation and hindering autonomous driving in such scenarios. This paper proposes a method to overcome these challenges by leveraging objects registered in a high-precision map. The proposed approach involves creating a semantic high-definition (HD) map with added objects, forming object-centric features, recognizing locations using these features, and accurately estimating the vehicle’s pose from the recognized location. This proposed method enhances the precision of vehicle pose estimation in environments where acquiring RTK sensor data is challenging, enabling more robust and stable autonomous driving. The paper demonstrates the proposed method’s effectiveness through simulation and real-world experiments, showcasing its capability for more precise pose estimation.

Intelligent robotic systems frequently operate under stringent energy limitations, especially in complex and dynamic environments. To enhance both adaptability and reliability, this study introduces a semantic planning framework that integrates ontology-driven reasoning with energy awareness. The framework estimates energy consumption based on the platform-specific behavior of sensing, actuation, and computational modules while continuously updating place-level semantic representations using real-time execution data. These representations encode not only spatial and contextual semantics but also energy characteristics acquired from prior operational history. By embedding historical energy usage profiles into hierarchical semantic maps, this framework enables more efficient route planning and context-aware task assignment. A shared semantic layer facilitates coordinated planning for both single-robot and multi-robot systems, with the decisions informed by energy-centric knowledge. This approach remains hardware-independent and can be applied across diverse platforms, such as indoor service robots and ground-based autonomous vehicles. Experimental validation using a differential-drive mobile platform in a structured indoor setting demonstrates improvements in energy efficiency, the robustness of planning, and the quality of the task distribution. This framework effectively connects high-level symbolic reasoning with low-level energy behavior, providing a unified mechanism for energy-informed semantic decision-making.

This paper introduces a hierarchical object-centric descriptor framework called TOSD (Triplet Object-Centric Semantic Descriptor). The goal of this method is to overcome the limitations of existing pixel-based and global feature embedding approaches. To this end, the framework adopts a hierarchical representation that is explicitly designed for multi-level reasoning. TOSD combines shape, color, and topological information without depending on predefined class labels. The shape descriptor captures the geometric configuration of each object. The color descriptor focuses on internal appearance by extracting normalized color features. The topology descriptor models the spatial and semantic relationships between objects in a scene. These components are integrated at both object and scene levels to produce compact and consistent embeddings. The resulting representation covers three levels of abstraction: low-level pixel details, mid-level object features, and high-level semantic structure. This hierarchical organization makes it possible to represent both local cues and global context in a unified form. We evaluate the proposed method on multiple vision tasks. The results show that TOSD performs competitively compared to baseline methods, while maintaining robustness in challenging cases such as occlusion and viewpoint changes. The framework is applicable to visual odometry, SLAM, object tracking, global localization, scene clustering, and image retrieval. In addition, this work extends our previous research on the Semantic Modeling Framework, which represents environments using layered structures of places, objects, and their ontological relations.