3D visual recognition is a prerequisite for most autonomous robotic systems operating in the real world. It empowers robots to perform a variety of tasks, such as tracking, understanding the environment, and human–robot interaction. Autonomous robots equipped with 3D recognition capability can better perform their social roles through supportive task assistance in professional jobs and effective domestic services. For active assistance, social robots must recognize their surroundings, including objects and places to perform the task more efficiently. This article first highlights the value-centric role of social robots in society by presenting recently developed robots and describes their main features. Instigated by the recognition capability of social robots, we present the analysis of data representation methods based on sensor modalities for 3D object and place recognition using deep learning models. In this direction, we delineate the research gaps that need to be addressed, summarize 3D recognition datasets, and present performance comparisons. Finally, a discussion of future research directions concludes the article. This survey is intended to show how recent developments in 3D visual recognition based on sensor modalities using deep-learning-based approaches can lay the groundwork to inspire further research and serves as a guide to those who are interested in vision-based robotics applications.

Pedestrian tracking is a challenging task in the area of visual object tracking research and it is a vital component of various vision-based applications such as surveillance systems, human-following robots, and autonomous vehicles. In this paper, we proposed a single pedestrian tracking (SPT) framework for identifying each instance of a person across all video frames through a tracking-by-detection paradigm that combines deep learning and metric learning-based approaches. The SPT framework comprises three main modules: detection, re-identification, and tracking. Our contribution is a significant improvement in the results by designing two compact metric learning-based models using Siamese architecture in the pedestrian re-identification module and combining one of the most robust re-identification models for data associated with the pedestrian detector in the tracking module. We carried out several analyses to evaluate the performance of our SPT framework for single pedestrian tracking in the videos. The results of the re-identification module validate that our two proposed re-identification models surpass existing state-of-the-art models with increased accuracies of 79.2% and 83.9% on the large dataset and 92% and 96% on the small dataset. Moreover, the proposed SPT tracker, along with six state-of-the-art (SOTA) tracking models, has been tested on various indoor and outdoor video sequences. A qualitative analysis considering six major environmental factors verifies the effectiveness of our SPT tracker under illumination changes, appearance variations due to pose changes, changes in target position, and partial occlusions. In addition, quantitative analysis based on experimental results also demonstrates that our proposed SPT tracker outperforms the GOTURN, CSRT, KCF, and SiamFC trackers with a success rate of 79.7% while beating the DiamSiamRPN, SiamFC, CSRT, GOTURN, and SiamMask trackers with an average of 18 tracking frames per second.

In heterogeneous multi-robot systems, allocating tasks efficiently requires determining whether each robot can actually carry out a given task. In existing multi-robot task allocation research, however, such task feasibility has typically been handled inside a particular optimizer or symbolic planner, while spatial traversability has been assessed against static criteria that cannot capture the changes induced by a robot’s loaded state. This paper proposes an ontology-based semantic feasibility reasoning method for heterogeneous multi-robot task allocation. The proposed method defines semantic models for robots, tasks, and places and applies hybrid reasoning, combining declarative reasoning with procedural evaluation, to determine multi-axis capability conditions and loaded-state place reachability. The reasoning result is formalized as an allocator-independent ReasonerOutput that serves as a common input for diverse allocation algorithms. In experiments spanning four scenarios over three fleet configurations and four allocators, together with an ablation study on 200 randomized instances at each of three problem scales, the proposed ReasonerOutput consistently functioned as a shared semantic feasibility constraint. The experiments further showed that both the fleet composition and the loaded state of the target item affect assignment feasibility. These results indicate that, for the static one-shot assignment setting evaluated here, the proposed method makes the task feasibility of heterogeneous robots explicit through semantic reasoning and allows allocators of differing algorithmic character to draw on this feasibility in common.

Heterogeneous robot fleets transport items that differ in shape and weight, and robots differ in what they can handle and where they can pass. Conventional task allocation methods often compute allocation costs and routes on a single load-invariant spatial model, which may overlook changes in place traversability after loading. Payload capacity typically enters allocation only as a feasibility or ranking condition, so light tasks may be allocated to high-capacity robots also needed for heavier ones. Taking the feasibility information from semantic reasoning as input, this study combines a route cost on load-state-dependent traversability graphs with a normalized payload-margin cost in a makespan-oriented sequential allocation procedure. In ablation experiments, load-invariant route planning produced non-traversable loaded routes for approximately one quarter of the tasks, whereas load-state-dependent planning and validation produced none, as expected by construction. Against a violation-free variant with load-invariant costs and no payload term, the proposed configuration shortened the makespan by 6.7–20.1%, mainly through the payload-margin term; its route cost made the predicted completion times consistent with the planned routes. The reduction persisted on a larger layout and with congested loading and unloading places. In ROS 2 and Gazebo, eight robots followed the planned place sequences in both completed runs.