Autonomous indoor service robots are affected by multiple factors when they are directly involved in manipulation tasks in daily life, such as scenes, objects, and actions. It is of self-evident importance to properly parse these factors and interpret intentions according to human cognition and semantics. In this study, the design of a semantic representation framework based on a knowledge graph is presented, including (1) a multi-layer knowledge-representation model, (2) a multi-module knowledge-representation system, and (3) a method to extract manipulation knowledge from multiple sources of information. Moreover, with the aim of generating semantic representations of entities and relations in the knowledge base, a knowledge-graph-embedding method based on graph convolutional neural networks is proposed in order to provide high-precision predictions of factors in manipulation tasks. Through the prediction of action sequences via this embedding method, robots in real-world environments can be effectively guided by the knowledge framework to complete task planning and object-oriented transfer.

In the quest for intelligent robots, it is essential to enable them to understand tasks beyond mere manipulation. Achieving this requires a robust parsing mode that can be used to understand human cognition and semantics. However, the existing methods for task and motion planning lack generalization and interpretability, while robotic knowledge bases primarily focus on static manipulation objects, neglecting the dynamic tasks and skills. To address these limitations, we present a knowledge-based framework for hierarchically understanding various factors and knowledge types in robotic manipulation. Using this framework as a foundation, we collect a knowledge graph dataset describing manipulation tasks from text datasets and an external knowledge base with the assistance of large language models and construct the knowledge base. The reasoning tasks of entity alignment and link prediction are accomplished using a graph embedding method. A robot in real-world environments can infer new task execution plans based on experience and knowledge, thereby achieving manipulation skill transfer.