Deep Reinforcement Learning (DRL) is a powerful framework for solving complex sequential decision-making problems, particularly in robotic control. However, its practical deployment is often hindered by the substantial amount of experience required for learning, which results in high computational and time costs. In this work, we propose a novel integration of DRL with semantic knowledge in the form of Knowledge Graph Embeddings (KGEs), aiming to enhance learning efficiency by providing contextual information to the agent. Our architecture combines KGEs with visual observations, enabling the agent to exploit environmental knowledge during training. Experimental validation with robotic manipulators in environments featuring both fixed and randomized target attributes demonstrates that our method achieves up to 60% reduction in learning time and improves task accuracy by approximately 15 percentage points, without increasing training time or computational complexity. These results highlight the potential of semantic knowledge to reduce sample complexity and improve the effectiveness of DRL in robotic applications.

Deep Reinforcement Learning (DRL) is a powerful solution for complex sequential decision problems, especially in robotic control tasks. However, DRL’s effectiveness is often restrained by the extensive experience required for learning, leading to high computational and time costs. We present a novel integration of DRL and semantic knowledge through Knowledge Graphs Embeddings (KGEs), to enhance robotic control tasks by incorporating contextual information. Our approach leverages semantic knowledge from pre-built knowledge graphs to provide additional context to the learning agent, effectively reducing the sample complexity and improving learning efficiency. We propose a DRL agent architecture that concatenates KGEs with visual observations, enabling the agent to use those inputs and environmental knowledge. We validate our method through experiments with robotic manipulators in environments with fixed and randomized target attributes, demonstrating significant improvements in task performance, learning speed, and accuracy compared to baseline DRL agents without contextual information. This integration leads to a reduction in the learning time and, therefore, an improvement in sampling efficiency, up to 60%, and to an improvement in the agents’ accuracy of approximately 15 percentage points, without increasing the training time and computational complexity.

The use of virtual environments to collect the experience required by deep reinforcement learning models is accelerating the deployment of these algorithms in industrial environments. However, once the experience-gathering problem is solved, it is necessary to address how to efficiently transfer the knowledge from the virtual scenario to reality. This paper focuses on examining Progressive Neural Networks (PNNs) as a promising transfer learning technique. The analyses carried out range from studying the capabilities and limits of the layers in charge of learning the state representation from a pixel space, which could arguably be the convolutional blocks, to the forgetting agents suffer when learning a new task. Introducing controlled visual changes in the environment scene can lead to a performance degradation of up to 50%. These visual discrepancies strongly influence the agent learning time and its accuracy when using a PNN architecture. Regarding the PNN forgetting assessment, partial forgetting occurs in two of the three environments analyzed, those where the agent masters its new task. This could be due to a balance between the relevance of the new features learned and the ones inherited from the teacher agent.

The use of virtual environments to collect the experience required by deep reinforcement learning models is accelerating the deployment of these algorithms in industrial environments. However, once the experience-gathering problem is solved, it is necessary to address how to efficiently transfer the knowledge from the virtual scenario to reality. This paper focuses on examining Progressive Neural Networks (PNNs) as a promising transfer learning technique. The analyses carried out range from studying the capabilities and limits of the layers responsible for learning the state representation from a pixel space, which could arguably be the convolutional blocks, to the forgetting agents suffer when learning a new task. Introducing controlled visual changes in the environment scene can lead to a performance degradation of 50.3% in the worst-case scenario. These visual discrepancies significantly impact the agent’s learning time and accuracy when using a PNN architecture. Regarding the PNN forgetting assessment, partial forgetting occurs in two of the three environments analyzed, those where the agent masters its new task. This could be due to a balance between the relevance of the new features learned and the ones inherited from the teacher agent.