In robotic manipulation, achieving efficient and reliable grasping in cluttered environments remains a significant challenge. This study presents a novel approach that integrates pushing and grasping actions using deep reinforcement learning. The proposed model employs two fully convolutional neural networks—Push-Net and Grasp-Net—that predict pixel-wise Q-values for potential pushing and grasping actions from heightmap images of the scene. The training process utilizes deep Q-learning with a reward structure that incentivizes both successful pushes and grasps, encouraging the robot to create favorable conditions for grasping through strategic pushing actions. Simulation results demonstrate that the proposed model significantly outperforms traditional grasp-only policies, achieving an 87% grasp success rate in cluttered environments, compared to 60% for grasp-only approaches. The model shows robust performance in various challenging scenarios, including well-ordered configurations and novel objects, with completion rates of up to 100% and grasp success rates as high as 95.8%. These findings highlight the model’s ability to generalize to unseen objects and configurations, making it a practical solution for real-world robotic manipulation tasks.

This paper presents deep reinforcement learning-based synergy between pushing and grasping systems to improve the grasping performance of the UR5 robot in a cluttered scene. In robotic manipulations, grasping an object in a clutter is fundamental yet a challenging activity for industrial applications. This is because numerous studies focused on improving grasping performance in cluttered environments using either a grasping-only policy or pushing and grasping without incorporating a pushing reward. Additionally, some research has been limited to using similar objects, such as cubes. This paper for mulated the mathematical modeling of the universal robot manipulator. The proposed model involves training two fully-connected convolutional neural networks that transfer visual observations of the scene to a dense pixel-wise sample of end-effector orientations and positions for each pushing and grasping action. A fixed RGB-D camera is used to take the raw images of the scene and generate a heightmap image. Before feeding the heightmap image to the fully convolutional network, it is rotated by 36 different angles to generate 36 pixel-wise Q-value predictions. Both pushing and grasping networks are self-supervised by trial and error from experience and are trained together in a deep Q-learning algorithm. Successful grasps have a reward of 1, while successful pushes have a 0.5 reward value. But unsuccessful actions have a reward of 0 value. The proposed policy learns pushing motions to improve future grasping in a cluttered scene. The experiment demonstrates that the proposed model can successfully grasp objects with an 87% grasp success rate while grasping only policy, no-reward for pushing policy, and stochastic gradient without momentum is 60%, 71%, and 79% respectively. Further, it has been demonstrated that the proposed model is capable of generalizing to randomly arranged cluttered objects, challenging arrangements, and novel objects.