The motivation behind our work is to review and analyze the most relevant studies on deep reinforcement learning-based object manipulation. Various studies are examined through a survey of existing literature and investigation of various aspects, namely, the intended applications, techniques applied, challenges faced by researchers and recommendations for minimizing obstacles. This review refers to all relevant articles on deep reinforcement learning-based object manipulation and solutions. The object grasping issue is a major manipulation challenge. Object grasping requires detection systems, methods and tools to facilitate efficient and fast agent training. Several studies have proposed that object grasping and its subtypes are the main elements in dealing with the environment and agent. Unlike other review articles, this review article provides different observations on deep reinforcement learning-based manipulation. The results of this comprehensive review of deep reinforcement learning in the manipulation field may be valuable for researchers and practitioners because they can expedite the establishment of important guidelines.

Robotic manipulation refers to how robots intelligently interact with the objects in their surroundings, such as grasping and carrying an object from one place to another. Dexterous manipulating skills enable robots to assist humans in accomplishing various tasks that might be too dangerous or difficult to do. This requires robots to intelligently plan and control the actions of their hands and arms. Object manipulation is a vital skill in several robotic tasks. However, it poses a challenge to robotics. The motivation behind this review paper is to review and analyze the most relevant studies on learning-based object manipulation in clutter. Unlike other reviews, this review paper provides valuable insights into the manipulation of objects using deep reinforcement learning (deep RL) in dense clutter. Various studies are examined by surveying existing literature and investigating various aspects, namely, the intended applications, the techniques applied, the challenges faced by researchers, and the recommendations adopted to overcome these obstacles. In this review, we divide deep RL-based robotic manipulation tasks in cluttered environments into three categories, namely, object removal, assembly and rearrangement, and object retrieval and singulation tasks. We then discuss the challenges and potential prospects of object manipulation in clutter. The findings of this review are intended to assist in establishing important guidelines and directions for academics and researchers in the future.

Traffic optimization remains a significant challenge in urban planning and transportation management. While efficient traffic optimization is crucial for enhancing urban mobility, reducing congestion, and promoting environmental sustainability, traditional computational methods often struggle with the complex, dynamic nature of traffic systems. Recent advances in quantum computing, particularly quantum annealing, offer promising new techniques that could revolutionize traffic flow optimization. This work systematically reviews the literature, starting with search term formulation and ending with the final set of articles. These articles are categorized into three groups: (1) traffic signal control, (2) traffic flow optimization, and (3) routing problems optimization (including vehicle routing problem and traveling salesman problem). The review critically examines current studies on quantum annealing-based traffic optimization, focusing on contributions, methods, solvers, problem suitability, key findings, benchmark fairness, and limitations. It identifies key challenges and provides recommendations for future research. Insights from this work offer researchers and practitioners a concise overview of current challenges and future directions in traffic optimization.

In robotic manipulation, object grasping is a basic yet challenging task. Dexterous grasping necessitates intelligent visual observation of the target objects by emphasizing the importance of spatial equivariance to learn the grasping policy. In this paper, two significant challenges associated with robotic grasping in both clutter and occlusion scenarios are addressed. The first challenge is the coordination of push and grasp actions, in which the robot may occasionally fail to disrupt the arrangement of the objects in a well-ordered object scenario. On the other hand, when employed in a randomly cluttered object scenario, the pushing behavior may be less efficient, as many objects are more likely to be pushed out of the workspace. The second challenge is the avoidance of occlusion that occurs when the camera itself is entirely or partially occluded during a grasping action. This paper proposes a multi-view change observation-based approach (MV-COBA) to overcome these two problems. The proposed approach is divided into two parts: 1) using multiple cameras to set up multiple views to address the occlusion issue; and 2) using visual change observation on the basis of the pixel depth difference to address the challenge of coordinating push and grasp actions. According to experimental simulation findings, the proposed approach achieved an average grasp success rate of 83.6%, 86.3%, and 97.8% in the cluttered, well-ordered object, and occlusion scenarios, respectively.

This paper analyzes the effects of the bilateral control parameters variation on the stability, the transparency and the accuracy, and on the operational force that is applied to DC motor and the master system. The bilateral controller is designed for rehabilitation process. PD controller is used to control the position tracking and a force gain controller is used to control the motor torque. DOB eliminate the internal disturbance and RTOB to estimate the joint torque without using sensors. The system consists of two manipulators, each manipulator has 1dof, master and slave teleoperation system, 4 control-architecture channel, DOB and reaction force observer. The master system is attached to human oberator. The slave system is attached to external load. The aim in this paper is to design the controller so that it requires less force to move the master manipulator and at the same time achieve high performance in position tracking.