Road object detection has been increasingly considered an essential task in automotive applications. Road objects are divided into two groups: i) above-ground objects such as vehicles and pedestrians and ii) road obstacles such as potholes and cracks. While above-ground object detection has been actively studied and achieved great advance, road obstacle detection has not been extensively studied. This paper proposes the road obstacle detection method based on 3D point cloud candidates extracted by height analysis. The 2D candidates of road obstacles are extracted from 3D candidates. A variant of RANSAC is applied to reduce point cloud matching errors in 3D reconstruction. The proposed system is implemented on the Jetson Xavier NX platform with a 20Hz camera, providing an accurate detection rate with feature tracking and visual inertial odometry in real-time. The proposed method is evaluated on a database consisting of concrete and tar-roads scenes. The experimental results verify the superiority of the proposed method in detecting road obstacles over the state-of-the-art methods.
1 January 2024 in International Journal of Agricultural and Biological Engineering
Task planning is a crucial component in facilitating robot multi-task manipulations. Language-based task planning methods offer practicality in receiving commands from humans in real-life scenarios and require only low-cost labeled data. However, existing methods often rely on sequence models for planning, which primarily focus on mapping language to sequences of sub-tasks while neglecting the knowledge about tasks and objects. To overcome these limitations, we propose a knowledge-based task planning approach called Recurrent Graph Convolutional Network (RGCN). It is devised with a novel structure that combined GCN (Kipf and Welling in International Conference on Learning Representations (ICLR), 2017) and LSTM (Hochreiter and chmidhuber in Neural Comput 9 (8): 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735) which enables it to leverage knowledge graph data and historical predictions. The experimental results demonstrate that our approach achieves the impressive task planning success rate of $${95.7\%}$$ , surpassing the best baseline method significantly, which achieves $${78.7\%}$$ . Furthermore, we evaluate the performance of multi-task manipulation across a specific set of 20 tasks within a simulated environment. Notably, RGCN combined with pre-trained primitive tasks exhibits the highest success rate compared with state-of-art multi-task learning methods. Our method is proven to be significant for language-conditioned task planning and is qualified for instructing robots for multi-task manipulation.
In view of the present situation of bridge inspection equipment, a novel bridge inspection method is proposed and a wall-climbing robot based on negative pressure adhesion mechanism with adaptive control is developed. To meet the bridge inspection requirements, the developed robot consists of adhesion module, movement module, control module and crack detect module. The rough bridge wall environment is a challenging problem for adhesion. To address the problem, an adhesion adaptive control method based on pressure closed-up control is proposed. The method is described in detail and the overall system model is analyzed, including gap interference model. The PID parameters are well tuned to make a better system performance. The simulations with different gap interference are done and results are discussed respectively. The feasibility of robot which adopts adhesion adaptive control method is demonstrated by the simulation results. Finally, the effectiveness of the system is verified by the experiments conducted on various roughness walls.
Simultaneous Localization and Mapping (SLAM) combined with visual and inertial measurements has attained considerable consideration in the Robotics and Computer Vision society. Nevertheless, balancing between real-time needs and precision could be a difficult challenge. Thus, a new tightly-coupled visual-inertial concurrent localization and mapping approach is proposed with precise and real-time motion estimating and map reconstruction capabilities. The nonlinear optimization is based on the concept that the frontend and backend in the visual-inertial SLAM (VISLAM) system can enhance one another. Moreover, a new inertial measurement unit (IMU) initialization approach is employed for rapid calculation of the scale, the gravity orientation, velocity, and gyroscope and accelerometer biases with high precision. Besides, precise motion estimation of the frontend could be provided, which improves the backend optimization due to achieving a more precise initial state for the backend. Also, feedback-based relocalization and continued SLAM frameworks are designed for autonomous robot navigation or SLAM. The accuracy of the presented VISLAM system is investigated via experiments performed on the public EuRoC dataset and actual environments. According to the experiments, the presented VISLAM system is more accurate with lower computational cost compared with existing VISLAM systems.
Currently, a bottleneck problem for battery-powered microflying robots is time of endurance. Inspired by flying animal behavior in nature, an innovative mechanism with active flying and perching in the three-dimensional space was proposed to greatly increase mission life and more importantly execute tasks perching on an object in the stationary way. In prior work, we have developed some prototypes of flying and perching robots. However, when the robots switch between flying and perching, it is a challenging issue to deal with the contact between the robot and environment under the traditional position control without considering the stationary obstacle and external force. Therefore, we propose a unified impedance control approach for bioinspired flying and perching robots to smoothly contact with the environment. The dynamic model of the bioinspired robot is deduced, and the proposed impedance control method is employed to control the contact force and displacement with the environment. Simulations including the top perching and side perching and the preliminary experiments were conducted to validate the proposed method. Both simulation and experimental results validate the feasibility of the proposed control methods for controlling a bioinspired flying and perching robot.