Extensive studies have shown that many animals’ capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot’s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.

Traditional underwater vehicles are limited in speed due to dramatic friction drag on the hull. Supercavitating vehicles exploit supercavitation as a means to reduce drag and increase their underwater speed. Compared with fully wetted vehicles, the non-linearity in the modelling of cavitator, fin and in particular the planing force make the control design of supercavitating vehicles more challenging. Dominant non-linearities associated with planing force are taken into account in the model of supercavitating vehicles in this paper. Two controllers are proposed to realize stable system dynamics and tracking responses, a linear quadratic regulator (LQR) control scheme and a robust backstepping control (RBC) scheme. The proposed backstepping procedure, in association with integral filters technique, exploits the possibility of avoiding the overparameterization problem existing in the classical backstepping process. In particular, the achieved stability is robust to modelling errors in supercavitating vehicles. Compared with the LQR control scheme, the RBC scheme is seen to increase the robustness with saturation compensation algorithm, which can be useful for avoiding actuator saturation in magnitude.

Supercavitation technology can greatly improve the speed of underwater vehicles by reducing drag. However, the time-delay characteristic of a supercavitating vehicle presents control challenge. In this paper, a linear parameter-varying (LPV) time-delay model for a supercavitating vehicle is established by analysing the time-delay characteristics of the planing force. Moreover, considering that the cavitation radius is easily perturbed by external disturbances, a supercavitating LPV delay model depicted as a polytope is established. Based on the model, a predictive controller based on time-delay characteristics is designed. The controller can solve linear matrix inequalities (LMI) online to find the optimal solution at a given moment. In addition, the controller introduces a different Lyapunov function and multiple free control variables to reduce the system’s conservativeness and expand the system’s initial feasible region, which can improve the control performance of the system. A simulation verifies that the system exhibits good stability and robustness.

In this paper, to effectively solve the joint motion planning of a nine-DOF redundant hybrid boarding system in the process of ocean wave active compensation, we present a multi-task motion planning (MTMP) method for the redundant hybrid boarding system. First, the hybrid mechanism is disassembled into a six-DOF parallel mechanism and a three-DOF serial mechanism for kinematic analysis separately to obtain the Jacobian matrix of the hybrid mechanism in the task space. Then, according to the configuration characteristics of the hybrid mechanism and to avoid the defects of the pseudo-inverse and weighted pseudo-inverse methods in motion planning, several secondary tasks are proposed to constrain the motion planning of the hybrid mechanism. Based on simulations and comparisons, the results show that the MTMP method solves the problems of joint limitation, reverse compensation, overlap compensation, and divergence with time in the motion planning of the pseudo-inverse method and the weighted pseudo-inverse method. Meanwhile, the MTMP method has more superiority in the joint margin and compensation space range.