Reinforcement learning (RL) is explored for motor control of a novel pneumatic-driven soft robot modeled after continuum media with a varying density. This model complies with closed-form Lagrangian dynamics, which fulfills the fundamental structural property of passivity, among others. Then, the question arises of how to synthesize a passivity-based RL model to control the unknown continuum soft robot dynamics to exploit its input–output energy properties advantageously throughout a reward-based neural network controller. Thus, we propose a continuous-time Actor–Critic scheme for tracking tasks of the continuum 3D soft robot subject to Lipschitz disturbances. A reward-based temporal difference leads to learning with a novel discontinuous adaptive mechanism of Critic neural weights. Finally, the reward and integral of the Bellman error approximation reinforce the adaptive mechanism of Actor neural weights. Closed-loop stability is guaranteed in the sense of Lyapunov, which leads to local exponential convergence of tracking errors based on integral sliding modes. Notably, it is assumed that dynamics are unknown, yet the control is continuous and robust. A representative simulation study shows the effectiveness of our proposal for tracking tasks.

Summary: Model‐based adaptive control suffers over parametrization from the many adaptive parameters compared to the order of system dynamics, leading to sluggish tracking with a poor adaptation transient without robustness. Likewise, adaptive model‐free neurocontrol that relies on the Stone–Weierstrass theorem also suffers from similar problems in addition to over‐fitting to approximate inverse dynamics. This article proposes a novel reinforced adaptive mechanism to guarantee a transient and robustness for the model‐free adaptive control of nonlinear Lagrangian systems. Inspired by the symbiosis of Actor‐Critic (AC) architecture and integral sliding modes, the reinforced stage neural network, analogous to the critic, injects excitation signals to reinforce the parametric learning of the adaptive stage neural network, analogous to the actor to improve the approximation of inverse dynamics. The underlying integral sliding surface error drives improved learning onto a low‐dimensional invariant manifold to guarantee local exponential convergence of tracking errors. Lyapunov stability substantiates the robustness with an improved transient response. Our proposal stands for a hybrid approach between AC and neurocontrol, where the reinforced stage does not require a value function nor reward to provide automatic reinforcement to the adaptive stage parametric adaptation. Dynamic simulations are presented for a nonlinear robot manipulator under different conditions.

Rocket Thrust Vectoring Attitude Control based on Convolutional Neural Networks

The conventional limitations of the robotic actuation mechanisms have led to many researchers needing to explore novel biomimetic motor mechanisms as the antagonistic human motor system. In this way, it is of interest to understand the inherent adaptive stiffness, or compliance, and modulation, in different alternative actuation architectures such as the antagonistic bi-articular (AbA) system. These novel AbA actuation mechanisms are characterized by resembling the efficient tendon and muscle build-up over our skeletal structure. In this paper, we propose a Cartesian neuro-controller for a robot manipulator actuated by a simplified adaptive viscoelastic linear AbA system. It is shown that the adaptive closed-loop system enforces terminal attractors, induced by a continuous model-free sliding mode control, simultaneously with a learning algorithm to compensate parametric uncertainties of AbA system through a low dimensional neural network. Numerical simulation results exhibit the feasibility of this approach.

En aplicaciones de control inteligente, uno de los problemas es determinar el número de capas y de neuronas en cada capa. Esto se vuelve más complejo con ciertos tipos de redes neuronales como las basadas en wavelets donde las traslaciones y dilataciones son parámetros adicionales. Este artículo presenta un estudio comparativo para determinar el tipo de wavelet y número de neuronas que muestran el mejor desempeño para controlar un helicóptero Quanser de dos grados de libertad (GDL). Se presenta un controlador tipo PID-WaveNet-IIR el cuál se compone de controladores PID discretos en el tiempo, con ganancias auto-sintonizables por una red neuronal de base radial cuyas funciones de activación son wavelets y un par de filtros de respuesta de impulso infinito (IIR) para ``podar'' algunas neuronas. Por medio de simulaciones numéricas, usando LabVIEW, se presenta el desempeño del sistema en lazo cerrado para: diferentes condiciones de operación, tipos de familia wavelets donde se fijan valores mínimos de los errores de seguimiento, tipo de wavelet, el número de neuronas de la red y número de coeficientes de adelanto y atraso de los filtros IIR.