In this paper, a model-based linear estimator and a non-linear control law for an Fe-zeolite urea-selective catalytic reduction (SCR) catalyst for heavy duty diesel engine applications is presented. The novel aspect of this work is that the relevant species, NO, NO2 and NH3 are estimated and controlled independently. The ability to target NH3 slip is important not only to minimize urea consumption, but also to reduce this unregulated emission. Being able to discriminate between NO and NO2 is important for two reasons. First, recent Fe-zeolite catalyst studies suggest that NOx reduction is highly favored by the NO 2 based reactions. Second, NO2 is more toxic than NO to both the environment and human health. The estimator and control law are based on a 4-state model of the urea-SCR plant. A linearized version of the model is used for state estimation while the full nonlinear model is used for control design. An experimentally validated, higher order simulation is used to evaluate the performance of the closed loop system. For the cases considered, the control strategy uses less urea, produces less NH3 slip, and less tailpipe NOx than a similar strategy where NO and NO2 are assumed as all NO during estimation and control law implementation.

The levellised cost of energy of wave energy converters (WECs) is not competitive with fossil fuel-powered stations yet. To improve the feasibility of wave energy, it is necessary to develop effective control strategies that maximise energy absorption in mild sea states, whilst limiting motions in high waves. Due to their model-based nature, state-of-the-art control schemes struggle to deal with model uncertainties, adapt to changes in the system dynamics with time, and provide real-time centralised control for large arrays of WECs. Here, an alternative solution is introduced to address these challenges, applying deep reinforcement learning (DRL) to the control of WECs for the first time. A DRL agent is initialised from data collected in multiple sea states under linear model predictive control in a linear simulation environment. The agent outperforms model predictive control for high wave heights and periods, but suffers close to the resonant period of the WEC. The computational cost at deployment time of DRL is also much lower by diverting the computational effort from deployment time to training. This provides confidence in the application of DRL to large arrays of WECs, enabling economies of scale. Additionally, model-free reinforcement learning can autonomously adapt to changes in the system dynamics, enabling fault-tolerant control.

To achieve persistent systems in the future, autonomous underwater vehicles (AUVs) will need to autonomously dock onto a charging station. Here, reinforcement learning strategies were applied for the first time to control the docking of an AUV onto a fixed platform in a simulation environment. Two reinforcement learning schemes were investigated: one with continuous state and action spaces, deep deterministic policy gradient (DDPG), and one with continuous state but discrete action spaces, deep Q network (DQN). For DQN, the discrete actions were selected as step changes in the control input signals. The performance of the reinforcement learning strategies was compared with classical and optimal control techniques. The control actions selected by DDPG suffer from chattering effects due to a hyperbolic tangent layer in the actor. Conversely, DQN presents the best compromise between short docking time and low control effort, whilst meeting the docking requirements. Whereas the reinforcement learning algorithms present a very high computational cost at training time, they are five orders of magnitude faster than optimal control at deployment time, thus enabling an on-line implementation. Therefore, reinforcement learning achieves a performance similar to optimal control at a much lower computational cost at deployment, whilst also presenting a more general framework.