1 October 2024 in Physics of Fluids
In this experimental study, a value-based reinforcement learning algorithm (deep Q-network, DQN) is used to control the flow separation behind a backward facing step at a Reynolds number of 2.9 × 104. The flow is forced by a dielectric barrier discharge (DBD) plasma actuator pasted at the upstream of the step edge, and the feedback information of the separation zone is provided by a hotwire sensor submerged in the downstream shear layer. The control law represented by a deep neural network is implemented on a field programable gate array (FPGA), able to execute in real-time at a frequency as high as 1000 Hz. Results show that both open-loop periodical control and DQN control can effectively reduce the reattachment length and the recirculation area. Compared with the former, which requires dozens of trail-and-error measurements lasting for hours, the latter is able to find an optimal control law in only two minutes, achieving a long-term reward 7% higher. Moreover, by introducing a weak penalty term for plasma actuation, the mean actuator power consumption in DQN can be cut down to only 60% of that in the optimal open-loop control, meanwhile sacrificing a negligible amount of control effectiveness. Physically, the open-loop periodical control destabilizes the shear layer earlier, increasing both the area and the peak amplitude of the high turbulent kinetic energy (TKE) zone, whereas under DQN control, only a slight increase in the TKE peak is observed, and the overall spatial distribution remains the same as baseline.
1 September 2024 in Physics of Fluids
Although deep reinforcement learning (DRL) has gained increasing popularity in numerical studies of active flow control, practical implementations of this technique in experiments remain challenging, particularly for fast air flows. In this study, we proposed a field programmable gate array (FPGA)-based DRL framework for high-speed experimental active flow control. By splitting the training and execution process of artificial neural network and implementing them effectively in central processing unit (CPU) and FPGA, experimental DRL control with an interaction frequency up to 10–100 kHz can be realized, which is two orders higher than that of the traditional CPU-based DRL framework. Such a new framework is tested in the classical problem of airfoil leading flow separation control at Rec = 2.13 × 105, with a plasma actuator and a hotwire serving as the flow disturber and the state sensor, respectively. As a result, experimental DRL control is rather attractive in terms of the time cost, able to find a converged closed-loop control strategy in only one run of 5 min, eliminating the tedious parameter tuning process in open-loop control (time cost: dozens of minutes to hours). Moreover, the magnitude of lift increment in the case of optimal DRL control is 3.2% higher than that of the best open-loop periodical control strategy.
24 June 2026 in Journal of Fluid Mechanics
In this study, experimental deep reinforcement learning (DRL) control of aerofoil flow separation is conducted at a chord-based Reynolds number of italic Re Subscript c Baseline equals 1.5 times 10 Superscript 5 ${\textit{Re}}_c=1.5\times 10^5$ . A dielectric barrier discharge plasma actuator mounted at the leading edge and a hotwire placed in the separated shear layer act as the flow disturber and the state monitor, respectively. The closed-loop control law is parameterised by a radial basis function network and executed in real time on a field-programmable gate array at 1 kHz. With the aid of a deep deterministic policy gradient algorithm, a satisfying closed-loop control strategy can be derived in less than one minute, and the resulting lift coefficient increment (21 %) is similar to that achieved by the best open-loop control. For stable and effective DRL control, the sensor should be placed at the position with strongest velocity fluctuation, and both time delay and cashing period should be accounted in the reward design. Incorporating historical sensor measurements are beneficial to DRL control, and the optimal history length is approximately equal to the ratio of the local Taylor microscale to the control period. The final control law sought by DRL dictates that strong plasma actuation should only be applied when the flow separation region contracts to a minimum. Statistically, control benefits can be ascribed to both the increase of average reward at each cluster and the elevation of occurrence probabilities of high-reward clusters. Physically, suppression of the aerofoil flow separation is attributed to the excitation of large-scale shear layer vortices, which promotes the momentum exchange between the free stream and the reverse flow.