12 April 2017 in Physical Review Letters
Smart active particles can acquire some limited knowledge of the fluid environment from simple mechanical cues and exert a control on their preferred steering direction. Their goal is to learn the best way to navigate by exploiting the underlying flow whenever possible. As an example, we focus our attention on smart gravitactic swimmers. These are active particles whose task is to reach the highest altitude within some time horizon, given the constraints enforced by fluid mechanics. By means of numerical experiments, we show that swimmers indeed learn nearly optimal strategies just by experience. A reinforcement learning algorithm allows particles to learn effective strategies even in difficult situations when, in the absence of control, they would end up being trapped by flow structures. These strategies are highly nontrivial and cannot be easily guessed in advance. This Letter illustrates the potential of reinforcement learning algorithms to model adaptive behavior in complex flows and paves the way towards the engineering of smart microswimmers that solve difficult navigation problems.
14 February 2020 in Nature Machine Intelligence
The availability of large datasets has boosted the application of machine learning in many fields and is now starting to shape active-matter research as well. Machine learning techniques have already been successfully applied to active-matter data—for example, deep neural networks to analyse images and track objects, and recurrent nets and random forests to analyse time series. Yet machine learning can also help to disentangle the complexity of biological active matter, helping, for example, to establish a relation between genetic code and emergent bacterial behaviour, to find navigation strategies in complex environments, and to map physical cues to animal behaviours. In this Review, we highlight the current state of the art in the application of machine learning to active matter and discuss opportunities and challenges that are emerging. We also emphasize how active matter and machine learning can work together for mutual benefit.
10 January 2020 in Journal of Fluid Mechanics
How non-spherical particles orient as they settle in a flow has important practical implications in a number of scientific and engineering problems. In a quiescent fluid, a slowly settling particle orients so that it settles with its broad side first. This is an effect of the torque due to convective inertia of the fluid that is set in motion by the settling particle, which maximises the drag experienced by the particle. Turbulent fluid-velocity gradients, on the other hand, tend to randomise the particle orientation. Recently the settling of non-spherical particles in turbulence was analysed neglecting the effect of convective fluid inertia, but taking into account the effect of the turbulent fluid-velocity gradients on the particle orientation. These studies reached the opposite conclusion, namely that the particle tends to settle with its narrow edge first, therefore minimising the drag on the particle. Here, we consider both effects, the convective inertial torque as well as the torque due to fluctuating fluid-velocity gradients. We ask under which circumstances either one or the other dominates. To this end we estimate the ratio of the magnitudes of the two torques. Our estimates suggest that the fluid-inertia torque prevails in high-Reynolds-number flows. In this case non-spherical particles tend to settle with orientations maximising drag. But when the Reynolds number is small, then the torque due to fluid-velocity gradients may dominate, causing the particle to settle with its narrow edge first, minimising the drag.
2 December 2021 in Journal of Fluid Mechanics
Marine micro-organisms must cope with complex flow patterns and even turbulence as they navigate the ocean. To survive they must avoid predation and find efficient energy sources. A major difficulty in analysing possible survival strategies is that the time series of environmental cues in nonlinear flow is complex and that it depends on the decisions taken by the organism. One way of determining and evaluating optimal strategies is reinforcement learning. In a proof-of-principle study, Colabreseet al.(Phys. Rev. Lett., vol. 118, 2017, 158004) used this method to find out how a micro-swimmer in a vortex flow can navigate towards the surface as quickly as possible, given a fixed swimming speed. The swimmer measured its instantaneous swimming direction and the local flow vorticity in the laboratory frame, and reacted to these cues by swimming either left, right, up or down. However, usually a motile micro-organism measures the local flow rather than global information, and it can only react in relation to the local flow because, in general, it cannot access global information (such as up or down in the laboratory frame). Here we analyse optimal strategies with local signals and actions that do not refer to the laboratory frame. We demonstrate that symmetry breaking is required to find such strategies. Using reinforcement learning, we analyse the emerging strategies for different sets of environmental cues that micro-organisms are known to measure.
6 August 2018 in Physical Review Fluids
We performed a numerical study to train smart inertial particles to target specific flow regions with high vorticity through the use of reinforcement learning algorithms. The particles are able to actively change their size to modify their inertia and density. In short, using local measurements of the flow vorticity, the smart particle explores the interplay between its choices of size and its dynamical behavior in the flow environment. This allows it to accumulate experience and learn approximately optimal strategies of how to modulate its size in order to reach the target high-vorticity regions. We consider flows with different complexities: a two-dimensional stationary Taylor-Green-like configuration, a two-dimensional time-dependent flow, and finally a three-dimensional flow given by the stationary Arnold-Beltrami-Childress (ABC) helical flow. We show that smart particles are able to learn how to reach extremely intense vortical structures in all the tackled cases.
27 January 2022 in Physical Review Fluids
Many plankton species undergo daily vertical migration to large depths in the turbulent ocean. To do this efficiently, the plankton can use a gyrotactic mechanism, aligning them with gravity to swim downwards or against gravity to swim upwards. Many species show passive mechanisms for gyrotactic stability. For example, bottom-heavy plankton tend to align upwards. This is efficient for upward migration in quiescent flows, but it is often sensitive to turbulence which upsets the alignment. Here we suggest a simple, robust active mechanism for gyrotactic stability, which is only lightly affected by turbulence and allows alignment both along and against gravity. We use a model for a plankton that swims with a constant speed and can actively steer in response to hydrodynamic signals encountered in simulations of a turbulent flow. Using reinforcement learning, we identify the optimal steering strategy. By using its setae to sense its settling velocity transversal to its swimming direction, the swimmer can deduce information about the direction of gravity, allowing it to actively align upwards. The mechanism leads to a rate of upward migration in a turbulent flow that is of the same order as in quiescent flows, unless the turbulence is very vigorous. In contrast, passive swimmers with typical parameters of copepods show much smaller upward velocity in turbulence. Settling may even cause them to migrate downwards in vigorous turbulence.