21 June 2022 in Communications Physics
Swimming microorganisms switch between locomotory gaits to enable complex navigation strategies such as run-and-tumble to explore their environments and search for specific targets. This ability of targeted navigation via adaptive gait-switching is particularly desirable for the development of smart artificial microswimmers that can perform complex biomedical tasks such as targeted drug delivery and microsurgery in an autonomous manner. Here we use a deep reinforcement learning approach to enable a model microswimmer to self-learn effective locomotory gaits for translation, rotation and combined motions. The Artificial Intelligence (AI) powered swimmer can switch between various locomotory gaits adaptively to navigate towards target locations. The multimodal navigation strategy is reminiscent of gait-switching behaviors adopted by swimming microorganisms. We show that the strategy advised by AI is robust to flow perturbations and versatile in enabling the swimmer to perform complex tasks such as path tracing without being explicitly programmed. Taken together, our results demonstrate the vast potential of these AI-powered swimmers for applications in unpredictable, complex fluid environments.
1 March 2023 in Physics of Fluids
The use of machine learning techniques in the development of microscopic swimmers has drawn considerable attention in recent years. In particular, reinforcement learning has been shown useful in enabling swimmers to learn effective propulsion strategies through its interactions with the surroundings. In this work, we apply a reinforcement learning approach to identify swimming gaits of a multi-link model swimmer. The swimmer consists of multiple rigid links connected serially with hinges, which can rotate freely to change the relative angles between neighboring links. Purcell [“Life at low Reynolds number,” Am. J. Phys. 45, 3 (1977)] demonstrated how the particular case of a three-link swimmer (now known as Purcell's swimmer) can perform a prescribed sequence of hinge rotation to generate self-propulsion in the absence of inertia. Here, without relying on any prior knowledge of low-Reynolds-number locomotion, we first demonstrate the use of reinforcement learning in identifying the classical swimming gaits of Purcell's swimmer for case of three links. We next examine the new swimming gaits acquired by the learning process as the number of links increases. We also consider the scenarios when only a single hinge is allowed to rotate at a time and when simultaneous rotation of multiple hinges is allowed. We contrast the difference in the locomotory gaits learned by the swimmers in these scenarios and discuss their propulsion performance. Taken together, our results demonstrate how a simple reinforcement learning technique can be applied to identify both classical and new swimming gaits at low Reynolds numbers.
1 June 2021 in Physics of Fluids
There is growing interest in the development of artificial microscopic machines that can perform complex maneuvers like swimming microorganisms for potential biomedical applications. At the microscopic scales, the dominance of viscous over inertial forces imposes stringent constraints on locomotion. In the absence of inertia, Purcell first proposed an elegant way to generate net translation using kinematically irreversible motions [E. M. Purcell, “Life at low Reynolds number,” Am. J. Phys. 45, 3–11 (1977)]. In addition to net translation, a more recent prototype known as Purcell's “rotator” has been proposed in Dreyfus et al. [“Purcell's “rotator”: Mechanical rotation at low Reynolds number,” Eur. Phys. J. B 47, 161–164 (2005)] as a mechanical implementation of net rotation at low Reynolds numbers. These ingenious designs rely on knowledge of the surrounding environment and the physics of locomotion within the environment, which may be incomplete or unclear in more complex scenarios. More recently, reinforcement learning has been used as an alternative approach to enable a machine to learn effective locomotory gaits for net translation based on its interaction with the surroundings. In this work, we demonstrate the use of reinforcement learning to generate net mechanical rotation at low Reynolds numbers without requiring prior knowledge of locomotion. For a three-sphere configuration, the reinforcement learning recovers the strategy proposed by Dreyfus et al. As the number of spheres increases, multiple effective rotational strategies emerge from the learning process. However, given sufficiently long learning processes, all machines considered in this work converge to a single type of rotational policies that consist of traveling waves of actuation, suggesting its optimality of the strategy in generating net rotation at low Reynolds numbers.
1 July 2021 in Physics of Fluids
27 August 2026 in Journal of Computational Physics