25 November 2021 in International Journal of Robust and Nonlinear Control
6 March 2025 in IEEE Transactions on Robotics
Robotic systems are typically composed of various subsystems, such as localization and navigation, each encompassing numerous configurable components (e.g., selecting different planning algorithms). Once an algorithm has been selected for a component, its associated configuration options must be set to the appropriate values. Configuration options across the system stack interact non-trivially. Finding optimal configurations for highly configurable robots to achieve desired performance poses a significant challenge due to the interactions between configuration options across software and hardware that result in an exponentially large and complex configuration space. These challenges are further compounded by the need for transferability between different environments and robotic platforms. Data efficient optimization algorithms (e.g., Bayesian optimization) have been increasingly employed to automate the tuning of configurable parameters in cyber-physical systems. However, such optimization algorithms converge at later stages, often after exhausting the allocated budget (e.g., optimization steps, allotted time) and lacking transferability. This paper proposes CURE—a method that identifies causally relevant configuration options, enabling the optimization process to operate in a reduced search space, thereby enabling faster optimization of robot performance. CURE abstracts the causal relationships between various configuration options and the robot performance objectives by learning a causal model in the source (a low-cost environment such as the Gazebo simulator) and applying the learned knowledge to perform optimization in the target (e.g., Turtlebot 3 physical robot). We demonstrate the effectiveness and transferability of CURE by conducting experiments that involve varying degrees of deployment changes in both physical robots and simulation.
3 August 2023 in IEEE Transactions on Automatic Control
This article deals with the embedding framework of Markov decision processes (MDPs) with discrete state and action space to find optimal actions. The optimal control problem of MDPs can be efficiently tackled by restructuring the same into an equivalent linearly-solvable Markov decision processes (LMDPs) through the method called embedding. However, state costs under the embedding may not exactly match the original costs and even assume unrealistic values. In this work, we derive a constructive sufficient condition to devise an exact embedding solution rendering the embedded state cost to match the original system. Furthermore, since, in this case, the embedding implies a transition from the discrete to continuous action space, the correlation between the obtained continuous action and an equivalent desired discrete action is investigated using a maximum a posteriori probability-based method. Finally, some examples, including mammalian cell-cycle network, are presented to demonstrate the effectiveness of the proposed method.
23 November 2021 in IEEE Access
The primary challenge with biological sciences is to control gene regulatory networks (GRNs), thereby creating therapeutic intervention methods that alter network dynamics in the desired manner. The optimal control of GRNs with probabilistic Boolean control networks (PBCNs) as the underlying structure is a solution to this challenge. Owing to the exponential growth in network size with the increase in the number of genes, we need an optimal control approach that scales to large systems without imposing any limitations on network dynamics. Furthermore, we are encouraged to use the graphics processing unit (GPU) to reduce time complexity utilizing the easily available and enhanced computational resources. The optimal control of PBCNs in the Markovian framework is developed in this paper employing an information-theoretic approach which includes Kullback-Leibler (KL) divergence. We convert the nonlinear optimal control problem of PBCN to a linear problem by using the exponential transformation of the cost function, also known as the desirability function. The linear formulation enables us to compute an optimal control using the path integral (PI) method. Furthermore, we offer sampling-based methodologies for approximating PI and therefore optimizing PBCN control. The sampling-based method can be implemented in parallel, which solves the optimal control problem for large PBCNs.