Florian Köpf

Florian Köpf

Auto-generated profile, based on publicly available information

Florian Köpf

Institute of Control Systems, Karlsruhe Institute of Technology, Karlsruhe, Germany

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 March 2020 in International Journal of Adaptive Control and Signal Processing
Verified
Journal Article
Open access
PDF file
FKFlorian Köpf
Florian Köpf
SRSimon Ramsteiner
Simon Ramsteiner
LPLuca Puccetti
Luca Puccetti
MFMichael Flad
Michael Flad
SHSören Hohmann
Sören Hohmann
Recently proposed adaptive dynamic programming (ADP) tracking controllers assume that the reference trajectory follows time-invariant exo-system dynamics—an assumption that does not hold for many applications. In order to overcome this limitation, we propose a new Q-function that explicitly incorporates a parametrized approximation of the reference trajectory. This allows learning to track a general class of trajectories by means of ADP. Once our Q-function has been learned, the associated controller handles time-varying reference trajectories without the need for further training and independent of exo-system dynamics. After proposing this general model-free off-policy tracking method, we provide an analysis of the important special case of linear quadratic tracking. An example demonstrates that our new method successfully learns the optimal tracking controller and outperforms existing approaches in terms of tracking error and cost.

25 January 2022 in IEEE Transactions on Automatic Control
Verified
Journal Article
Closed access
PKPhilipp Karg
Philipp Karg
FKFlorian Kopf
Florian Kopf
CBChristian A. Braun
Christian A. Braun
SHSören Hohmann
Sören Hohmann
This article focuses on the fulfillment of the persistent excitation (PE) condition for signals which result from transformations by means of polynomials. This is essential, e.g., for the convergence of adaptive dynamic programming algorithms due to commonly used polynomial function approximators. As theoretical statements are scarce regarding the nonlinear transformation of PE signals, we propose conditions on the system state such that its transformation by polynomials is PE. To validate our theoretical statements, we develop an exemplary excitation procedure based on our conditions using a feed-forward control approach and demonstrate the effectiveness of our method in a nonzero-sum differential game. In this setting, our approach outperforms commonly used probing noise in terms of convergence time and the degree of PE, shown by a numerical example.

1 October 2017
Verified
Conference Paper
Open access
JIJairo Inga
Jairo Inga
FKFlorian Kopf
Florian Kopf
MFMichael Flad
Michael Flad
SHSören Hohmann
Sören Hohmann
Shared control techniques have a great potential to create synergies in human-machine interaction for efficient and safe applications. However, an optimal interaction requires the machine to consider the individual behavior of the human partner. A widespread approach for modeling human behavior is given by optimal control theory, where the movement trajectories of a human arise from an optimized cost function. The aim of the identification is thus to determine parameters of a cost function which explains observed human motion. The central thesis of this paper is that individual cost function parameters which describe specific behavior can be determined by means of Inverse Reinforcement Learning. We show the applicability of the approach with a tracking control task example. The experiment consists in following a reference trajectory by means of a steering wheel. The study confirms that optimal control is suitable for modeling individual human behavior and demonstrates the suitability of Inverse Reinforcement Learning in order to determine the cost function parameters which explain measured data.

29 October 2019 in arXiv
Verified
Preprint
Open access
PDF file
FKFlorian Köpf
Florian Köpf
STSamuel Tesfazgi
Samuel Tesfazgi
MFMichael Flad
Michael Flad
SHSören Hohmann
Sören Hohmann
In order to collaborate efficiently with unknown partners in cooperative control settings, adaptation of the partners based on online experience is required. The rather general and widely applicable control setting, where each cooperation partner might strive for individual goals while the control laws and objectives of the partners are unknown, entails various challenges such as the non-stationarity of the environment, the multi-agent credit assignment problem, the alter-exploration problem and the coordination problem. We propose new, modular deep decentralized Multi-Agent Reinforcement Learning mechanisms to account for these challenges. Therefore, our method uses a time-dependent prioritization of samples, incorporates a model of the system dynamics and utilizes variable, accountability-driven learning rates and simulated, artificial experiences in order to guide the learning process. The effectiveness of our method is demonstrated by means of a simulated, nonlinear cooperative control task.

20 January 2022 in arXiv
Verified
Preprint
Open access
PDF file
PKPhilipp Karg
Philipp Karg
FKFlorian Köpf
Florian Köpf
CBChristian A. Braun
Christian A. Braun
SHSören Hohmann
Sören Hohmann
This work focuses on the fulfillment of the Persistent Excitation (PE) condition for signals which result from transformations by means of polynomials. This is essential e.g. for the convergence of Adaptive Dynamic Programming algorithms due to commonly used polynomial function approximators. As theoretical statements are scarce regarding the nonlinear transformation of PE signals, we propose conditions on the system state such that its transformation by polynomials is PE. To validate our theoretical statements, we develop an exemplary excitation procedure based on our conditions using a feedforward control approach and demonstrate the effectiveness of our method in a nonzero-sum differential game. In this setting, our approach outperforms commonly used probing noise in terms of convergence time and the degree of PE, shown by a numerical example.

2 March 2020 in arXiv
Verified
Preprint
Open access
PDF file
FKFlorian Köpf
Florian Köpf
ANAlexander Nitsch
Alexander Nitsch
MFMichael Flad
Michael Flad
SHSören Hohmann
Sören Hohmann
Mixed cooperative-competitive control scenarios such as human-machine interaction with individual goals of the interacting partners are very challenging for reinforcement learning agents. In order to contribute towards intuitive human-machine collaboration, we focus on problems in the continuous state and control domain where no explicit communication is considered and the agents do not know the others' goals or control laws but only sense their control inputs retrospectively. Our proposed framework combines a learned partner model based on online data with a reinforcement learning agent that is trained in a simulated environment including the partner model. Thus, we overcome drawbacks of independent learners and, in addition, benefit from a reduced amount of real world data required for reinforcement learning which is vital in the human-machine context. We finally analyze an example that demonstrates the merits of our proposed framework which learns fast due to the simulated environment and adapts to the continuously changing partner due to the partner approximation.