Daniel Nikovski

Daniel Nikovski

Auto-generated profile, based on publicly available information

Daniel Nikovski

Mitsubishi Electric Research Laboratories 201 Broadway 02139 Cambridge MA USA

Research Focus & Metrics

Academic Career Length

21Years (2006-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 July 2022 in IEEE Transactions on Robotics
Verified
Journal Article
Closed access
FAFabio Amadio
Fabio Amadio
ALAlberto Dalla Libera
Alberto Dalla Libera
RARiccardo Antonello
Riccardo Antonello
DNDaniel Nikovski
Daniel Nikovski
RCRuggero Carli
Ruggero Carli
DRDiego Romeres
Diego Romeres
In this article, we present a model-based reinforcement learning (MBRL) algorithm named Monte Carlo probabilistic inference for learning control (MC-PILCO). This algorithm relies on Gaussian processes (GPs) to model the system dynamics and on a Monte Carlo approach to estimate the policy gradient. This defines a framework in which we ablate the choice of the components, which are the selection of the cost function, the optimization of policies using dropout, and an improved data efficiency through the use of structured kernels in the GP models. The combination of the aforementioned aspects affects dramatically the performance of MC-PILCO. Numerical comparisons in a simulated cart–pole environment show that MC-PILCO exhibits better data efficiency and control performance w.r.t. state-of-the-art GP-based MBRL algorithms. Finally, we apply MC-PILCO to real systems, considering, in particular, systems with partially measurable states. We discuss the importance of modeling both the measurement system and the state estimators during policy optimization. The effectiveness of the proposed solutions has been tested in simulation and on two real systems, which are the Furuta pendulum and the ball-and-plate rig.

25 January 2016 in Data Mining and Knowledge Discovery
Verified
Journal Article
Closed access
PDF file
MJMichael Jones
Michael Jones
DNDaniel Nikovski
Daniel Nikovski
MIMakoto Imamura
Makoto Imamura
THTakahisa Hirata
Takahisa Hirata

18 July 2017 in IEEE Transactions on Energy Conversion
Verified
Journal Article
Closed access
SSShahil Shah
Shahil Shah
HSHongbo Sun
Hongbo Sun
DNDaniel Nikovski
Daniel Nikovski
JZJinyun Zhang
Jinyun Zhang
Fast synchronization of generators and microgrids will be a critical technology in future power systems with high penetration of non-dispatchable power resources. Existing synchronization methods rely on generator controls and their performance is limited by the generator characteristics. Speed of microgrid synchronization is further limited by the communication link among generation units. These factors lead to a slow and sometimes faulty synchronization, predominantly because of the phase mismatch during interconnection. This paper frames the generator synchronization problem as a phase-locked loop (PLL) design problem and introduces a voltage-source converter (VSC) based synchronizer for implementing the PLL based active synchronization method. The proposed synchronizer is connected to a generator or microgrid at the point of common coupling and it requires only local measurements for control. It ensures zero phase error during the interconnection by taking the advantage of the fact that the phase is not regulated in the generator controls. Relation between the synchronizer rating, control design, and synchronization speed is developed using the describing function analysis of the frequency and phase control loop gains. The operation and performance of the VSC-based synchronizer is demonstrated using numerical simulations for a 555 MVA, 24 kV synchronous generator.

1 November 2019
Verified
Conference Paper
Open access
KOKei Ota
Kei Ota
DJDevesh K. Jha
Devesh K. Jha
TOTomoaki Oiki
Tomoaki Oiki
MMMamoru Miura
Mamoru Miura
TNTakashi Nammoto
Takashi Nammoto
TMToshisada Mariyama
Toshisada Mariyama
In this paper, we propose a reinforcement learning-based algorithm for trajectory optimization for constrained dynamical systems. This problem is motivated by the fact that for most robotic systems, the dynamics may not always be known. Generating smooth, dynamically feasible trajectories could be difficult for such systems. Using sampling-based algorithms for motion planning may result in trajectories that are prone to undesirable control jumps. However, they can usually provide a good reference trajectory which a model-free reinforcement learning algorithm can then exploit by limiting the search domain and quickly finding a dynamically smooth trajectory. We use this idea to train a reinforcement learning agent to learn a dynamically smooth trajectory in a curriculum learning setting. Furthermore, for generalization, we parameterize the policies with goal locations, so that the agent can be trained for multiple goals simultaneously. We show result in both simulated environments as well as real experiments, for a 6-DoF manipulator arm operated in position-controlled mode to validate the proposed idea. We compare the proposed ideas against a PID controller which is used to track a designed trajectory in configuration space. Our experiments show that our RL agent trained with a reference path outperformed a model-free PID controller of the type commonly used on many robotic platforms for trajectory tracking.

16 July 2009 in Machine Learning
Verified
Journal Article
Open access
PDF file
DNDaniel Nikovski
Daniel Nikovski
AJAnkur Jain
Ankur Jain
We propose two fast algorithms for abrupt change detection in streaming data that can operate on arbitrary unknown data distributions before and after the change. The first algorithm, \(\textsf{MB-GT}\) , computes efficiently the average Euclidean distance between all pairs of data points before and after the hypothesized change. The second algorithm, \(\textsf{MB-CUSUM}\) , computes the log-likelihood ratio statistic for the data distributions before and after the change, similarly to the classical CUSUM algorithm, but unlike that algorithm, \(\textsf{MB-CUSUM}\) does not need to know the exact distributions, and uses kernel density estimates instead. Although a straightforward computation of the two change statistics would have computational complexity of O(N 4) with respect to the size N of the streaming data buffer, the proposed algorithms are able to use the computational structure of these statistics to achieve a computational complexity of only O(N 2) and memory requirement of O(N). Furthermore, the algorithms perform surprisingly well on dependent observations generated by underlying dynamical systems, unlike traditional change detection algorithms.

28 February 2020 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
ALAlberto Dalla Libera
Alberto Dalla Libera
DRDiego Romeres
Diego Romeres
DJDevesh K. Jha
Devesh K. Jha
BYBill Yerazunis
Bill Yerazunis
DNDaniel Nikovski
Daniel Nikovski
In this paper, we propose a derivative-free model learning framework for Reinforcement Learning (RL) algorithms based on Gaussian Process Regression (GPR). In many mechanical systems, only positions can be measured by the sensing instruments. Then, instead of representing the system state as suggested by the physics with a collection of positions, velocities, and accelerations, we define the state as the set of past position measurements. However, the equation of motions derived by physical first principles cannot be directly applied in this framework, being functions of velocities and accelerations. For this reason, we introduce a novel derivative-free physically-inspired kernel, which can be easily combined with nonparametric derivative-free Gaussian Process models. Tests performed on two real platforms show that the considered state definition combined with the proposed model improves estimation performance and data-efficiency w.r.t. traditional models based on GPR. Finally, we validate the proposed framework by solving two RL control problems for two real robotic systems.