We address the problem of learning robot control by model-free reinforcement learning (RL). We adopt the probabilistic model for model-free RL of Vlassis and Toussaint (Proceedings of the international conference on machine learning, Montreal, Canada, 2009), and we propose a Monte Carlo EM algorithm (MCEM) for control learning that searches directly in the space of controller parameters using information obtained from randomly generated robot trajectories. MCEM is related to, and generalizes, the PoWER algorithm of Kober and Peters (Proceedings of the neural information processing systems, 2009). In the finite-horizon case MCEM reduces precisely to PoWER, but MCEM can also handle the discounted infinite-horizon case. An interesting result is that the infinite-horizon case can be viewed as a ‘randomized’ version of the finite-horizon case, in the sense that the length of each sampled trajectory is a random draw from an appropriately constructed geometric distribution. We provide some preliminary experiments demonstrating the effects of fixed (PoWER) vs randomized (MCEM) horizon length in two simulated and one real robot control tasks.

The electric vehicle (EV) market has grown over the last few years and even though electric vehicles do not currently possess a high market segment, it is projected that they will do so by 2030. Currently, the electric vehicle industry is looking to resolve the issue of vehicle range, using higher battery capacities and fast charging. Energy consumption is a key issue which heavily effects charging frequency and infrastructure and, therefore, the widespread use of EVs. Although several factors that influence energy consumption of EVs have been identified, a key technology that can make electric vehicles more energy efficient is drivetrain design and development. Based on electric motors’ high torque capabilities, single-speed transmissions are preferred on many light and urban vehicles. In the context of this paper, a prototype electric vehicle is used as a test bed to evaluate energy consumption related to different gear ratio usage on single-speed transmission. For this purpose, real-time data are recorded from experimental road tests and a dynamic model of the vehicle is created and fine-tuned using dedicated software. Dynamic simulations are performed to compare and evaluate different gear ratio set-ups, providing valuable insights into their effect on energy consumption. The correlation of experimental and simulation data is used for the validation of the dynamic model and the evaluation of the results towards the selection of the optimal gear ratio. Based on the aforementioned data, we provide useful information from numerous experimental and simulation results that can be used to evaluate gear ratio effects on electric vehicles’ energy consumption and, at the same time, help to formulate evolving concepts of smart grid and EV integration.

In this paper the development of a low cost robotic vehicle for research and education is presented. The vehicle was designed considering minimum cost and maximum capabilities. As a base for testing devices and different type of sensors, a commercially available vehicle was used and modified. Two different version of the prototype vehicle were developed accompanied by the proper software that allows the end user to operate the vehicles as an educational or research platform. The functionality of the vehicles was verified after extensive experimentation.

In this paper, a detailed study and implementation of a reliable and efficient regenerative braking system is presented. It is applied on a prototype electric vehicle, that uses a hydrogen fuel cell as its only power source. Supercapacitors are used to store the energy that is generated during braking, transistors for switching the alternative circuits and an embedded computer controller program undertakes the synchronization of the system tasks. A finite state machine was designed to create a simple but robust technique to control the transistor switches according to the system's sensory inputs. The system is powered by its own supercapacitors, and thus it may be used in a plug-and-play manner. On-road test drives proved the system's reliability and efficiency.

The automotive industry has been rapidly transforming and moving further from internal combustion engines, towards hybrid or electric vehicles. A key component for the successful adoption of the aforementioned approach is their Energy Management Systems (EMSs). In the proposed work, we describe in detail a custom EMS, with unique characteristics, which was developed and installed in a hydrogen-powered prototype vehicle. The development of the EMS was based on off-the-shelf components and the adoption of a Robot Operating System (ROS), a meta-operating system developed for robotic-oriented applications. Our approach offers soft real-time control and the ability to organize the controller of the EMS as a straightforward and comprehensive message system that provides the necessary inter-process communication at the core of the EMS control procedure. We describe in detail the software-based implementation and validate our approach through experimental results obtained while the prototype was racing in a low-energy consumption competition.