The aim of this paper is to find the optimal mother wavelet function and wavelet decomposition level when denoising the Doppler cardiogram (DCG), the heart signal obtained by the Doppler radar sensor system. To select the best suited mother wavelet function and wavelet decomposition level, this paper presents the quantitative analysis results. Both the optimal mother wavelet and decomposition level are selected by evaluating signal-to-noise-ratio (SNR) efficiency of the denoised signals obtained by using the wavelet thresholding method. A total of 115 potential functions from six wavelet families were examined for the selection of the optimal mother wavelet function and 10 levels (1 to 10) were evaluated for the choice of the best decomposition level. According to the experimental results, the most efficient selections of the mother wavelet function are “db9” and “sym9” from Daubechies and Symlets families, and the most suitable decomposition level for the used signal is seven. As the evaluation criterion in this study rates the efficiency of the denoising process, it was found that a mother wavelet function longer than 22 is excessive. The experiment also revealed that the decomposition level can be predictable based on the frequency features of the DCG signal. The proposed selection of the mother wavelet function and the decomposition level could reduce noise effectively so as to improve the quality of the DCG signal in information field.

In this paper, we propose a deep deterministic policy gradient (DDPG)-based path-planning method for mobile robots by applying the hindsight experience replay (HER) technique to overcome the performance degradation resulting from sparse reward problems occurring in autonomous driving mobile robots. The mobile robot in our analysis was a robot operating system-based TurtleBot3, and the experimental environment was a virtual simulation based on Gazebo. A fully connected neural network was used as the DDPG network based on the actor–critic architecture. Noise was added to the actor network. The robot recognized an unknown environment by measuring distances using a laser sensor and determined the optimized policy to reach its destination. The HER technique improved the learning performance by generating three new episodes with normal experience from a failed episode. The proposed method demonstrated that the HER technique could help mitigate the sparse reward problem; this was further corroborated by the successful autonomous driving results obtained after applying the proposed method to two reward systems, as well as actual experimental results.

In this paper, we propose a reinforcement learning-based end-to-end learning method for the autonomous driving of a mobile robot in a dynamic environment with obstacles. Applying two additional techniques for reinforcement learning simultaneously helps the mobile robot in finding an optimal policy to reach the destination without collisions. First, the multifunctional reward-shaping technique guides the agent toward the goal by utilizing information about the destination and obstacles. Next, employing the hindsight experience replay technique to address the experience imbalance caused by the sparse reward problem assists the agent in finding the optimal policy. We validated the proposed technique in both simulation and real-world environments. To assess the effectiveness of the proposed method, we compared experiments for five different cases.

This paper proposes a task decomposition and dedicated reward-system-based reinforcement learning algorithm for the Pick-and-Place task, which is one of the high-level tasks of robot manipulators. The proposed method decomposes the Pick-and-Place task into three subtasks: two reaching tasks and one grasping task. One of the two reaching tasks is approaching the object, and the other is reaching the place position. These two reaching tasks are carried out using each optimal policy of the agents which are trained using Soft Actor-Critic (SAC). Different from the two reaching tasks, the grasping is implemented via simple logic which is easily designable but may result in improper gripping. To assist the grasping task properly, a dedicated reward system for approaching the object is designed through using individual axis-based weights. To verify the validity of the proposed method, wecarry out various experiments in the MuJoCo physics engine with the Robosuite framework. According to the simulation results of four trials, the robot manipulator picked up and released the object in the goal position with an average success rate of 93.2%.

This paper deals with an admissibilization problem of singular systems with uniform input quantization. The aim is to design a controller to guarantee the admissibility of the closed-loop system. To achieve this, this paper proposes a proportional-derivative state-feedback controller which includes non-linear control part to reject the effect of uniform input quantization. Based on the proposed controller, sufficient conditions are obtained in terms of linear matrix inequalities. Two examples show the feasibility of the proposed controller.