4 November 2020 in IEEE Sensors Journal
Wireless sensor networks (WSNs) are typically used with dynamic conditions of task-related environments for sensing(monitoring) and gathering of raw sensor data for subsequent forwarding to a base station. In order to deploy WSNs in real environments, a variety of technical challenges must be addressed. With traditional techniques developed for a specific task, it is hard to react in dynamic situations beyond the scope of the intended task. As a solution to this problem, machine learning (ML) techniques that are able to handle dynamic situations with successful learning process have been applied lately in WSNs. Particularly, deep learning (DL) techniques, a class of ML techniques characterized by the use of deep neural network, are used for WSNs to extract higher level features from raw sensor data. A range of benefits obtained from ML techniques applied to WSNs can be described as reduced computational complexity, increased feasibility in finding optimal solutions, increased energy efficiency, etc. On the other hand, it is found from our survey that large training time and large dataset to get acceptable performance are accompanied with large energy consumption which is not favorable for resource-restrained WSNs. Reviews on the applications of ML techniques in WSNs appeared in the literature. However, few reviews have dealt with the applications of DL techniques in WSNs. In this review, recent developments of ML techniques for WSNs are presented with much emphasis on DL techniques. The DL techniques developed for various applications in WSNs are addressed together with their respective deep neural network architectures.
27 July 2020 in IEEE Access
In multigoal reinforcement learning, an agent interacts with an environment and learns to achieve multiple goals. The goal-conditioned policy is trained to effectively generalize its behavior for multiple goals. During training, the experiences collected by the agent are randomly sampled from a replay buffer. Because biased sampling of achieved goals affects the success rate of a given task, it should be avoided by considering the valid goal space, introduced here as the set of goals to achieve, and the current competence of the policy. To this end, a novel prioritization method for creation of batches, e.g., collections of samples, is proposed. Candidate batches are sampled and associated with costs; in each iteration the batch with the minimum cost is chosen to train the policy. The cost function is modeled by an intended goal, which is proposed as a hypothetical goal that the policy is trying to learn in each cycle, and the information of the valid goal space. The minimum cost of the batch selected for each iteration decreases throughout training as the policy learns to achieve goals near the center of the valid goal space. The proposed batch prioritization method is combined with hindsight experience replay (HER) for experiments in robotic control tasks presented in the OpenAI gym suite to demonstrate learning performance comparable to that of other state-of-the-art prioritization methods. As a result, the proposed batch prioritization method can achieve improved learning performance in 4 out of 5 tasks, particularly for harder tasks. The experimental results suggest that the proposed method for the creation of training batches, using the valid goal space information and current competence of the policy, can enhance learning performance in multigoal tasks with high-dimensional goal space.