This research article presents a comparison between two mainstream Deep Reinforcement Learning (DRL) algorithms, Asynchronous Advantage Actor-Critic (A3C) and Proximal Policy Optimization (PPO), in the context of two diverse environments: CartPole and Lunar Lander. DRL algorithms are widely known for their effectiveness in training agents to navigate complex environments and achieve optimal policies. Nevertheless, a methodical assessment of their effectiveness in various settings is crucial for comprehending their advantages and disadvantages. In this study, we conduct experiments on the CartPole and Lunar Lander environments using both A3C and PPO algorithms. We compare their performance in terms of convergence speed and stability. Our results indicate that A3C typically achieves quicker training times, but exhibits greater instability in reward values. Conversely, PPO demonstrates a more stable training process at the expense of longer execution times. An evaluation of the environment is needed in terms of algorithm selection, based on specific application needs, balancing between training time and stability. A3C is ideal for applications requiring rapid training, while PPO is better suited for those prioritizing training stability.

Background: In the last decade, numerous methods have been proposed to define and detect outliers, particularly in complex environments like networks, where anomalies significantly deviate from normal patterns. Although defining a clear standard is challenging, anomaly detection systems have become essential for network administrators to efficiently identify and resolve irregularities. Methods: This study develops and evaluates a machine learning-based system for network anomaly detection, focusing on point anomalies within network traffic. It employs both unsupervised and supervised learning techniques, including change point detection, clustering, and classification models, to identify anomalies. SHAP values are utilized to enhance model interpretability. Results: Unsupervised models effectively captured temporal patterns, while supervised models, particularly Random Forest (94.3%), demonstrated high accuracy in classifying anomalies, closely approximating the actual anomaly rate. Conclusions: Experimental results indicate that the system can accurately predict network anomalies in advance. Congestion and packet loss were identified as key factors in anomaly detection. This study demonstrates the potential for real-world deployment of the anomaly detection system to validate its scalability.

Air pollution and climate change are some of the main problems that humankind is currently facing. The electrification of the transport sector will help to reduce these problems, but one of the major barriers for the massive adoption of electric vehicles is their limited range. The energy consumption in these vehicles is affected, among other variables, by the driving behavior, making range a value that must be personalized to each driver and each type of electric vehicle. In this paper we offer a way to estimate a personalized energy consumption model by the use of the vehicle dynamics and the driving events detected by the use of the smartphone inertial sensors, allowing an easy and non-intrusive manner to predict the correct range for each user. This paper proposes, for the classification of events, a deep neural network (Long-Short Time Memory) which has been trained with more than 22,000 car trips, and the application to improve the consumption model taking into account the driver behavior captured across different trips, allowing a personalized prediction. Results and validation in real cases show that errors in the predicted consumption values are halved when abrupt events are considered in the model.