Optimizing the cooling system plays a central role for capping the data center power consumption. However, the performance of traditional cooling management strategies is not satisfactory due to the complexity of thermodynamic process. Recently, several works leveraged Reinforcement Learning (RL) to improve the energy efficiency of data center cooling system. While they demonstrated that it is possible to reduce the cooling power consumption via RL, there are still some key challenges that have to be addressed before field deployment, such as safe operation guarantee and sample complexity, etc. In this paper, we propose SafeCool , an actor-critic Model-Based Reinforcement Learning (MBRL) algorithm for data center cooling management. SafeCool incorporates two system models, i.e., a transition model to predict the future system state and a risk model to estimate the negative effect of executing an action. The safety of proposed algorithm is ensured by Model Predictive Control (MPC) and risk-guided exploration. In addition, by employing the MBRL framework, SafeCool achieves higher sample efficiency and accelerated convergence. Simulations using real-world workload trace reveal that SafeCool saves up to 13.18% cooling power compared with state-of-the-art MBRL data center cooling solutions.

Sharding technology, which divides a network into multiple disjoint groups so that transactions can be processed in parallel, is applied to blockchain systems as a promising solution to improve Transactions Per Second (TPS). This paper considers the Optimal Blockchain Sharding (OBCS) problem as a Markov Decision Process (MDP) where the decision variables are the number of shards, block size and block interval. Previous works solved the OBCS problem via Deep Reinforcement Learning (DRL)-based methods, where the action space must be discretized to increase processability. However, the discretization degrades the quality of the solution since the optimal solution usually lies between discrete values. In this paper, we treat the block size and block interval as continuous decision variables and provide dynamic sharding strategies based on them. The Branching Dueling Q-Network Blockchain Sharding (BDQBS) algorithm is designed for discrete action spaces. Compared with traditional DRL algorithms, the BDQBS overcomes the drawbacks of high action space dimensions and difficulty in training neural networks. And it improves the performance of the blockchain system by 1.25 times. We also propose a sharding control algorithm based on the Parameterized Deep Q-Networks (P-DQN) algorithm, i.e., the Parameterized Deep Q-Networks Blockchain Sharding (P-DQNBS) algorithm, to efficiently handle the discrete–continuous hybrid action space without the scalability issues. Also, the method can effectively improve the TPS by up to $28\%$.

The output power prediction of wind farm is the key to effective utilization of wind energy and reduction of wind curtailment. However, the prediction of output power has long been a difficulty faced by both academia and the wind power industry, due to the high stochasticity of wind energy. This paper attempts to improve the ultra-short-term prediction accuracy of output power in wind farm. For this purpose, an output power prediction model was constructed for wind farm based on the time sliding window (TSW) and long short-term memory (LSTM) network. Firstly, the wind power data from multiple sources were fused, and cleaned through operations like dimension reduction and standardization. Then, the cyclic features of the actual output powers were extracted, and used to construct the input dataset by the TSW algorithm. On this basis, the TSW-LSTM prediction model was established to predict the output power of wind farm in ultra-short-term. Next, two regression evaluation metrics were designed to evaluate the prediction accuracy. Finally, the proposed TSW-LSTM model was compared with four other models through experiments on the dataset from an actual wind farm. Our model achieved a super-high prediction accuracy 92.7% as measured by d_MAE, an evidence of its effectiveness. To sum up, this research simplifies the complex prediction features, unifies the evaluation metrics, and provides an accurate prediction model for output power of wind farm with strong generalization ability.

The rapid increase in carbon emissions from the logistics transportation industry has underscored the urgent need for low-carbon logistics solutions. Electric logistics vehicles (ELVs) are increasingly being considered as replacements for traditional fuel-powered vehicles to reduce emissions in urban logistics. However, ELVs are typically limited by their battery capacity and load constraints. Additionally, effective scheduling of charging and the management of transportation duration are critical factors that must be addressed. This paper addresses low energy consumption scheduling (LECS) problem, which aims to minimize the total energy consumption of heterogeneous ELVs with varying load and battery capacities, considering the availability of multiple charging stations (CSs). Given that the complexity of LECS problem, this study proposes a heterogeneous attention model based on encoder-decoder architecture (HAMEDA) approach, which employs a heterogeneous graph attention network and introduces a novel decoding procedure to enhance solution quality and learning efficiency during the encoding and decoding phases. Trained via deep reinforcement learning (DRL) in an unsupervised manner, HAMEDA is adept at autonomously deriving optimal transportation routes for each ELV from specific cases presented. Comprehensive simulations have verified that HAMEDA can diminish overall energy utilization by no less than 1.64% compared with other traditional heuristic or learning-based algorithms. Additionally, HAMEDA excels in maintaining an advantageous equilibrium between execution speed and the quality of solutions, rendering it exceptionally apt for expansive tasks that necessitate swift decision-making processes.

Although the traditional P2P botnet has significant resilience against termination, its dependence on neighbor lists (NL) has left it vulnerable to infiltration and destruction. In addition, it is not sufficient in protecting the botmaster’s identity. To overcome these weaknesses, we proposed BlockchainBot, a botnet model that leveraged IoT devices as maintainers, and integrated blockchain, also known as distributed ledger technology (DLT). The BlockchainBot was able to fully deploy bots on public blockchains. It was versatile for multiple botnet applications and eliminated the dependence on NL. In addition, we further introduced a novel method, the forking of a channel, to kick out spy nodes that infiltrate a botnet. To further enforce the resistance against a single point of failure (SPoF), we introduced bot-cluster dispersing to prevent clustering around full nodes and more evenly scatter bots to prevent hostile takeovers. The analysis of the security of BlockchainBot indicated that it had strong resilience against DDoS attacks, Sybil attacks, and forensic investigations. Furthermore, the security of the forking of the channel and bot-cluster dispersing were also shown to be effective. The robustness of the BlockchainBot against the Sybil attack was also briefly discussed. Experimental results authenticated the effectiveness and performance of the BlockchainBot, as compared to previous models.

Data stored in the Internet of Vehicles (IoV) face problems with ease of tampering, easy disclosure and single access control. Based on this problem, we propose an access control scheme for the IoV based on blockchain, trust values and weighted attribute-based encryption, called the Blockchain Trust and Weighted Attribute-Based Access Control Strategy (BTWACS). First, we utilize both local and global blockchains to jointly maintain the generation, verification and storage of blocks, achieving distributed data storage and ensuring that data cannot arbitrarily be tampered with. Local blockchain mainly uses Road Side Unit (RSU) technology to calculate trust values, while global blockchain is mainly responsible for data storage and access policy selection. Secondly, we design a blockchain-based trust evaluation scheme called Blockchain-Based Trust Evaluation (BBTE). In this evaluation scheme, the trust value of the vehicle node is based on four factors: initial trust, historical experience trust, recommendation trust and RSU observation trust. CRITIC is used to determine the optimal weights of four factors to obtain the trust value. Then, we use the Network Simulator version 3 (NS3) to verify the security and accuracy of BBTE, improving the recognition accuracy and detection rate of malicious vehicle nodes. Finally, by mining the association relationships between attribute permissions among various roles, we construct a hierarchical access control strategy based on weight and trust, and further optimize the access strategy through pruning techniques. The experiment results indicate that this scheme can effectively respond to gray hole attacks, defamation attacks and collusion attacks from other vehicle nodes. This method can effectively reduce the computing and transmission costs of vehicles and meet the access requirements of multiple entities and roles in the IoV.