Optimizing the cooling system plays a central role for capping the data center power consumption. However, the performance of traditional cooling management strategies is not satisfactory due to the complexity of thermodynamic process. Recently, several works leveraged Reinforcement Learning (RL) to improve the energy efficiency of data center cooling system. While they demonstrated that it is possible to reduce the cooling power consumption via RL, there are still some key challenges that have to be addressed before field deployment, such as safe operation guarantee and sample complexity, etc. In this paper, we propose SafeCool , an actor-critic Model-Based Reinforcement Learning (MBRL) algorithm for data center cooling management. SafeCool incorporates two system models, i.e., a transition model to predict the future system state and a risk model to estimate the negative effect of executing an action. The safety of proposed algorithm is ensured by Model Predictive Control (MPC) and risk-guided exploration. In addition, by employing the MBRL framework, SafeCool achieves higher sample efficiency and accelerated convergence. Simulations using real-world workload trace reveal that SafeCool saves up to 13.18% cooling power compared with state-of-the-art MBRL data center cooling solutions.

Sharding technology, which divides a network into multiple disjoint groups so that transactions can be processed in parallel, is applied to blockchain systems as a promising solution to improve Transactions Per Second (TPS). This paper considers the Optimal Blockchain Sharding (OBCS) problem as a Markov Decision Process (MDP) where the decision variables are the number of shards, block size and block interval. Previous works solved the OBCS problem via Deep Reinforcement Learning (DRL)-based methods, where the action space must be discretized to increase processability. However, the discretization degrades the quality of the solution since the optimal solution usually lies between discrete values. In this paper, we treat the block size and block interval as continuous decision variables and provide dynamic sharding strategies based on them. The Branching Dueling Q-Network Blockchain Sharding (BDQBS) algorithm is designed for discrete action spaces. Compared with traditional DRL algorithms, the BDQBS overcomes the drawbacks of high action space dimensions and difficulty in training neural networks. And it improves the performance of the blockchain system by 1.25 times. We also propose a sharding control algorithm based on the Parameterized Deep Q-Networks (P-DQN) algorithm, i.e., the Parameterized Deep Q-Networks Blockchain Sharding (P-DQNBS) algorithm, to efficiently handle the discrete–continuous hybrid action space without the scalability issues. Also, the method can effectively improve the TPS by up to $28\%$.

Data stored in the Internet of Vehicles (IoV) face problems with ease of tampering, easy disclosure and single access control. Based on this problem, we propose an access control scheme for the IoV based on blockchain, trust values and weighted attribute-based encryption, called the Blockchain Trust and Weighted Attribute-Based Access Control Strategy (BTWACS). First, we utilize both local and global blockchains to jointly maintain the generation, verification and storage of blocks, achieving distributed data storage and ensuring that data cannot arbitrarily be tampered with. Local blockchain mainly uses Road Side Unit (RSU) technology to calculate trust values, while global blockchain is mainly responsible for data storage and access policy selection. Secondly, we design a blockchain-based trust evaluation scheme called Blockchain-Based Trust Evaluation (BBTE). In this evaluation scheme, the trust value of the vehicle node is based on four factors: initial trust, historical experience trust, recommendation trust and RSU observation trust. CRITIC is used to determine the optimal weights of four factors to obtain the trust value. Then, we use the Network Simulator version 3 (NS3) to verify the security and accuracy of BBTE, improving the recognition accuracy and detection rate of malicious vehicle nodes. Finally, by mining the association relationships between attribute permissions among various roles, we construct a hierarchical access control strategy based on weight and trust, and further optimize the access strategy through pruning techniques. The experiment results indicate that this scheme can effectively respond to gray hole attacks, defamation attacks and collusion attacks from other vehicle nodes. This method can effectively reduce the computing and transmission costs of vehicles and meet the access requirements of multiple entities and roles in the IoV.

With the continuous expansion of data centers, their carbon emission has become a serious issue. A number of studies are committing to reduce the carbon emission of data centers. Carbon trading, carbon capture, and power-to-gas technologies are promising emission reduction techniques which are, however, seldom applied to data centers. To bridge this gap, we propose a carbon-neutral architecture for distributed data centers, where each data center consists of three subsystems, i.e., an energy subsystem for energy supply, thermal subsystem for data center cooling, and carbon subsystem for carbon trading. Then, we formulate the energy management problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) and develop a distributed solution framework using Multi-Agent Deep Deterministic Policy Gradient (MADDPG). Finally, simulations using real-world data show that a cost saving of 20.3% is provided.

To address the increasingly severe issue of insufficient vehicular caching in the Internet of Vehicles (IoV), Vehicular edge computing uses base stations (BS) and vehicles with computing and storage capabilities as edge servers to cache content. However, most existing studies focus on reducing task costs, assuming vehicles can simultaneously act as requesters and providers, without considering whether a vehicle is suitable to serve as a provider, willing to share resources or ensure secure data transmission, thus lowering matching efficiency. Deep reinforcement learning and blockchain technology were integrated into the Internet of Vehicles to address distributed vehicle-to-vehicle collaborative caching. An optimal collaborative caching strategy was designed, considering dynamic vehicle attributes, with the system's objective being to maximize vehicle utility to encourage resource sharing. Simulation results confirm that the proposed blockchain-based vehicular edge computing collaborative caching scheme with deep reinforcement learning significantly outperforms other strategies.

In the consensus process of blockchain, the miner who successfully mines the block first is the one who receives the reward.Mining requires a large amount of computing resources, the limited resources of the vehicle equipment can not meet this requirement. The emergence of Mobile Edge Computing (MEC) technology can effectively solve this problem, miners can rent the resources of MEC servers to get rewards. However, how to balance the rewards and expenditures to maximise the miner's gain still faces a huge challenge.Existing studies are paid with fixed payment rules after resource allocation, and miners are unable to pay according to their own needs and budgets.Therefore, to solve the problem of Miner gain, this paper proposes the IoV Blockchain Miner Gain Optimisation Algorithm. The method uses partial offloading to process mining tasks, and miners bid for the resources of MEC servers. The simulation experiment results verify the performance of the proposed method, which can effectively improve the mining gain of miners compared with other baseline algorithms.