Public blockchain networks using proof of work-based consensus protocols are considered a promising platform for decentralized resource management with financial incentive mechanisms. In order to maintain a secured, universal state of the blockchain, proof of work-based consensus protocols financially incentivize the nodes in the network to compete for the privilege of block generation through cryptographic puzzle solving. For rational consensus nodes, i.e., miners with limited local computational resources, offloading the computation load for proof of work to the cloud/fog providers becomes a viable option. In this paper, we study the interaction between the cloud/fog providers and the miners in a proof of work-based blockchain network using a game theoretic approach. In particular, we propose a lightweight infrastructure of the proof of work-based blockchains, where the computation-intensive part of the consensus process is offloaded to the cloud/fog. We formulate the computation resource management in the blockchain consensus process as a two-stage Stackelberg game, where the profit of the cloud/fog provider and the utilities of the individual miners are jointly optimized. In the first stage of the game, the cloud/fog provider sets the price of offered computing resource. In the second stage, the miners decide on the amount of service to purchase accordingly. We apply backward induction to analyze the sub-game perfect equilibria in each stage for both uniform and discriminatory pricing schemes. For uniform pricing where the same price applies to all miners, the uniqueness of the Stackelberg equilibrium is validated by identifying the best response strategies of the miners. For discriminatory pricing where the different prices are applied, the uniqueness of the Stackelberg equilibrium is proved by capitalizing on the variational inequality theory. Further, the real experimental results are employed to justify our proposed model.

In proof-of-work-based blockchain networks, the block miners participate in a crypto-puzzle solving competition to win the reward of publishing (i.e., mining) new blocks. Due to the remarkable difficulty of the crypto-puzzle, individual miners tend to join mining pools to secure stable profits. We study the dynamics of mining pool selection in a blockchain network, where mining pools may choose arbitrary block mining strategies. We identify the hash rate for puzzle-solving and the block propagation delay as two major factors determining the mining competition results. We then model the strategy evolution of individual miners as an evolutionary game. We provide the theoretical analysis of evolutionary stability in the pool selection dynamics for a two-pool case. Numerical simulations support our theoretical findings as well as demonstrate the stability in the evolution of miners’ strategies in a general case.

Unplanned breakdown of critical equipment interrupts production throughput in Industrial IoT (IIoT), and data-driven predictive maintenance (PdM) becomes increasingly important for companies seeking a competitive business advantage. Manufacturers, however, are constantly faced with the onerous challenge of manually allocating suitably competent manpower resources in the event of an unexpected machine breakdown. Furthermore, human error has a negative rippling impact on both overall equipment downtime and production schedules. In this article, we formulate the complex resource management problem as a resource optimization problem to determine if a model-free deep reinforcement learning (DRL)-based PdM framework can be used to automatically learn an optimal decision policy from a stochastic environment. Unlike the existing PdM frameworks, our approach considers PdM sensor information and the resources of both physical equipment and human as part of the optimization problem. The proposed DRL-based framework and proximal policy optimization long short term memory (PPO-LSTM) model are evaluated alongside baselines results from human participants using a maintenance repair simulator. Empirical results indicate that our PPO-LSTM efficiently learns the optimal decision-policy for the resource management problem, outperforming comparable DRL methods and human participants by 53% and 65%, respectively. Overall, the simulation results corroborate the proposed DRL-based PdM framework’s superiority in terms of convergence efficiency, simulation performance, and flexibility.

In this paper, we study the transmission strategy adaptation problem in an RF-powered cognitive radio network, in which hybrid secondary users are able to switch between the harvest-then-transmit mode and the ambient backscatter mode for their communication with the secondary gateway. In the network, a monetary incentive is introduced for managing the interference caused by the secondary transmission with imperfect channel sensing. The sensing-pricing-transmitting process of the secondary gateway and the transmitters is modeled as a singleleader- multi-follower Stackelberg game. Furthermore, the follower sub-game among the secondary transmitters is modeled as a generalized Nash equilibrium problem with shared constraints. Based on our theoretical discoveries regarding the properties of equilibria in the follower sub-game and the Stackelberg game, we propose a distributed, iterative strategy searching scheme that guarantees the convergence to the Stackelberg equilibrium. The numerical simulations show that the proposed hybrid transmission scheme always outperforms the schemes with fixed transmission modes. Furthermore, the simulations reveal that the adopted hybrid scheme is able to achieve a higher throughput than the sum of the throughput obtained from the schemes with fixed transmission modes.

The Industrial Internet of Things (IIoT) is crucial for accurately assessing the state of complex equipment in order to perform predictive maintenance (PdM) successfully. However, existing IIoT-based PdM frameworks do not consider the influence of various practical yet complex system factors, such as the real-time production states, machine health, and maintenance manpower resources. For this reason, we propose a generic PdM optimization framework to assist maintenance teams in prioritizing and resolving maintenance task conflicts under real-world manufacturing conditions. Specifically, the PdM framework aims to jointly optimize the edge-based machine network uptime and the allocation of manpower resources in a stochastic IIoT-enabled manufacturing environment using the model-free deep reinforcement learning (DRL) methods. Since DRL requires a significant amount of training data, we propose and demonstrate the use of the transfer learning (TL) method to assist DRL in learning more efficiently by incorporating expert demonstrations, termed TL with demonstrations (TLDs). TLD reduces training wall time by 58% compared to baseline methods, and we conduct numerous experiments to illustrate the performance, robustness, and scalability of TLD. Finally, we discuss the general benefits and limitations of the proposed TL method, which are not well addressed in the existing literature but could be beneficial to both researchers and industry practitioners.

Industrial systems require reliable predictive maintenance strategies to enhance operational efficiency and reduce downtime. Existing studies rely on the heuristic models which may struggle to capture complex temporal dependencies. This article introduces an integrated framework that leverages the capabilities of the Transformer and deep reinforcement learning (DRL) algorithms to optimize the system maintenance actions. Our approach employs the Transformer model to effectively capture complex temporal patterns in IoT sensor data, thus accurately predicting the remaining useful life (RUL) of equipment. Additionally, the DRL component of our framework provides cost-effective and timely maintenance recommendations. Numerous experiments conducted on the NASA C-MPASS data set demonstrate that our approach has a performance similar to the ground truth results and could be obviously better than the baseline methods in terms of RUL prediction accuracy as the time cycle increases. Additionally, the experimental results demonstrate the effectiveness of optimizing maintenance actions.