Modern IoT industry cannot operate well without task scheduling, while the merging of edge computing and blockchain can empower more secure task scheduling models in distributed IoT environments. Due to various desirable properties (self-verifying, self-executing, immutability, reliability, confidentiality, etc.) provided by blockchain, the tasks generated from IoT devices can be safely scheduled. However, merely incorporating a blockchain framework into an IoT edge computing environment cannot guarantee the most desirable rewards for task participants. So in this article, we propose to use a Nash bargaining method in blockchain smart contract designing to strive the rewards for task participants and stimulate their proactivity while extending the above properties. The proposed blockchain-based Nash bargaining architecture can be applied in many distributed IoT scenarios, such as smart logistics, health data exchange, etc. For a practical implementation, a configurable blockchain architecture is developed consisting of both regular and constrained IoT devices. We additionally implement mutual authentication protocol and proof of authority to maintain the privacy and security for scheduling. Further, we experimentally study the feasibility of such architecture built in static and mobile IoT devices. Evaluations demonstrate that, the proposed architecture can well support a task scheduling process with tamper resistance, which is suitable for IoT edge computing with high-level security and creditability.
11 January 2023 in IEEE Transactions on Automation Science and Engineering
Value function approximation, such as Q-learning, is widely used in the discrete control rather than the continuous one because the optimal action in the discrete setting is more easily selected. Optimizing the action is a non-convex optimization problem with respect to the complex value function. Some notable studies simplify the non-convex optimization problem by assuming the value function as quadratic in the actions or by discretizing the action space. However, the performance of the output policy will decline if these studies’ premises do not hold. In order to address the problem, we propose a framework that combines swarm intelligence algorithms with value-based Reinforcement Learning, where the swarm intelligence algorithms are employed to search for the optimal action with respect to the state and the value function. To ensure the correctness of this framework, we conditionally claim the convergence rate of swarm intelligence algorithms with high probability. We then implement it by searching the batch optimal actions to various states on the GPU platform for the batch training. Furthermore, we employ the population-based atomic actions for the compatibility with the existing related work about solving discrete control problems. Four classical control models and four robot simulation environments are utilized in the comparisons. According to empirical results, our framework outputs a policy comparable with that of the policy-based algorithms by 10% timesteps in the continuous control. Note to Practitioners —This paper is motivated by the exploration-exploitation dilemma of Reinforcement Learning to solve continuous control tasks. To balance the exploration and exploitation, the stochastic exploration and the prioritized exploration are roughly two feasible ways, where the prioritized one is a better choice due to the higher data efficiency than the stochastic one, e.g. $\varepsilon$ -greedy. Normally, the prioritized exploration works well in the value-based Reinforcement Learning algorithms rather than the policy-based ones; meanwhile, the policy-based algorithms are more suitable to continuous control tasks than the value-based ones. To tackle this conflict, we especially design a particle swarm optimization to maximize the Q-value of action in Q-learning. Our design can be hybridized by various swarm intelligence and value-based Reinforcement Learning algorithms. Also, it can be embedded in most intelligent control systems easily. The aim of this study is to solve the continuous control tasks by value-based algorithms as the first step of applying the prioritized exploration. The simulative results verify the effectiveness and efficiency of our design.
Optimizing agricultural structure serves as a crucial pathway to promote sustainable rural economic development. This study focuses on a representative village in the mountainous region of North China, where agricultural production is constrained by perennial low-temperature conditions, resulting in widespread adoption of single-cropping systems. There exists an urgent need to enhance both economic returns and risk resilience of limited arable land through refined cultivation planning. However, traditional planting strategies face difficulties in synergistically optimizing long-term benefits from multi-crop combinations, while remaining vulnerable to climate fluctuations, market volatility, and complex inter-crop relationships. These limitations lead to constrained land productivity and inadequate economic resilience. To address these challenges, we propose an integrated decision-making approach combining stochastic programming, robust optimization, and data-driven modeling. The methodology unfolds in three phases: First, we construct a stochastic programming model targeting seven-year total profit maximization, which quantitatively analyzes relationships between decision variables (crop planting areas) and stochastic variables (climate/market factors), with optimal planting solutions derived through robust optimization algorithms. Second, to address natural uncertainties, we develop an integer programming model for ideal scenarios, obtaining deterministic optimization solutions via genetic algorithms. Furthermore, this study conducts correlation analyses between expected sales volumes and cost/unit price for three crop categories (staples, vegetables, and edible fungi), establishing both linear and nonlinear regression models to quantify how crop complementarity–substitution effects influence profitability. Experimental results demonstrate that the optimized strategy significantly improves land-use efficiency, achieving a 16.93% increase in projected total revenue. Moreover, the multi-scenario collaborative optimization enhances production system resilience, effectively mitigating market and environmental risks. Our proposal provides a replicable decision-making framework for sustainable intensification of agriculture in cold-region rural areas.