Since the submarine has become the major threat to maritime security, there is an urgent need to find a more efficient method of anti-submarine warfare (ASW). The digital twin theory is one of the most outstanding information technologies, and has been quite popular in recent years. The most influential change produced by digital twin is the ability to enable real-time dynamic interactions between the simulation world and the real world. Digital twin can be regarded as a paradigm by means of which selected online measurements are dynamically assimilated into the simulation world, with the running simulation model guiding the real world adaptively in reverse. By combining digital twin theory and random finite sets (RFSs) closely, a new framework of sensor control in ASW is proposed. Two key algorithms are proposed for supporting the digital twin-based framework. First, the RFS-based data-assimilation algorithm is proposed for online assimilating the sequence of real-time measurements with detection uncertainty, data association uncertainty, noise, and clutters. Second, the computation of the reward function by using the results of the proposed data-assimilation algorithm is introduced to find the optimal control action. The results of three groups of experiments successfully verify the feasibility and effectiveness of the proposed approach.

Control charts are widely used in statistical process control (SPC) to monitor the quality of products or production processes. When dealing with a variable (e.g., the diameter of a shaft, the hardness of a component surface), it is necessary to monitor both its mean and variability (Montgomery 2009 Montgomery, DC . 2009. Introduction to statistical quality control, New York : John Wiley & Sons. [Google Scholar] [Montgomery, D.C., 2009. Introduction to statistical quality control. New York: John Wiley & Sons.]). This article studies and compares the overall performances of the X chart and the 3-CUSUM chart for this purpose. The latter is a combined scheme incorporating three individual CUSUM charts and is considered as the most effective scheme for detecting mean shift δ_μ and/or standard deviation shift δ_σ in current SPC literature. The results of the performance studies reveal two interesting findings: (1) the best sample size n for an chart is always n = 1, in other words, the simplest X chart (i.e., the chart with n = 1) is the most effective chart for detecting δ_μ and/or δ_σ; (2) the simplest X chart often outperforms the 3-CUSUM chart from an overall viewpoint unless the latter is redesigned by a difficult optimisation procedure. However, even the optimal 3-CUSUM chart is only slightly more effective than the X chart unless the process shift domain is quite small. Since the X chart is very simple to understand, implement and design, it may be more suitable in many SPC applications, in which both the mean and variance of a variable need to be monitored.

With the gradual expansion of high buildings and underground spaces, deep foundation pits have been widely used in these engineering projects, but if they are not well-designed, safety problems occur. Proper deep foundation pit design requires proper exit distribution. However, calculating an adequate number of exit distributions for evaluation is difficult due to the numerous influential factors existing in the deep foundation pit environment. To this end, this paper presents a prototype of a decision-making system that uses agent-based modeling to simulate deep foundation pit evacuation in the presence of collapse disaster. By modeling the collapse occurrence process and agent escape process, an agent-based evacuation model is built, and a modified simulation-based particle swarm optimization algorithm is used to solve the optimization problem of exit distribution. Extensive experiments are conducted to verify the system, and the results show that the system provides a feasible framework for deep foundation pit evacuation.

Hierarchical reinforcement learning works on temporally extended actions or skills to facilitate learning. How to automatically form such abstraction is challenging, and many efforts tackle this issue in the options framework. While various approaches exist to construct options from different perspectives, few of them concentrate on options’ adaptability during learning. This paper presents an algorithm to create options and enhance their quality online. Both aspects operate on detected communities of the learning environment’s state transition graph. We first construct options from initial samples as the basis of online learning. Then a rule-based community revision algorithm is proposed to update graph partitions, based on which existing options can be continuously tuned. Experimental results in two problems indicate that options from initial samples may perform poorly in more complex environments, and our presented strategy can effectively improve options and get better results compared with flat reinforcement learning.

In the military field, multi-agent simulation (MAS) plays an important role in studying wars statistically. For a military simulation system, which involves large-scale entities and generates a very large number of interactions during the runtime, the issue of how to improve the running efficiency is of great concern for researchers. Current solutions mainly use hybrid simulation to gain fewer updates and synchronizations, where some important continuous models are maintained implicitly to keep the system dynamics, and partial resynchronization (PR) is chosen as the preferable state update mechanism. However, problems, such as resynchronization interval selection and cyclic dependency, remain unsolved in PR, which easily lead to low update efficiency and infinite looping of the state update process. To address these problems, this paper proposes a lookahead behavior model (LBM) to implement a PR-based hybrid simulation. In LBM, a minimal safe time window is used to predict the interactions between implicit models, upon which the resynchronization interval can be efficiently determined. Moreover, the LBM gives an estimated state value in the lookahead process so as to break the state-dependent cycle. The simulation results show that, compared with traditional mechanisms, LBM requires fewer updates and synchronizations.