The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

Many kernel classifier construction algorithms adopt classification accuracy as performance metrics in model evaluation. Moreover, equal weighting is often applied to each data sample in parameter estimation. These modeling practices often become problematic if the data sets are imbalanced. We present a kernel classifier construction algorithm using orthogonal forward selection (OFS) in order to optimize the model generalization for imbalanced two-class data sets. This kernel classifier identification algorithm is based on a new regularized orthogonal weighted least squares (ROWLS) estimator and the model selection criterion of maximal leave-one-out area under curve (LOO-AUC) of the receiver operating characteristics (ROCs). It is shown that, owing to the orthogonalization procedure, the LOO-AUC can be calculated via an analytic formula based on the new regularized orthogonal weighted least squares parameter estimator, without actually splitting the estimation data set. The proposed algorithm can achieve minimal computational expense via a set of forward recursive updating formula in searching model terms with maximal incremental LOO-AUC value. Numerical examples are used to demonstrate the efficacy of the algorithm

This paper introduces a novel sparse dynamic inner principal component analysis (SDiPCA) based monitoring for multimode dynamic processes. Different from traditional multimode monitoring algorithms, a model is updated for sequential modes by memorizing the significant features of existing modes. By adopting the concept of intelligent synapses in continual learning, a loss of quadratic term is introduced to penalize the changes of mode–relevant parameters, where modified synaptic intelligence (MSI) is proposed to estimate the parameter importance. Thus, the proposed algorithm is referred to as SDiPCA–MSI. When a new mode arrives, a set of normal samples should be collected. The previous significant features are consolidated without explicitly storing training samples, while extracting new information from the current mode. Consequently, SDiPCA–MSI can provide outstanding performance for successive modes. Characteristics of the proposed approach are discussed, including the computational complexity, advantages and potential limitations. Compared with several state-of-the-art monitoring methods, the effectiveness and superiorities of the proposed method are demonstrated by a continuous stirred tank heater case and a practical industrial system. Note to Practitioners—Multimode process monitoring is increasingly significant as industrial systems generally operate in varying operating conditions. However, most researches focus on multiple local monitoring models for complex multimode processes and assume that data of all possible modes are available and stored before learning. When similar or new modes arrive, local models are rebuilt corresponding to each mode and the model’s capacity would increase with the continuous emergence of modes. Adaptive methods are a branch of multimode monitoring algorithms, but they strive to extract information of the current mode to ensure the monitoring performance while forgetting the previously learned knowledge gradually. This paper proposes a novel sparse dynamic inner principal component analysis with continual learning ability for multimode dynamic process monitoring, where modified synaptic intelligence is developed to measure the parameter importance accurately. It requires limited computation and storage resources for successive modes, which is convenient for practical applications. Similar to current multimode process monitoring algorithms, a set of data should be collected before learning a new mode, which may bring difficulties to real–time monitoring. For industrial systems, such as large–scale power plants and chemical systems, the proposed method has outstanding ability to monitor successive dynamic modes.

A novel continual learning-based probabilistic slow feature analysis algorithm is introduced for monitoring multimode nonstationary processes. Multimode slow features are extracted and an elastic weight consolidation (EWC) is adopted for sequential modes. EWC was originally introduced in the setting of machine learning of sequential multi-tasks with the aim of avoiding catastrophic forgetting issue, which equally poses as a major challenge in multimode nonstationary process monitoring. When a new mode arrives, a small set of data are collected for continual learning by the proposed algorithm. A regularization term is introduced to prevent new data from significantly interfering with the learned knowledge, where the parameter importance measures are estimated. The proposed method is referred to as PSFA–EWC, which is updated continually and is capable of achieving excellent performance. PSFA–EWC furnishes backward and forward transfer ability by a single model. The significant features of previous modes are retained while consolidating new information, which may contribute to learning new relevant modes. The effectiveness of the proposed method is demonstrated via a continuous stirred tank heater and a practical coal pulverizing system. Note to Practitioners —Since industrial systems operate in varying modes and data are nonstationary within each mode, multimode nonstationary process monitoring is increasingly important. Traditional multimode monitoring methods generally need complete data from all possible modes and may need to be retrained from scratch when a new mode arrives, which require expensive computation and storage resources. Besides, it is difficult to distinguish real faults from normal variations in multimode nonstationary processes. This paper proposes a novel continual learning-based probabilistic slow feature analysis, where elastic weight consolidation is employed to consolidate the previously learned knowledge while extracting multimode slow features. The monitoring model is updated sequentially and provides backward as well as forward transfer learning ability for successive modes. It is able to separate real faults from normal dynamics, which is beneficial to identifying a new mode for multimode nonstationary processes. In addition, the proposed approach delivers excellent model interpretability and deals with missing data as well as uncertainty. In industrial applications, such as power plants and intelligent manufacturing processes, the proposed method can provide excellent monitoring performance.

In this brief, a new complex-valued B-spline neural network is introduced in order to model the complex-valued Wiener system using observational input/output data. The complex-valued nonlinear static function in the Wiener system is represented using the tensor product from two univariate B-spline neural networks, using the real and imaginary parts of the system input. Following the use of a simple least squares parameter initialization scheme, the Gauss-Newton algorithm is applied for the parameter estimation, which incorporates the De Boor algorithm, including both the B-spline curve and the first-order derivatives recursion. Numerical examples, including a nonlinear high-power amplifier model in communication systems, are used to demonstrate the efficacy of the proposed approaches.

Familial hypophosphatemic rickets (HR), the most common inherited form of rickets, is a group of inherited renal phosphate wasting disorders characterized by growth retardation, rickets with bone deformities, osteomalacia, poor dental development, and hypophosphatemia. The purpose of this study was to identify the genetic defect responsible for familial HR in a four-generation Chinese Han pedigree by exome sequencing and Sanger sequencing. Clinical features include skeletal deformities, teeth abnormalities, hearing impairments and variable serum phosphate level in patients of this family. A novel deletion mutation, c.1553delT (p.F518Sfs*4), was identified in the X-linked phosphate regulating endopeptidase homolog gene (PHEX). The mutation is predicted to result in prematurely truncated and loss-of-function PHEX protein. Our data suggest that exome sequencing is a powerful tool to discover mutation(s) in HR, a disorder with genetic and clinical heterogeneity. The findings may also provide new insights into the cause and diagnosis of HR, and have implications for genetic counseling and clinical management.