This article introduces the deep neural network method into the field of high-dimensional microwave modeling. Deep learning is nowadays highly successful in solving complex and challenging pattern recognition and classification problems. This article investigates the use of deep neural networks to solve microwave modeling problems that are much more challenging than that solved by the previous shallow neural networks. The most commonly used activation function in the existing deep neural network is the rectified linear unit (ReLU), which is a piecewise hard switch function. However, such a ReLU is not suitable for microwave modeling where the input-output relationships are smooth and continuous. In this article, we propose a new deep neural network to perform high-dimensional microwave modeling. A smooth ReLU is proposed for the new deep neural network. The proposed deep neural network employs both the sigmoid function and the smooth ReLU as activation functions. The new deep neural network can represent the smooth input-output relationship that is required for microwave modeling. An advanced three-stage deep learning algorithm is proposed to train the new deep neural network model. This algorithm can determine the number of hidden layers with sigmoid functions and those with smooth ReLUs in the training process. It can also overcome the vanishing gradient problem for training the deep neural network. The proposed deep neural network technique can solve microwave modeling problems in a higher dimension than the previous neural network method, i.e., shallow neural network method. Two high-dimensional parameter-extraction modeling examples of microwave filters are presented to demonstrate the proposed deep neural network technique.

This article presents an overview of artificial neural network (ANN) techniques for a microwave computer-aided design (CAD). ANN-based techniques are becoming useful for performing forward/inverse modeling for active/passive components to enhance a circuit design. With measured or simulated data of microwave devices, ANNs can be trained to learn relevant microwave relationships, which are, otherwise, computationally expensive or for which efficient analytical formulas are not available. Fundamental concepts of the ANN structure and training, such as feedforward neural networks (FFNNs), recurrent neural networks (RNNs)/dynamic neural networks (DNNs)/time-delay neural networks (TDNNs), deep neural networks, and neural network training and extrapolation, are described. Knowledge-based neural networks (KBNNs) are described for improving the accuracy and reliability of modeling and design optimization. Various advanced ANN techniques, such as neuro-transfer function (neuro-TF) modeling, neural network inverse modeling, and deep neural network modeling, are discussed. The existing and emerging applications of ANN in microwave CAD are identified, such as electromagnetic (EM)/multiphysics modeling, modeling of nonlinear circuits and transistors, filter design, very large-scale integration (VLSI) interconnects, oscillator, transmitter and receiver modeling, and CAD applications in such as gallium nitride (GaN) high electron-mobility transistor (HEMT), wireless power transfer (WPT), microelectromechanical system (MEMS), and substrate-integrated waveguide (SIW).

This paper presents a new technique for artificial neural network (ANN) inverse modeling and applications to microwave filters. In inverse modeling of a microwave component, the inputs to the model are electrical parameters such as S-parameters, and the outputs of the model are geometrical or physical parameters. Since the analytical formula of the inverse input-output relationship does not exist, the ANN becomes a logical choice, because it can be trained to learn from the data in inverse modeling. The main challenge of inverse modeling is the nonuniqueness problem. This problem in the ANN inverse modeling is that different training samples with the same or very similar input values have quite different (contradictory) output values (multivalued solutions). In this paper, we propose a multivalued neural network inverse modeling technique to associate a single set of electrical parameters with multiple sets of geometrical or physical parameters. One set of geometrical or physical parameters is called one value of our proposed inverse model. Our proposed multivalued neural network is structured to accommodate multiple values for the model output. We also propose a new training error function to focus on matching each training sample using only one value of our proposed inverse model, while other values are free and can be trained to match other contradictory samples. In this way, our proposed multivalued neural network can learn all the training data by automatically redirecting contradictory information into different values of the proposed inverse model. Therefore, our proposed technique can solve the nonuniqueness problem in a simpler and more automated way compared with the existing ANN inverse modeling techniques. This technique is illustrated by inverse modeling and parameter extraction of four microwave filter examples.

Feature-based electromagnetic (EM) optimization techniques can help avoid local minima in microwave design. Zeros of the transfer functions are recently used to help extract the features when the features of filter responses are not explicitly identifiable. This letter proposes a feature zero-adaptation approach to enlarge the surrogate range by overcoming the problem of varying orders of the transfer function w.r.t. the changes in design variables. In this way, the proposed technique allows larger step sizes for optimization, therefore, speeding up the overall EM optimization process. During each optimization iteration, parallel techniques are proposed to be used to generate multiple EM geometrical samples simultaneously for creating the feature-based surrogate model. The proposed technique is demonstrated using two microwave filter examples.

This article proposes a novel parallel gradient-based electromagnetic (EM) optimization approach to microwave components using adjoint-sensitivity-based neuro-transfer function (neuro-TF) surrogate. In the proposed technique, the surrogate model is trained using not only the input-output behavior but also the adjoint sensitivity information generated from the EM simulation simultaneously. By exploiting adjoint EM sensitivity for surrogate modeling, the proposed technique can obtain accurate surrogate models with larger valid range using the same amount of fine model evaluations compared with the existing gradient-based surrogate optimization without adjoint sensitivity. Furthermore, because the surrogate model is developed using adjoint EM sensitivity, the gradients calculated using the developed surrogate model in the proposed technique are much more accurate. The accurate gradients lead to further speedup of the surrogate optimization and improved quality of surrogate optimal solution in each surrogate optimization iteration. Since the surrogate model is valid in a large neighborhood and the gradients are sufficiently accurate, the proposed technique can achieve the optimal EM solution faster than the existing gradient-based surrogate optimization without adjoint sensitivity. Three examples of EM optimizations of microwave components are used to demonstrate the proposed technique.