Fuzzy or neuro-fuzzy systems have been successfully employed in many areas, but their limitation in solving high-dimensional problems remains a challenging task. On the other hand, fuzzy model based feature selection (FS) approaches have been well studied but, because of the above limitation, are rarely applied to high-dimensional data, which exactly needs feature reduction. In this paper, we design a novel adaptive Ln-Exp softmin operator as the approximator of the minimum T-norm, which is equipped in Takagi-Sugeno-Kang (TSK) fuzzy systems to make them capable of dealing with high-dimensional problems. The adaptive Ln-Exp softmin based TSK (ALETSK) model is developed. Then, we improve the gate function by introducing an enhanced scheme and propose a double groups of gates based ALETSK (DG-ALETSK) fuzzy approach to simultaneously conduct FS and rule extraction (RE), where the gate function is a function measuring the importance of the features or rules and it acts like a gate. More specifically, a group of feature gates are embedded in the antecedents of ALETSK for FS. Meanwhile, another group of rule gates are embedded in the consequents of ALETSK for RE. In one training phase, the gate parameters are trained along with the system parameters, so that the FS and RE are done together. Our proposed DG-ALETSK is effective and time-saving for FS and RE in the high-dimensional tasks, which is verified by the numerical experiments.
Approximation ability is one of the most important topics in the field of neural networks (NNs). Feedforward NNs, activated by rectified linear units and some of their specific smoothed versions, provide universal approximators to convex as well as continuous functions. However, most of these networks are investigated empirically, or their characteristics are analyzed based on specific operation rules. Moreover, an adequate level of interpretability of the networks is missing as well. In this work, we propose a class of new network architecture, built with reusable neural modules (functional blocks), to supply differentiable and interpretable approximators for convex and continuous target functions. Specifically, first, we introduce a concrete model construction mechanism with particular blocks based on differentiable programming and the composition essence of the max operator, extending the scope of existing activation functions. Moreover, explicit block diagrams are provided for a clear understanding of the external architecture and the internal processing mechanism. Subsequently, the approximation behavior of the proposed network to convex functions and continuous functions is rigorously proved as well, by virtue of mathematical induction. Finally, plenty of numerical experiments are conducted on a wide variety of problems, which exhibit the effectiveness and the superiority of the proposed model over some existing ones.