A major limitation of fuzzy or neuro-fuzzy systems is their failure to deal with high-dimensional datasets. This happens primarily due to the use of T-norm, particularly, product or minimum (or a softer version of it). Thus, there are hardly any work dealing with datasets having features more than hundred or so. Here, we propose a neuro-fuzzy framework that can handle datasets with even more than 7000 features! In this context, we propose an adaptive softmin (Ada-softmin) which effectively overcomes the drawbacks of “numeric underflow” and “fake minimum” that arise for existing fuzzy systems while dealing with high-dimensional problems. We call it an Adaptive Takagi-Sugeno-Kang (AdaTSK) fuzzy system. We then equip the AdaTSK system to perform feature selection and rule extraction in an integrated manner. In this context, a novel gate function is introduced and embedded only in the consequent parts, which can determine the useful features and rules, in two successive phases of learning. Unlike conventional fuzzy rule bases, we design an enhanced fuzzy rule base (En-FRB), which maintains adequate rules but does not grow the number of rules exponentially with features that typically happens for fuzzy neural networks. The integrated Feature Selection and Rule Extraction AdaTSK (FSRE-AdaTSK) system consists of three sequential phases: (i) feature selection, (ii) rule extraction, and (iii) fine tuning. The effectiveness of the FSRE-AdaTSK is demonstrated on 19 datasets of which five are in more than 2000 dimension including two with more than 7000 features. This may be the first attempt to develop fuzzy rule based classifiers that can directly deal with more than 7000 features without requiring separate selection of features or any other dimensionality reduction method.

The application and theoretical analysis of fault tolerant learning are very important for neural networks. Our objective here is to realize fault tolerant sparse multilayer perceptron (MLP) networks. The stochastic gradient descent method has been employed to perform online learning for MLPs. For weight noise injection-based network models, it is a common strategy to add a weight decay regularizer while constructing the objective function for learning. However, this l(2)-norm penalty does not generate sparse optimal solutions. In this paper, a group lasso penalty term is used as a regularizer, where a group is defined by the set of weights connected to a node from nodes in the preceding layer. Group lasso penalty enables us to prune redundant hidden nodes. Due to its nondifferentiability at the origin, a smooth approximation of the group lasso penalty is developed. Then, a rigorous proof for the asymptotic convergence of the learning algorithm is provided. Finally, some simulations have been performed to verify the sparseness of the network and the theoretical results.

Multi-Label Classification (MLC) refers to associating each instance with multiple labels simultaneously. MLC has gained much importance due to its ability to better reflect the complexity of the real world classification problems. Fuzzy System (FS) has excellent nonlinear modeling capability and strong interpretability, which makes it a promising model for complex MLC problems. However, it is widely known that FS suffers from the “curse of dimensionality”. Here, an adaptive membership function (MF) along with its generalized version are proposed to address high-dimensional problems. These MFs can effectively overcome “numeric underflow” in FS while preserving interpretability as much as possible.On this basis, a novel fuzzy rule based multi-label classification framework called Multi-Label High-Dimensional Takagi-Sugeno-Kang Fuzzy System (ML-HDTSK FS) is proposed. This model can handle data with over ten thousand dimensionality. Additionally, MLHDTSK FS uses a decomposed label correlation learning strategy to efficiently capture both high and low levels of relationship between labels, and adopts a group L ₂₁ penalty to realize the learning of label-specific features. Combining these two new multi-label learning strategies and the novel adaptive membership function, ML-HDTSK FS becomes a more powerful tool for various MLC problems. The effectiveness of ML-HDTSK FS is demonstrated on seventeen benchmark multi-label data sets, and its performance is compared with eleven MLC algorithms. The experimental results confirm the validity of the proposed MLHDTSK FS, and demonstrate the superiority of it in dealing with MLC problems, especially for high dimensional ones.

The multilayer perceptron (MLP) network has been widely used in various fields. However, the performance of neural networks can be heavily affected by the choice of the parameters, such as the number of hidden nodes. Here we propose a new scheme for finding an “optimal” architecture of a multilayer perceptron network for a given problem. Our philosophy is to associate a tunable gate with every node in the hidden layer. Initially, all gates remain almost closed indicating a scenario where all hidden nodes are practically rejected (i.e., non-existent). Then during training, the gates are opened depending on the ability of the nodes in reducing the objective function. In order to reduce the use of redundant (unnecessary) hidden nodes, we use an L ₁ regularization on the extent of gate opening. Typically, the network begins with a large number of hidden nodes and after training, not-useful hidden nodes are deleted. The network then may further be tuned to adapt itself with the reduced architecture. The performance of the pruning method is tested on several datasets and compared with the Weight Decay and Weight Elimination regularizer methods. The results demonstrate the advantages of the proposed method on both pruning efficiency and generalization.

Purpose: The aim of the present study was to construct a novel long non-coding RNA (lncRNA) signature to predict the prognosis of patients with cervical cancer (CC). Materials and methods: We downloaded lncRNA expression profiles and clinical characteristics from The Cancer Genome Atlas database and randomly divided them into a training dataset (n=200) and a testing dataset (n=87). Using a Cox-based iterative sure independence screening procedure combined with a resampling technique, a lncRNA signature was calculated from prognostic lncRNAs in the training dataset and was independently verified in the testing and the entire datasets. In addition, multivariate Cox regression and further stratified analyses were performed, taking into consideration the lncRNA signature as well as other clinical characteristics. Finally, we predicted the underlying functional effects of the prognostic lncRNAs by using Gene Ontology and Kyoto Encyclopedia of Genes and Genomes pathway enrichment analyses. Results: We constructed a promising ten-lncRNA signature that was significantly associated with the prognosis of CC on the basis of a risk score formula. The risk score was used to classify patients into high-risk and low-risk groups with different overall survival in the training dataset, and was confirmed in the testing and entire datasets. Compared with the clinical factors, the ten-lncRNA signature was found to be an independent prognostic indicator and displayed robust prognostic performance. A functional analysis indicated that these ten lncRNAs were enriched in immune response, cell adhesion molecules and nuclear factor kappa B signaling. Conclusion: Our results demonstrated that this ten-lncRNA signature may serve as a prognostic biomarker for patients with CC.

Fuzzy systems are widely used for their strong ability to handle uncertainty; however, they often struggle with high-dimensional problems. Here, a novel scalable model, the Multihead Takagi-Sugeno-Kang Fuzzy System (MHTSK), is proposed to efficiently handle high-dimensional data while maintaining strong interpretability. Unlike traditional fuzzy systems that construct a single antecedent using all features, MHTSK builds multiple sub-antecedents by randomly sampling subsets of features to address high-dimensional challenges. The rules generated by these sub-antecedents are jointly optimized, resulting in a naturally sparse consequent structure that enhances interpretability. Since the size and number of feature subsets directly influence the length and complexity of fuzzy rules, we propose an optimal parameter configuration scheme that balances: (1) preventing “numeric underflow” in high-dimensional data and (2) ensuring sufficient coverage of the original feature space. Furthermore, considering the randomness introduced by sampling, a rule extraction scheme is proposed to eliminate redundant rules. The refined rule base obtained after the extraction process is used to retrain the model, resulting in a new variant model called MHTSK_RE. The effectiveness of MHTSK and MHTSK_RE is demonstrated on eleven high-dimensional datasets. The results confirm their superior performance and outstanding interpretability in handling high-dimensional data.