Since sparse neural networks usually contain many zero weights, these unnecessary network connections can potentially be eliminated without degrading network performance. Therefore, well-designed sparse neural networks have the potential to significantly reduce the number of floating-point operations (FLOPs) and computational resources. In this work, we propose a new automatic pruning method--sparse connectivity learning (SCL). Specifically, a weight is reparameterized as an elementwise multiplication of a trainable weight variable and a binary mask. Thus, network connectivity is fully described by the binary mask, which is modulated by a unit step function. We theoretically prove the fundamental principle of using a straight-through estimator (STE) for network pruning. This principle is that the proxy gradients of STE should be positive, ensuring that mask variables converge at their minima. After finding Leaky ReLU, Softplus, and identity STEs can satisfy this principle, we propose to adopt identity STE in SCL for discrete mask relaxation. We find that mask gradients of different features are very unbalanced; hence, we propose to normalize mask gradients of each feature to optimize mask variable training. In order to automatically train sparse masks, we include the total number of network connections as a regularization term in our objective function. As SCL does not require pruning criteria or hyperparameters defined by designers for network layers, the network is explored in a larger hypothesis space to achieve optimized sparse connectivity for the best performance. SCL overcomes the limitations of existing automatic pruning methods. Experimental results demonstrate that SCL can automatically learn and select important network connections for various baseline network structures. Deep learning models trained by SCL outperform the state-of-the-art human-designed and automatic pruning methods in sparsity, accuracy, and FLOPs reduction.

Purpose: COVID-19 has become global threaten. CT acts as an important method of diagnosis. However, human–based interpretation of CT imaging is time consuming. More than that, substantial inter-observer-variation cannot be ignored. We aim at developing a diagnostic tool for artificial intelligence (AI)–based classification of CT images for recognizing COVID-19 and other common infectious diseases of the lung. Experimental Design: In this study, images were retrospectively collected and prospectively analyzed using machine learning. CT scan images of the lung that show or do not show COVID-19 were used to train and validate a classification framework based on convolutional neural network. Five conditions including COVID-19 pneumonia, non-COVID-19 viral pneumonia, bacterial pneumonia, pulmonary tuberculosis, and normal lung were evaluated. Training and validation set of images were collected from Wuhan Jin Yin-Tan Hospital whereas test set of images were collected from Zhongshan Hospital Xiamen University and the fifth Hospital of Wuhan. Results: Accuracy, sensitivity, and specificity of the AI framework were reported. For test dataset, accuracies for recognizing normal lung, COVID-19 pneumonia, non-COVID-19 viral pneumonia, bacterial pneumonia, and pulmonary tuberculosis were 99.4%, 98.8%, 98.5%, 98.3%, and 98.6%, respectively. For the test dataset, accuracy, sensitivity, specificity, PPV, and NPV of recognizing COVID-19 were 98.8%, 98.2%, 98.9%, 94.5%, and 99.7%, respectively. Conclusions: The performance of the proposed AI framework has excellent performance of recognizing COVID-19 and other common infectious diseases of the lung, which also has balanced sensitivity and specificity.

The gene expression data are usually provided with a large number of genes and a relatively small number of samples, which brings a lot of new challenges. Selecting those informative genes becomes the main issue in microarray data analysis. Recursive cluster elimination based on support vector machine (SVM-RCE) has shown the better classification accuracy on some microarray data sets than recursive feature elimination based on support vector machine (SVM-RFE). However, SVM-RCE is extremely time-consuming. In this paper, we propose an improved method of SVM-RCE called ISVM-RCE. ISVM-RCE first trains a SVM model with all clusters, then applies the infinite norm of weight coefficient vector in each cluster to score the cluster, finally eliminates the gene clusters with the lowest score. In addition, ISVM-RCE eliminates genes within the clusters instead of removing a cluster of genes when the number of clusters is small. We have tested ISVM-RCE on six gene expression data sets and compared their performances with SVM-RCE and linear-discriminant-analysis-based RFE (LDA-RFE). The experiment results on these data sets show that ISVM-RCE greatly reduces the time cost of SVM-RCE, meanwhile obtains comparable classification performance as SVM-RCE, while LDA-RFE is not stable.