We investigate high-dimensional nonconvex penalized regression, where the number of covariates may grow at an exponential rate. Although recent asymptotic theory established that there exists a local minimum possessing the oracle property under general conditions, it is still largely an open problem how to identify the oracle estimator among potentially multiple local minima. There are two main obstacles: (1) due to the presence of multiple minima, the solution path is nonunique and is not guaranteed to contain the oracle estimator; (2) even if a solution path is known to contain the oracle estimator, the optimal tuning parameter depends on many unknown factors and is hard to estimate. To address these two challenging issues, we first prove that an easy-to-calculate calibrated CCCP algorithm produces a consistent solution path which contains the oracle estimator with probability approaching one. Furthermore, we propose a high-dimensional BIC criterion and show that it can be applied to the solution path to select the optimal tuning parameter which asymptotically identifies the oracle estimator. The theory for a general class of nonconvex penalties in the ultra-high dimensional setup is established when the random errors follow the sub-Gaussian distribution. Monte Carlo studies confirm that the calibrated CCCP algorithm combined with the proposed high-dimensional BIC has desirable performance in identifying the underlying sparsity pattern for high-dimensional data analysis.

We derive the fast convergence rates of a deep neural network (DNN) classifier with the rectified linear unit (ReLU) activation function learned using the hinge loss. We consider three cases for a true model: (1) a smooth decision boundary, (2) smooth conditional class probability, and (3) the margin condition (i.e., the probability of inputs near the decision boundary is small). We show that the DNN classifier learned using the hinge loss achieves fast rate convergences for all three cases provided that the architecture (i.e., the number of layers, number of nodes and sparsity) is carefully selected. An important implication is that DNN architectures are very flexible for use in various cases without much modification. In addition, we consider a DNN classifier learned by minimizing the cross-entropy, and show that the DNN classifier achieves a fast convergence rate under the conditions that the noise exponent and margin exponent are large. Even though they are strong, we explain that these two conditions are not too absurd for image classification problems. To confirm our theoretical explanation, we present the results of a small numerical study conducted to compare the hinge loss and cross-entropy.

There has been a growing interest in expressivity of deep neural networks. However, most of the existing work about this topic focuses only on the specific activation function such as ReLU or sigmoid. In this paper, we investigate the approximation ability of deep neural networks with a broad class of activation functions. This class of activation functions includes most of frequently used activation functions. We derive the required depth, width and sparsity of a deep neural network to approximate any Hölder smooth function upto a given approximation error for the large class of activation functions. Based on our approximation error analysis, we derive the minimax optimality of the deep neural network estimators with the general activation functions in both regression and classification problems.

Park JY, et al. J Korean Med Sci. 2020 Feb;35(11):e124. https://doi.org/10.3346/jkms.2020.35.e124

The aim of this study was to compare the performance of image analysis for predicting breast cancer using two distinct regression models and to evaluate the usefulness of incorporating clinical and demographic data (CDD) into the image analysis in order to improve the diagnosis of breast cancer. This study included 139 solid masses from 139 patients who underwent a ultrasonography-guided core biopsy and had available CDD between June 2009 and April 2010. Three breast radiologists retrospectively reviewed 139 breast masses and described each lesion using the Breast Imaging Reporting and Data System (BI-RADS) lexicon. We applied and compared two regression methods-stepwise logistic (SL) regression and logistic least absolute shrinkage and selection operator (LASSO) regression-in which the BI-RADS descriptors and CDD were used as covariates. We investigated the performances of these regression methods and the agreement of radiologists in terms of test misclassification error and the area under the curve (AUC) of the tests. Logistic LASSO regression was superior (P<0.05) to SL regression, regardless of whether CDD was included in the covariates, in terms of test misclassification errors (0.234 vs. 0.253, without CDD; 0.196 vs. 0.258, with CDD) and AUC (0.785 vs. 0.759, without CDD; 0.873 vs. 0.735, with CDD). However, it was inferior (P<0.05) to the agreement of three radiologists in terms of test misclassification errors (0.234 vs. 0.168, without CDD; 0.196 vs. 0.088, with CDD) and the AUC without CDD (0.785 vs. 0.844, P<0.001), but was comparable to the AUC with CDD (0.873 vs. 0.880, P=0.141). Logistic LASSO regression based on BI-RADS descriptors and CDD showed better performance than SL in predicting the presence of breast cancer. The use of CDD as a supplement to the BI-RADS descriptors significantly improved the prediction of breast cancer using logistic LASSO regression.

Background: The Omicron variant (B.1.1.529) is estimated to be more transmissible than previous strains of SARS-CoV-2 especially among children, potentially resulting in croup which is a characteristic disease in children. Current coronavirus disease 2019 (COVID-19) cases among children might be higher because (i) school-aged children have higher contact rates and (ii) the COVID-19 vaccination strategy prioritizes the elderly in most countries. However, there have been no reports confirming the age-varying susceptibility to the Omicron variant to date. Methods: We developed an age-structured compartmental model, combining age-specific contact matrix in South Korea and observed distribution of periods between each stage of infection in the national epidemiological investigation. A Bayesian inference method was used to estimate the age-specific force of infection and, accordingly, age-specific susceptibility, given epidemic data during the third (pre-Delta), fourth (Delta driven), and fifth (Omicron driven) waves in South Korea. As vaccine uptake increased, individuals who were vaccinated were excluded from the susceptible population in accordance with vaccine effectiveness against the Delta and Omicron variants, respectively. Results: A significant difference between the age-specific susceptibility to the Omicron and that to the pre-Omicron variants was found in the younger age group. The rise in susceptibility to the Omicron/pre-Delta variant was highest in the 10–15 years age group (5.28 times [95% CI, 4.94–5.60]), and the rise in susceptibility to the Omicron/Delta variant was highest in the 15–19 years age group (3.21 times [95% CI, 3.12–3.31]), whereas in those aged 50 years or more, the susceptibility to the Omicron/pre-Omicron remained stable at approximately twofold. Conclusions: Even after adjusting for contact pattern, vaccination status, and waning of vaccine effectiveness, the Omicron variant of SARS-CoV-2 tends to propagate more easily among children than the pre-Omicron strains.