Nowadays, in the topics related to prediction, in addition to increasing the accuracy of existing algorithms, the reduction of computational time is a challenging issue that has attracted much attention. Since the existing methods may not have enough efficiency and accuracy, we use a combination of machine-learning algorithms and statistical methods to solve this problem. Furthermore, we reduce the computational time in the testing model by automatically reducing the number of trees using penalized methods and ensembling the remaining trees. We call this efficient combinatorial method “ensemble of clustered and penalized random forest (ECAPRAF)”. This method consists of four fundamental parts. In the first part, k-means clustering is used to identify homogeneous subsets of data and assign them to similar groups. In the second part, a tree-based algorithm is used within each cluster as a predictor model; in this work, random forest is selected. In the next part, penalized methods are used to reduce the number of random-forest trees and remove high-variance trees from the proposed model. This increases model accuracy and decreases the computational time in the test phase. In the last part, the remaining trees within each cluster are combined. The results of the simulation and two real datasets based on the WRMSE criterion show that our proposed method has better performance than the traditional random forest by reducing approximately 12.75%, 11.82%, 12.93%, and 11.68% and selecting 99, 106, 113, and 118 trees for the ECAPRAF–EN algorithm.
Deep learning techniques face the problem of overfitting due to their complex layer structure. Regularization methods are used to overcome this problem and improve the designed models. In this article, we use the combination of L1 regularization, L2 regularization, Elastic Net-regularization, and Dropout methods. The designed deep model using combination of these methods is considered with different rates. The deep network model using a combination of these methods is designed with different rates. Finally, the performance of all combination methods is compared with the Convolutional Neural Network model which does not use regularization methods. Experiments are performed using the Gold price per ounce data set and linear simulation model. The obtained results show that the performance of the combination model of Dropout and Elastic Net regularization is better than the other models.
1
1 December 2018 in Journal of Computational and Applied Mathematics
Recently, the beta regression model has been used in several fields of science to model data in the form of rate or proportion. In this paper, some novel and improved methods to estimate parameters in the beta regression model are proposed. We consider a sub-space on the regression coefficients of the beta regression model and combine the unrestricted and restricted estimators then we present Stein-type and preliminary estimators. We develop the expressions for the proposed estimators' asymptotic biases and their quadratic risks. Numerical studies through Monte Carlo simulations are used to evaluate the performance of the proposed estimators in terms of their simulated relative efficiency. The results show that the proposed estimators outperform the unrestricted estimator when the restrictions hold. Finally, an empirical application is given to show how useful the proposed estimators are in the practical area.
6 September 2016 in Methodology and Computing in Applied Probability
This article is improved the random forest algorithm by selecting the most appropriate penalized regression methods, and it is tried to improve the post-selection boosting random forest (PBRF) algorithm using elastic net regression. The proposed method with the highest efficiency is called Reducing and Aggregating Random Forest Trees by Elastic Net (RARTEN). The introduced method consists of three steps. In the first step, the random forest algorithm is used as a predictor. In the second step, Elastic Net, as a penalized regression method, is applied to reduce the number of trees and improve the random forest and PBRF. In the last step, selected trees are aggregated. The obtained results of the real data and Monte Carlo simulation are evaluated using various statistical performance criteria. The simulation study shows that the RARTEN with 7%, 5%, and 8.5% reduction in the linear, nonlinear, and noise model, respectively improve the accuracy of the traditional random forest and the proposed method by Wang. In addition, this method has a significant reduction compared to other penalized regression methods. Moreover, the real data results show that the proposed method in our study with a reduction of almost 16% confirms the validity of the proposed model.