Background: While random forests are one of the most successful machine learning methods, it is necessary to optimize their performance for use with datasets resulting from a two-phase sampling design with a small number of cases—a common situation in biomedical studies, which often have rare outcomes and covariates whose measurement is resource-intensive.Methods: Using an immunologic marker dataset from a phase III HIV vaccine efficacy trial, we seek to optimize random forest prediction performance using combinations of variable screening, class balancing, weighting, and hyperparameter tuning.Results: Our experiments show that while class balancing helps improve random forest prediction performance when variable screening is not applied, class balancing has a negative impact on performance in the presence of variable screening. The impact of the weighting similarly depends on whether variable screening is applied. Hyperparameter tuning is ineffective in situations with small sample sizes. We further show that random forests under-perform generalized linear models for some subsets of markers, and prediction performance on this dataset can be improved by stacking random forests and generalized linear models trained on different subsets of predictors, and that the extent of improvement depends critically on the dissimilarities between candidate learner predictions.Conclusion: In small datasets from two-phase sampling design, variable screening and inverse sampling probability weighting are important for achieving good prediction performance of random forests. In addition, stacking random forests and simple linear models can offer improvements over random forests.

One of the most important hyper-parameters in the Random Forest (RF) algorithm is the feature set size used to search for the best partitioning rule at each node of trees. Most existing research on feature set size has been done primarily with a focus on classification problems. We studied the effect of feature set size in the context of regression. Through experimental studies using many datasets, we first investigated whether the RF regression predictions are affected by the feature set size. Then, we found a rule associated with the optimal size based on the characteristics of each data. Lastly, we developed a search algorithm for estimating the best feature set size in RF regression. We showed that the proposed search algorithm can provide improvements over other choices, such as using the default size specified in the randomForest R package and using the common grid search method.

Random forest is an ensemble method that combines many decision trees. Each level of trees is determined by an optimal rule among a candidate feature set. The candidate feature set is a random subset of all features, and is different at each level of trees. In this article, we investigated whether the accuracy of Random forest is affected by the size of the candidate feature set. We found that the optimal size differs from data to data without any specific pattern. To estimate the optimal size of feature set, we proposed a novel algorithm which uses the out-of-bag error and the ‘SearchSize’ exploration. The proposed method is significantly faster than the standard grid search method while giving almost the same accuracy. Finally, we demonstrated that the accuracy of Random forest using the proposed algorithm has increased significantly compared to using a typical size of feature set.

Purpose: Significant progress has occurred in developing quantitative PET/CT biomarkers in diffuse large B-cell lymphoma (DLBCL). Total metabolic tumor volume (MTV) is the most extensively studied, enabling assessment of FDG-avid tumor burden associated with outcomes. However, prior studies evaluated the outcome of cytotoxic chemotherapy or chimeric antigen receptor T-cell therapy without data on recently approved FDA agents. Therefore, we aimed to assess the prognosis of PET/CT biomarkers in patients treated with loncastuximab tesirine. Experimental Design: We centrally reviewed screening PET/CT scans of patients with relapsed/refractory DLBCL enrolled in the LOTIS-2 (NCT03589469) study. MTV was obtained by computing individual volumes using the SUV ≥4.0 threshold. Other PET/CT metrics, clinical factors, and the International Metabolic Prognostic Index (IMPI) were evaluated. Logistic regression was used to assess the association between biomarkers and treatment response. Cox regression was used to determine the effect of biomarkers on time-to-event outcomes. We estimated biomarker prediction as continuous and binary variables defined by cutoff points. Results: Across 138 patients included in this study, MTV with a cutoff point of 96 mL was the biomarker associated with the highest predictive performance in univariable and multivariable models to predict failure to achieve complete metabolic response (OR, 5.42; P = 0.002), progression-free survival (HR, 2.68; P = 0.002), and overall survival (HR, 3.09; P < 0.0001). IMPI demonstrated an appropriate performance, however, not better than MTV alone. Conclusions: Pretreatment MTV demonstrated robust risk stratification, with those patients demonstrating high MTV achieving lower responses and survival to loncastuximab tesirine in relapsed/refractory DLBCL.