The rapid evolution of machine learning has led to a proliferation of sophisticated models for predicting therapeutic responses in cancer. While many of these show promise in research, standards for clinical evaluation and adoption are lacking. Here, we propose seven hallmarks by which predictive oncology models can be assessed and compared. These are Data Relevance and Actionability, Expressive Architecture, Standardized Benchmarking, Generalizability, Interpretability, Accessibility and Reproducibility, and Fairness. Considerations for each hallmark are discussed along with an example model scorecard. We encourage the broader community, including researchers, clinicians, and regulators, to engage in shaping these guidelines toward a concise set of standards. Significance: As the field of artificial intelligence evolves rapidly, these hallmarks are intended to capture fundamental, complementary concepts necessary for the progress and timely adoption of predictive modeling in precision oncology. Through these hallmarks, we hope to establish standards and guidelines that enable the symbiotic development of artificial intelligence and precision oncology.

It is well-known that cancers of the same histology type can respond differently to a treatment. Thus, computational drug response prediction is of paramount importance for both preclinical drug screening studies and clinical treatment design. To build drug response prediction models, treatment response data need to be generated through screening experiments and used as input to train the prediction models. In this study, we investigate various active learning strategies of selecting experiments to generate response data for the purposes of (1) improving the performance of drug response prediction models built on the data and (2) identifying effective treatments. Here, we focus on constructing drug-specific response prediction models for cancer cell lines. Various approaches have been designed and applied to select cell lines for screening, including a random, greedy, uncertainty, diversity, combination of greedy and uncertainty, sampling-based hybrid, and iteration-based hybrid approach. All of these approaches are evaluated and compared using two criteria: (1) the number of identified hits that are selected experiments validated to be responsive, and (2) the performance of the response prediction model trained on the data of selected experiments. The analysis was conducted for 57 drugs and the results show a significant improvement on identifying hits using active learning approaches compared with the random and greedy sampling method. Active learning approaches also show an improvement on response prediction performance for some of the drugs and analysis runs compared with the greedy sampling method.

Deep learning and machine learning models have shown promise in drug response prediction (DRP), yet their ability to generalize across datasets remains an open question, raising concerns about their real-world applicability. Due to the lack of standardized benchmarking approaches, model evaluations and comparisons often rely on inconsistent datasets and evaluation criteria, making it difficult to assess true predictive capabilities. In this work, we introduce a benchmarking framework for evaluating cross-dataset prediction generalization in DRP models. Our framework incorporates five publicly available drug screening datasets, seven standardized DRP models, and a scalable workflow for systematic evaluation. To assess model generalization, we introduce a set of evaluation metrics that quantify both absolute performance (e.g. predictive accuracy across datasets) and relative performance (e.g. performance drop compared to within-dataset results), enabling a more comprehensive assessment of model transferability. Our results reveal substantial performance drops when models are tested on unseen datasets, underscoring the importance of rigorous generalization assessments. While several models demonstrate relatively strong cross-dataset generalization, no single model consistently outperforms across all datasets. Furthermore, we identify CTRPv2 as the most effective source dataset for training, yielding higher generalization scores across target datasets. By sharing this standardized evaluation framework with the community, our study aims to establish a rigorous foundation for model comparison, and accelerate the development of robust DRP models for real-world applications.

Cancer is a heterogeneous disease in that tumors of the same histology type can respond differently to a treatment. Anti-cancer drug response prediction is of paramount importance for both drug development and patient treatment design. Although various computational methods and data have been used to develop drug response prediction models, it remains a challenging problem due to the complexities of cancer mechanisms and cancer-drug interactions. To better characterize the interaction between cancer and drugs, we investigate the feasibility of integrating computationally derived features of molecular mechanisms of action into prediction models. Specifically, we add docking scores of drug molecules and target proteins in combination with cancer gene expressions and molecular drug descriptors for building response models. The results demonstrate a marginal improvement in drug response prediction performance when adding docking scores as additional features, through tests on large drug screening data. We discuss the limitations of the current approach and provide the research community with a baseline dataset of the large-scale computational docking for anti-cancer drugs.

Drug response prediction (DRP) methods tackle the complex task of associating the effectiveness of small molecules with the specific genetic makeup of the patient. Anti-cancer DRP is a particularly challenging task requiring costly experiments as underlying pathogenic mechanisms are broad and associated with multiple genomic pathways. The scientific community has exerted significant efforts to generate public drug screening datasets, giving a path to various machine learning models that attempt to reason over complex data space of small compounds and biological characteristics of tumors. However, the data depth is still lacking compared to application domains like computer vision or natural language processing domains, limiting current learning capabilities. To combat this issue and improves the generalizability of the DRP models, we are exploring strategies that explicitly address the imbalance in the DRP datasets. We reframe the problem as a multi-objective optimization across multiple drugs to maximize deep learning model performance. We implement this approach by constructing Multi-Objective Optimization Regularized by Loss Entropy loss function and plugging it into a Deep Learning model. We demonstrate the utility of proposed drug discovery methods and make suggestions for further potential application of the work to achieve desirable outcomes in the healthcare field.

This work studies a particular setting for regression problems – tasks with complex combinatorial data space where samples can be divided into distinct groups. Anti-cancer drug response prediction is a perfect example of this setting, in which each sample includes cancer biological features and drug chemical information. Many existing works of pan-drug and pan-cancer response modeling treat different combinations of drugs and cancers as individual samples. A potential problem in these works is that a model may be heavily influenced and biased toward overrepresented drugs and cancers. Our work develops a method to solve this issue by adjusting the model training process in a deep learning framework. In the drug response prediction field, the performance of pan-cancer pan-drug models is commonly evaluated on a holdout test set through cross-validation (CV) using performance metrics like the coefficient of determination (R2) and the mean squared error (MSE). However, drug response prediction can be viewed as a multi-objective optimization task, attempting to maximize the prediction performance over different drugs and cancers. We consider the performance for each drug as a separate objective and attempt to find a model on the corresponding Pareto front that provides balanced performances for all compounds. We propose adding an entropy-based regularization to the loss function for model training to reach this balanced state. The intuition behind it is straightforward – we minimize the MSE on all data points while encouraging the drug-specific model fitting error variability to stay as low as possible. We achieve this by grouping samples by their drug identity and computing the MSE for each group in the training batch. Then we calculate entropy over normalized group-specific losses. This value is plugged into the regularization term that incentivizes the loss function to maximize it. The maximum entropy can be achieved when the MSEs across all drugs take the same value. We investigate the regularization effect on response modeling using a drug screening dataset of Cancer Cell Line Encyclopedia (CCLE) and a state-of-the-art drug response prediction model DeepTTA [1]. We consider two CV strategies for model evaluation – random split and drug-blind split. In a random split, the testing set can share both cell lines and drugs with the training set, while in a drug-blind split, a drug can not appear simultaneously in both the training and testing sets. We perform 10-fold CV analyses and evaluate the model performance using R2. For the random split, we see that adding the entropy-based regularization leads to a marginal improvement in prediction performance, which is 0.736 without regularization versus 0.746 with regularization and a p-value of 0.130 from the pairwise t-test. Importantly, we observe a substantial improvement in the more challenging setting of drug-blind split, where the prediction performance increases from -0.128 (without regularization) to 0.168 (with regularization) with a statistically significant p-value of 0.005 from pair-wise t-test.