Free energy calculations are rapidly becoming indispensable in structure-enabled drug discovery programs. As new methods, force fields, and implementations are developed, assessing their accuracy on real-world systems (benchmarking) becomes critical to provide users with an assessment of the accuracy expected when these methods are applied within their domain of applicability, and developers with a way to assess the expected impact of new methodologies. These assessments require construction of a benchmark—a set of well-prepared, high quality systems with corresponding experimental measurements designed to ensure the resulting calculations provide a realistic assessment of expected performance. To date, the community has not yet adopted a common standardized benchmark, and existing benchmark reports suffer from a myriad of issues, including poor data quality, limited statistical power, and statistically deficient analyses, all of which can conspire to produce benchmarks that are poorly predictive of real-world performance. Here, we address these issues by presenting guidelines for (1) curating experimental data to develop meaningful benchmark sets, (2) preparing benchmark inputs according to best practices to facilitate widespread adoption, and (3) analysis of the resulting predictions to enable statistically meaningful comparisons among methods and force fields. We highlight challenges and open questions that remain to be solved in these areas, as well as recommendations for the collection of new datasets that might optimally serve to measure progress as methods become systematically more reliable. Finally, we provide a curated, versioned, open, standardized benchmark set adherent to these standards (protein-ligand-benchmark) and an open source toolkit for implementing standardized best practices assessments (arsenic) for the community to use as a standardized assessment tool. While our main focus is benchmarking free energy methods based on molecular simulations, these guidelines should prove useful for assessment of the rapidly growing field of machine learning methods for affinity prediction as well.
5 March 2019 in Journal of the American Chemical Society
We consider the influence of population size on the accuracy of diffusion quantum Monte Carlo simulations that employ descendant weighting or forward walking techniques to compute expectation values of observables that do not commute with the Hamiltonian. We show that for a simple model system, the -dimensional isotropic harmonic oscillator, the population size must increase rapidly with in order to ensure that the simulations produce accurate results. When the population size is too small, expectation values computed using descendant-weighted diffusion quantum Monte Carlo simulations exhibit significant systematic biases.
27 March 2020 in Journal of Chemical Theory and Computation
Toxicological profiling is a critical component of the regulatory process required to inform the hazard characterisation and labelling of chemicals, thus guiding their safe handling and use. Skin irritation is a key endpoint for regulatory registrations, traditionally assessed by using the Draize rabbit test, which was introduced in 1944. Advances in New Approach Methodologies (NAMs), and particularly those based on reconstructed skin models, have led to more human-relevant alternatives, as compared to the in vivo test method. Here, we report on some groundbreaking high-throughput adaptations to the validated in vitro Skin Irritation Test (SIT; OECD TG 439) with the commercially available EpiDerm™ reconstructed human epidermis (RhE) model from MatTek Corporation. These minor adaptations enabled the safety evaluation of over 3000 chemicals in just ten months. This was necessary due to the large number of chemicals that needed to be pre-screened for dermal safety, after the insect repellent or insecticidal activity was predicted and assessed. The data obtained in the SIT were then used to train a new Artificial Intelligence (AI) tool, designed to predict the skin irritation potential of large libraries of molecules within (but not limited to) the chemical space which the AI tool is based upon. Our work exemplifies the integration of AI with existing non-animal-based approaches, in order to further develop chemical safety assessment by taking advantage of this new technology.