Reinforcement learning (RL) is a powerful and flexible paradigm for searching for solutions in high-dimensional action spaces. However, bridging the gap between playing computer games with thousands of simulated episodes and solving real scientific problems with complex and involved environments (up to actual laboratory experiments) requires improvements in terms of sample efficiency to make the most of expensive information. The discovery of new drugs is a major commercial application of RL, motivated by the very large nature of the chemical space and the need to perform multiparameter optimization (MPO) across different properties. In silico methods, such as virtual library screening (VS) and de novo molecular generation with RL, show great promise in accelerating this search. However, incorporation of increasingly complex computational models in these workflows requires increasing sample efficiency. Here, we introduce an active learning system linked with an RL model (RL–AL) for molecular design, which aims to improve the sample-efficiency of the optimization process. We identity and characterize unique challenges combining RL and AL, investigate the interplay between the systems, and develop a novel AL approach to solve the MPO problem. Our approach greatly expedites the search for novel solutions relative to baseline-RL for simple ligand- and structure-based oracle functions, with a 5–66-fold increase in hits generated for a fixed oracle budget and a 4–64-fold reduction in computational time to find a specific number of hits. Furthermore, compounds discovered through RL–AL display substantial enrichment of a multi-parameter scoring objective, indicating superior efficacy in curating high-scoring compounds, without a reduction in output diversity. This significant acceleration improves the feasibility of oracle functions that have largely been overlooked in RL due to high computational costs, for example free energy perturbation methods, and in principle is applicable to any RL domain.
How many near-neighbors does a molecule have? This fundamental question in chemistry is crucial for molecular optimization problems under the similarity principle assumption. Generative models can sample molecules from a vast chemical space but lack explicit knowledge about molecular similarity. Therefore, these models need guidance from reinforcement learning to sample a relevant similar chemical space. However, they still miss a mechanism to measure the coverage of a specific region of the chemical space. To overcome these limitations, a source-target molecular transformer model, regularized via a similarity kernel function, is proposed. Trained on a largest dataset of ≥200 billion molecular pairs, the model enforces a direct relationship between generating a target molecule and its similarity to a source molecule. Results indicate that the regularization term significantly improves the correlation between generation probability and molecular similarity, enabling exhaustive exploration of molecule near-neighborhoods.
1 November 2024 in Journal of Chemical Information and Modeling
This paper presents an innovative framework for fast-developing broadband, low-profile, dual-polarized cross-dipole antennas. This approach integrates a broadband, petal-shaped cross dipole with a 3D-printed structure comprising multi-layers of high permittivity material with varying heights and radii. In the absence of a theoretical model and a satisfactory initial design, the design process employs the enhanced Tree-structured Parzen Estimator Bayesian Optimization (TPE-BO) algorithm, a significant advancement over traditional Bayesian methods, for effective hyper-parameter tuning to streamline the design. The proposed method only adjusts the high dielectric constant loading (HDL) structure without changing the original dimensions of the dipole. Compared to conventional optimization algorithms integrated within CST, this method efficiently achieves the desired broadband and low profile performance. Meanwhile, the mechanism of the HDL structure in increasing the bandwidth (BW) while enabling the dipole’s low profile performance is explained. The proposed approach has been validated by fabricating and measuring a hybrid antenna prototype. The measured results show that the antenna achieves an overlap BW between the impedance matching BW and the 3 dB gain BW of 79.3% (from 1.62 to 3.75 GHz) with a voltage standing wave ratio (VSWR) of less than 1.6. Additionally, the isolation between the two ports exceeds 20.5 dB. Significantly, this HDL technique notably enhances the BW and reduces the antenna’s profile from 0.2λ L to 0.1λ L compared to a single petal-shaped cross dipole. Thus, this study provides a simple, effective, and repeatable design strategy for realizing broadband and low profile performance in dipole antennas.
Here we introduce FLOWR, a structure-based framework for the generation and optimization of three-dimensional ligands. FLOWR integrates continuous and categorical flow matching with equivariant optimal transport, enhanced by an efficient protein pocket conditioning. Alongside FLOWR, we present SPINDR, a curated dataset comprising ligand–pocket cocrystal complexes specifically designed to address existing data quality issues. Empirical evaluations demonstrate that FLOWR surpasses current state-of-the-art diffusion- and flow-based methods in terms of PoseBusters-validity, pose accuracy and interaction recovery, while offering an inference speed-up, achieving up to 70-fold faster performance. In addition, we introduce FLOWR.MULTI, a highly accurate multi-purpose model allowing for the targeted sampling of ligands that adhere to predefined interaction profiles and chemical substructures for fragment-based design without the need of retraining or any resampling strategies. Collectively, our results indicate that FLOWR and FLOWR.MULTI represent an advancement in artificial intelligence-driven structure-based drug design, substantially enhancing the reliability and applicability of de novo, interaction- and fragment-based ligand generation in real-world drug discovery settings.
REINVENT4 is a modern open–source generative AI framework for the design of small molecules. The software utilizes recurrent neural networks and transformer architectures to drive molecule generation. These generators are seamlessly embedded within the general machine learning optimization algorithms transfer learning, reinforcement learning and curriculum learning. REINVENT4 enables and facilitates de novo design, R-group replacement, library design, linker design, scaffold hopping and molecule optimization. This contribution gives an overview of the software and describes its design. Algorithms and their applications are discussed in detail. REINVENT4 is a command line tool which reads a user configuration in either TOML or JSON format. The aim of this release is to provide reference implementations for some of the most common algorithms in AI based molecule generation. An additional goal with the release is to create a framework for education and future innovation in AI based molecular design. The software is available from https://github.com/ MolecularAI/REINVENT4 and released under the permissive Apache 2.0 license.