Transformer models have emerged as pivotal tools within the realm of drug discovery, distinguished by their unique architectural features and exceptional performance in managing intricate data landscapes. Leveraging the innate capabilities of transformer architectures to comprehend intricate hierarchical dependencies inherent in sequential data, these models showcase remarkable efficacy across various tasks, including new drug design and drug target identification. The adaptability of pre-trained transformer-based models renders them indispensable assets for driving data-centric advancements in drug discovery, chemistry, and biology, furnishing a robust framework that expedites innovation and discovery within these domains. Beyond their technical prowess, the success of transformer-based models in drug discovery, chemistry, and biology extends to their interdisciplinary potential, seamlessly combining biological, physical, chemical, and pharmacological insights to bridge gaps across diverse disciplines. This integrative approach not only enhances the depth and breadth of research endeavors but also fosters synergistic collaborations and exchange of ideas among disparate fields. In our review, we elucidate the myriad applications of transformers in drug discovery, as well as chemistry and biology, spanning from protein design and protein engineering, to molecular dynamics (MD), drug target identification, transformer-enabled drug virtual screening (VS), drug lead optimization, drug addiction, small data set challenges, chemical and biological image analysis, chemical language understanding, and single cell data. Finally, we conclude the survey by deliberating on promising trends in transformer models within the context of drug discovery and other sciences.

A transformer is the foundational architecture behind large language models designed to handle sequential data by using mechanisms of self‐attention to weigh the importance of different elements, enabling efficient processing and understanding of complex patterns. Recently, transformer‐based models have become some of the most popular and powerful deep learning (DL) algorithms in molecular science, owing to their distinctive architectural characteristics and proficiency in handling intricate data. These models leverage the capacity of transformer architectures to capture complex hierarchical dependencies within sequential data. As the applications of transformers in molecular science are very widespread, in this review, we only focus on the technical aspects of transformer technology in molecule domain. Specifically, we will provide an in‐depth investigation into the algorithms of transformer‐based machine learning techniques in molecular science. The models under consideration include generative pre‐trained transformer (GPT), bidirectional and auto‐regressive transformers (BART), bidirectional encoder representations from transformers (BERT), graph transformer, transformer‐XL, text‐to‐text transfer transformer, vision transformers (ViT), detection transformer (DETR), conformer, contrastive language‐image pre‐training (CLIP), sparse transformers, and mobile and efficient transformers. By examining the inner workings of these models, we aim to elucidate how their architectural innovations contribute to their effectiveness in processing complex molecular data. We will also discuss promising trends in transformer models within the context of molecular science, emphasizing their technical capabilities and potential for interdisciplinary research. This review seeks to provide a comprehensive understanding of the transformer‐based machine learning techniques that are driving advancements in molecular science.This article is categorized under: Data Science > Chemoinformatics Data Science > Artificial Intelligence/Machine Learning

Pain is a significant global health issue, and the current treatment options for pain management have limitations in terms of effectiveness, side effects, and potential for addiction. There is a pressing need for improved pain treatments and the development of new drugs. Voltage-gated sodium channels, particularly Nav1.3, Nav1.7, Nav1.8, and Nav1.9, play a crucial role in neuronal excitability and are predominantly expressed in the peripheral nervous system. Targeting these channels may provide a means to treat pain while minimizing central and cardiac adverse effects. In this study, we construct protein–protein interaction (PPI) networks based on pain-related sodium channels and develop a corresponding drug–target interaction network to identify potential lead compounds for pain management. To ensure reliable machine learning predictions, we carefully select 111 inhibitor data sets from a pool of more than 1000 targets in the PPI network. We employ 3 distinct machine learning algorithms combined with advanced natural language processing (NLP)–based embeddings, specifically pretrained transformer and autoencoder representations. Through a systematic screening process, we evaluate the side effects and repurposing potential of more than 150,000 drug candidates targeting Nav1.7 and Nav1.8 sodium channels. In addition, we assess the ADMET (absorption, distribution, metabolism, excretion, and toxicity) properties of these candidates to identify leads with near-optimal characteristics. Our strategy provides an innovative platform for the pharmacological development of pain treatments, offering the potential for improved efficacy and reduced side effects.

Anesthetics are crucial in surgical procedures and therapeutic interventions, but they come with side effects and varying levels of effectiveness, calling for novel anesthetic agents that offer more precise and controllable effects. Targeting Gamma-aminobutyric acid (GABA) receptors, the primary inhibitory receptors in the central nervous system, could enhance their inhibitory action, potentially reducing side effects while improving the potency of anesthetics. In this study, we introduce a proteomic learning of GABA receptor-mediated anesthesia based on 24 GABA receptor subtypes by considering over 4000 proteins in protein-protein interaction (PPI) networks and over 1.5 millions known binding compounds. We develop a corresponding drug-target interaction network to identify potential lead compounds for novel anesthetic design. To ensure robust proteomic learning predictions, we curated a data set comprising 136 targets from a pool of 980 targets within the PPI networks. We employed three machine learning algorithms, integrating advanced natural language processing (NLP) models such as pretrained transformers and autoencoder embeddings. Through a comprehensive screening process, we evaluated the side effects and repurposing potential of over 180,000 drug candidates targeting the GABRA5 receptor. Additionally, we assessed the ADMET (absorption, distribution, metabolism, excretion, and toxicity) properties of these candidates to identify those with near-optimal characteristics. This approach also involved optimizing the structures of existing anesthetics. Our work presents an innovative strategy for the development of new anesthetic drugs, optimization of anesthetic use, and a deeper understanding of potential anesthesia-related side effects.