Atomic force microscopy (AFM) provides a platform for high-resolution topographical imaging and the mechanical characterization of a wide range of samples, including live cells, proteins, and other biomolecules. AFM is also instrumental for measuring interaction forces and binding kinetics for protein–protein or receptor–ligand interactions on live cells at a single-molecule level. However, performing force measurements and high-resolution imaging with AFM and data analytics are time-consuming and require special skill sets and continuous human supervision. Recently, researchers have explored the applications of artificial intelligence (AI) and deep learning (DL) in the bioimaging field. However, the applications of AI to AFM operations for live-cell characterization are little-known. In this work, we implemented a DL framework to perform automatic sample selection based on the cell shape for AFM probe navigation during AFM biomechanical mapping. We also established a closed-loop scanner trajectory control for measuring multiple cell samples at high speed for automated navigation. With this, we achieved a speed-up in AFM navigation and reduced the time involved in searching for the particular cell shape in a large sample. Our innovation directly applies to many bio-AFM applications with AI-guided intelligent automation through image data analysis together with smart navigation.
7 September 2023 in Engineering Applications of Artificial Intelligence
Atomic Force Microscopy (AFM) has emerged as a powerful tool for nanoscale imaging and quantitative characterization of organic (e.g., live cells, proteins, DNA, lipid bilayers) and inorganic (e.g., silicon wafers, polymers) specimens. However, image artifacts in AFM height and peak force error images directly affect the precision of nanomechanical measurements. Experimentalists face considerable challenges in obtaining high-quality AFM images due to the requirement of specialized expertise and constant manual monitoring. Another challenge is the lack of high-quality AFM datasets to train machine learning models for automated defect detection. In this work, we propose a two-step AI framework that combines a vision-based deep learning (DL) model for classifying AFM image defects with a Large Language Models (LLMs)-based conversational assistant that provides real-time corrective guidance in natural language, making it particularly valuable for non-AFM experts aiming to obtain high-quality images. We curated an annotated AFM defect dataset spanning organic and inorganic samples to train the defect detection model. Our defect classification model achieves 91.43% overall accuracy, with a recall of 93% for tip contamination and 60% not-tracking defects. We further develop an intuitive user interface that enables seamless interaction with the DL model and integrates an LLM-based guidance feature to support users in understanding defects and improving future experiments. We then evaluate the performance of multiple state-of-the-art LLMs on AFM-related queries, offering users flexibility in LLM selection based on their specific needs. LLM evaluations and the benchmark questions are available at: https://github.com/idealab-isu/AFM-LLM-Defect-Guidance
Recent advances in generative modeling, namely Diffusion models, have revolutionized generative modeling, enabling high-quality image generation tailored to user needs. This paper proposes a framework for the generative design of structural components. Specifically, we employ a Latent Diffusion model to generate potential designs of a component that can satisfy a set of problem-specific loading conditions. One of the distinct advantages our approach offers over other generative approaches, such as generative adversarial networks (GANs), is that it permits the editing of existing designs. We train our model using a dataset of geometries obtained from structural topology optimization utilizing the SIMP algorithm. Consequently, our framework generates inherently near-optimal designs. Our work presents quantitative results that support the structural performance of the generated designs and the variability in potential candidate designs. Furthermore, we provide evidence of the scalability of our framework by operating over voxel domains with resolutions varying from $32^3$ to $128^3$. Our framework can be used as a starting point for generating novel near-optimal designs similar to topology-optimized designs.
Recent advancements in deep learning for predicting 3D protein structures have shown promise, particularly when leveraging inputs like protein sequences and Cryo-Electron microscopy (Cryo-EM) images. However, these techniques often fall short when predicting the structures of protein complexes (PCs), which involve multiple proteins. In our study, we investigate using atomic force microscopy (AFM) combined with deep learning to predict the 3D structures of PCs. AFM generates height maps that depict the PCs in various random orientations, providing a rich information for training a neural network to predict the 3D structures. We then employ the pre-trained UpFusion model (which utilizes a conditional diffusion model for synthesizing novel views) to train an instance-specific NeRF model for 3D reconstruction. The performance of UpFusion is evaluated through zero-shot predictions of 3D protein structures using AFM images. The challenge, however, lies in the time-intensive and impractical nature of collecting actual AFM images. To address this, we use a virtual AFM imaging process that transforms a `PDB' protein file into multi-view 2D virtual AFM images via volume rendering techniques. We extensively validate the UpFusion architecture using both virtual and actual multi-view AFM images. Our results include a comparison of structures predicted with varying numbers of views and different sets of views. This novel approach holds significant potential for enhancing the accuracy of protein complex structure predictions with further fine-tuning of the UpFusion network.