Analysis of genomic and metagenomic sequences is inherently more challenging than that of amino acid sequences due to the higher divergence among evolutionarily related nucleotide sequences, variable k-mer and codon usage within and among genomes of diverse species, and poorly understood selective constraints. We introduce Scorpio (Sequence Contrastive Optimization for Representation and Predictive Inference on DNA), a versatile framework designed for nucleotide sequences that employ contrastive learning to improve embeddings. By leveraging pre-trained genomic language models and k-mer frequency embeddings, Scorpio demonstrates competitive performance in diverse applications, including taxonomic and gene classification, antimicrobial resistance (AMR) gene identification, and promoter detection. A key strength of Scorpio is its ability to generalize to novel DNA sequences and taxa, addressing a significant limitation of alignment-based methods. Scorpio has been tested on multiple datasets with DNA sequences of varying lengths (long and short) and shows robust inference capabilities. Additionally, we provide an analysis of the biological information underlying this representation, including correlations between codon adaptation index as a gene expression factor, sequence similarity, and taxonomy, as well as the functional and structural information of genes.
This study demonstrates that generative large language models can be utilized in a more flexible manner for DNA sequence analysis and classification tasks compared to traditional transformer encoder-based models. While recent encoder-based models such as DNABERT and Nucleotide Transformer have shown significant performance in DNA sequence classification, transformer decoder-based generative models have not yet been extensively explored in this field. This study evaluates how effectively generative Large Language Models handle DNA sequences with various labels and analyzes performance changes when additional textual information is provided. Experiments were conducted on antimicrobial resistance genes, and the results show that generative Large Language Models can offer comparable or potentially better predictions, demonstrating flexibility and accuracy when incorporating both sequence and textual information. The code and data used in this work are available at the following GitHub repository: https://github.com/biocomgit/llm4dna.
Microbiome sample representation to input into LLMs is essential for downstream tasks such as phenotype prediction and environmental classification. While prior studies have explored embedding-based representations of each microbiome sample, most rely on simple averaging over sequence embeddings, often overlooking the biological importance of taxa abundance. In this work, we propose an abundance-aware variant of the Set Transformer to construct fixed-size sample-level embeddings by weighting sequence embeddings according to their relative abundance. Without modifying the model architecture, we replicate embedding vectors proportional to their abundance and apply self-attention-based aggregation. Our method outperforms average pooling and unweighted Set Transformers on real-world microbiome classification tasks, achieving perfect performance in some cases. These results demonstrate the utility of abundance-aware aggregation for robust and biologically informed microbiome representation. To the best of our knowledge, this is one of the first approaches to integrate sequence-level abundance into Transformer-based sample embeddings.
Since the advent of GANs and VAEs, image generation models have continuously evolved, opening up various real-world applications with the introduction of Stable Diffusion and DALL-E models. These text-to-image models can generate high-quality images for fields such as art, design, and advertising. However, they often produce aberrant images for certain prompts. This study proposes a method to mitigate such issues by fine-tuning the Stable Diffusion 3 model using the DreamBooth technique. Experimental results targeting the prompt "lying on the grass/street" demonstrate that the fine-tuned model shows improved performance in visual evaluation and metrics such as Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Frechet Inception Distance (FID). User surveys also indicated a higher preference for the fine-tuned model. This research is expected to make contributions to enhancing the practicality and reliability of text-to-image models.
Target-conditioned molecular design requires optimizing binding affinity to a proposed therapeutic protein target while balancing competing developability constraints (e.g., absorption, distribution, metabolism, excretion, and toxicity; ADMET). Yet many computational pipelines either optimize a single objective or rely on fully de novo generation that can be difficult to control and interpret. We present Nevermore, a target-conditioned, database-grounded framework that combines a geometry-aware protein–ligand affinity oracle with Pareto-aware multi-objective search over an explicit molecular feature space. A central design choice is to optimize in count-based Morgan fingerprint space, where each feature corresponds to a chemically meaningful substructure count, enabling discrete, interpretable “bucket-level” edits. Nevermore learns target-conditioned scores by aligning protein and ligand representations under contrastive objectives and using a similarity-based prediction head; the resulting affinity oracle improves over previously reported benchmark baselines, providing a stronger scoring signal for downstream optimization. Nevermore then steers candidate selection by proposing sparse fingerprint edits, re-ranking candidates under multiple objectives, and projecting edited fingerprints back to valid molecules via nearest-neighbor retrieval from a large compound library. This yields efficient screening without exhaustive enumeration and provides transparent attributions that connect optimization steps to concrete chemical motifs. We evaluate Nevermore on two target case studies (Menin and SARS-CoV-2 Mpro). Across targets, the closed-loop search consistently retrieves candidate sets with improved affinity–property trade-offs compared with random sampling and similarity-only retrieval baselines, while maintaining explicit control and interpretability through discrete feature-space edits. These results support database-grounded, feature-space steering as a practical route to target-conditioned multi-objective lead refinement without relying on fully de novo generation.