22 April 2023 in npj Computational Materials
Accurate and efficient prediction of polymer properties is of great significance in polymer design. Conventionally, expensive and time-consuming experiments or simulations are required to evaluate polymer functions. Recently, Transformer models, equipped with self-attention mechanisms, have exhibited superior performance in natural language processing. However, such methods have not been investigated in polymer sciences. Herein, we report TransPolymer, a Transformer-based language model for polymer property prediction. Our proposed polymer tokenizer with chemical awareness enables learning representations from polymer sequences. Rigorous experiments on ten polymer property prediction benchmarks demonstrate the superior performance of TransPolymer. Moreover, we show that TransPolymer benefits from pretraining on large unlabeled dataset via Masked Language Modeling. Experimental results further manifest the important role of self-attention in modeling polymer sequences. We highlight this model as a promising computational tool for promoting rational polymer design and understanding structure-property relationships from a data science view.
17 March 2026 in Nature Communications
Predicting crystal properties is essential for understanding structure-property relationships and accelerating material discovery. However, conventional approaches like experimental measurements or density functional theory calculations are resource-intensive, limiting their scalability. While machine learning offers a promising alternative by learning complex structure-property relationships from data, existing models often rely on labeled data, adopt representations insufficiently capturing essential structural characteristics, and lack integration of physics, limiting their generalizability and interpretability. Here, we introduce CLOUD (Crystal Language mOdel for Unified and Differentiable materials modeling), a transformer-based framework trained on a Symmetry-Consistent Ordered Parameter Encoding (SCOPE) that encodes crystal symmetry, Wyckoff positions, and composition in a compact, coordinate-free string representation. Pre-trained on over six million crystals, CLOUD is fine-tuned on downstream tasks and achieves competitive performance across diverse material properties, demonstrating strong scaling with respect to data and model size. Furthermore, as a proof-of-concept of differentiable materials modeling, CLOUD is applied to predict the phonon-related properties by integrating with the Debye model. This approach enforces thermodynamic consistency and enables temperature-dependent property prediction without requiring additional data. These results demonstrate CLOUD’s potential as a scalable and physics-informed foundation model for crystalline materials, unifying symmetry-consistent representations with physics-grounded learning for property prediction and materials discovery.