The R & D of new materials is accelerating. A team from Shanghai Jiao Tong University has developed a new AI material design model, CGformer, which integrates the global attention mechanism.
The research team led by Professors Li Jinjin and Huang Fuqiang from the Artificial Intelligence and Microstructure Laboratory at Shanghai Jiao Tong University has developed a brand - new AI material design model called CGformer, successfully breaking through the limitations of traditional crystal graph neural networks.
Artificial intelligence is profoundly reshaping the R & D paradigm of materials science and demonstrating breakthrough value in accelerating the discovery of new materials and optimizing their performance. Through the in - depth integration of high - throughput computing and machine learning, the pain points of the traditional "trial - and - error method", such as long experimental cycles and high resource consumption, have been effectively addressed. Materials exploration has entered an efficient iterative stage of "computation - driven and experiment - verified". However, with the innovation of human technology and lifestyle, the performance requirements for new materials in fields such as new energy and aerospace are becoming increasingly stringent. The limitations of traditional machine - learning methods are gradually emerging, especially in the field of high - entropy material R & D.
So - called "high - entropy" materials are a new type of materials prepared by mixing multiple principal elements. High - entropy materials significantly increase the configurational entropy (i.e., disorder) of atomic arrangements through the synergistic effect of multiple principal elements, thus endowing them with more excellent comprehensive properties such as mechanics, high - temperature resistance, and corrosion resistance compared with traditional materials. They have important application potential in fields such as energy storage, aerospace, and equipment for extreme environments.
Previous methods, such as artificial intelligence models like Crystal graph convolutional neural networks (CGCNN) and Atomistic line graph neural network (ALIGNN), all have architectural defects: Limited by the local information interaction mechanism, they have difficulty in modeling the long - distance atomic synergistic effect and cannot fully capture the global information unique to complex crystal structures, resulting in limited prediction accuracy. Meanwhile, the inherent characteristics of high - entropy materials also pose far greater challenges to their R & D than traditional materials. The complex microstructure, scarce high - quality experimental data, and dynamic and disordered atomic behavior together constitute the key obstacles on the road to high - entropy material R & D.
In response to the tool defects and upgraded requirements, the research team led by Professors Li Jinjin and Huang Fuqiang from the Artificial Intelligence and Microstructure Laboratory (AIMS - Lab) at Shanghai Jiao Tong University has developed a brand - new AI material design model called CGformer, successfully breaking through the limitations of traditional models. This model innovatively integrates the global attention mechanism of Graphormer with CGCNN and incorporates centrality encoding and spatial encoding. It can not only intuitively describe the material structure through the crystal graph but also capture the interactions between long - distance atoms through the "global attention" mechanism, thus obtaining the global information processing ability that traditional models "only focusing on adjacent atoms" do not possess.
This method provides more comprehensive structural information, which helps to more accurately predict the ion migration behavior inside the structure and provides a reliable tool for the R & D of new materials, especially high - entropy and complex - structure crystal materials. The research results were published in the top - tier journal Matter under the title "CGformer: Transformer - enhanced crystal graph network with global attention for material property prediction".
Research Highlights:
* The research developed an AI material design model called CGformer based on the global attention mechanism, providing a reliable and powerful tool for materials R & D science and helping to accelerate the discovery process of complex crystal structures.
* Compared with CGCNN, in the research on high - entropy sodium - ion solid - state electrolytes (HE - NSEs), the mean absolute error of CGformer was reduced by 25%, effectively confirming its practicality and advancement.
* 18 structures were screened from 148,995 possible high - entropy structures, and 6 high - entropy sodium - ion solid - state electrolytes (HE - NSEs) were successfully synthesized and verified. The sodium - ion conductivity at room temperature was as high as 0.256 mS/cm, demonstrating its practical application value.
Paper address: https://www.cell.com/matter/abstract/S2590 - 2385(25)00423 - 0
Multi - category Datasets Improve the Capability of CGformer Model
The purpose of this research is to address the challenges brought by data scarcity and structural complexity in high - entropy systems through a case - based solution. The research cases focus on the applications of new - energy electric vehicles and grid energy storage, specifically around the performance prediction and screening of high - entropy sodium - ion solid - state electrolytes. Multiple types of datasets were constructed and used to support the training, fine - tuning, and experimental verification of the CGformer model, as follows:
Basic dataset of sodium - ion diffusion energy barrier (Eb): This is the largest known dataset of sodium - ion diffusion energy barriers in high - entropy structures constructed by the researchers for this study, based on the crystal structure analysis (CAVD) of Voronoi decomposition and the Bond Valence Site Energy (BVSE) method. This dataset is mainly used for the pre - training of CGformer, enabling the model to learn the graph information related to sodium - containing structures and then transfer it to the calculated high - entropy dataset, laying the foundation for the subsequent prediction of Eb of high - entropy materials.
CGformer screening workflow
Computational dataset of HE - NSEs: Based on Na₃Zr₂Si₂PO₁₂ (as shown in the figure above), 45 potential high - entropy doping elements were considered at the Zr site, initially forming an initial chemical space containing 148,995 possible high - entropy structures. After multiple rounds of screening, including excluding unsuitable elements (radioactive, highly toxic, and expensive elements), constraining the atomic radius difference and charge balance, etc., the chemical space was further reduced to 826 relatively stable structures. Then, through unsupervised hierarchical clustering, they were divided into 20 groups. 30% of the structures were sampled from each group (a total of 238 structures), and their Eb values were obtained through density functional theory (DFT) calculations. Finally, a dedicated dataset for fine - tuning CGformer was formed, so that the model could be specifically adapted to the prediction task of sodium - ion Eb in high - entropy NASICON structures and improve the model's accuracy in the target scenario.
Thermal stability evaluation dataset: The researchers extracted the energy above the convex hull value (Ehull) of all sodium - containing structures from the Materials Project database and compiled them into a dedicated training set. This dataset is mainly used to train a supplementary model for evaluating the thermodynamic stability of HE - NSEs. Combined with the Eb predicted by CGformer, candidate materials with "performance + stability" can be screened.
Innovative Fusion Architecture Enables CGformer's "Global Perception"
CGformer has fundamentally innovated in response to the deficiencies of traditional methods, organically integrating two advanced technologies to achieve complementary advantages. Its core is to retain the ability to graphically represent crystal structures and break the limitation of only focusing on local atomic interactions through the global attention mechanism. Specifically, it combines the global attention mechanism of Graphormer with the crystal graph representation method of CGCNN and adds key encoding modules, thus constructing a brand - new information processing process.
The following figure a shows the crystal graph encoding process. This process is to convert the real three - dimensional crystal structure into a crystal graph that the model can process. Atoms in the crystal structure are represented as nodes, and chemical bonds between atoms are represented as edges. Through the conversion process, researchers can extract node and edge features, such as various element properties, charges, covalent radii, inter - atomic distances, bond types, and crystal symmetry information, and then combine these features to obtain the initial input data required by CGformer, ensuring that the chemical and structural information of the crystal is completely retained.
Schematic diagram of the encoding process from crystal to crystal graph
The following figure b shows the network architecture of CGformer. Through the collaboration of multiple modules, global information integration and accurate prediction are achieved. First, the input crystal graph undergoes a round of graph convolution operations to generate a simplified graph structure, thereby reducing the computational load of subsequent network layers and accelerating the training process of CGformer. Then, on this basis, the researchers calculate the central encoding and update the node features of the crystal graph. The central encoding includes the in - degree and out - degree of each node and is then integrated into the original node features. Subsequently, each node uses the multi - head attention module to represent the positional relationship between nodes in combination with variable features and spatial encoding. The central encoding converts the average features of adjacent nodes into a sum form, while the spatial encoding enables the self - attention mechanism to distinguish adjacent nodes, promoting effective message aggregation and enhancing the information connection between different atoms. Finally, the output vector undergoes the "pooling (integrating global features)" and "activation (functional operation)" processes to complete the final material property prediction.
Network architecture of CGformer
It is worth mentioning that the multi - head attention module enables each node to "pay attention" to all other nodes in the crystal graph, rather than just adjacent nodes, achieving the capture of long - range atomic interactions. Meanwhile, the addition of central encoding and spatial encoding also enables the model to not only recognize the chemical properties of atoms but also perceive their "positional importance" and "spatial relationship" in the structure, improving the model's characterization accuracy of complex crystals.
In general, compared with traditional crystal networks, CGformer has achieved a qualitative leap, realizing three major advantages of global vision, information enhancement, and efficiency balance, and providing a credible and reliable tool for the discovery and performance optimization of complex high - entropy materials.
CGformer Demonstrates Powerful Performance and Highlights Practical Guiding Value
To accurately evaluate the performance and advancement of the CGformer model, the researchers compared it with traditional models such as CGCNN, ALIGNN, and SchNet. The experiments verified the prediction accuracy of CGformer from two stages: "pre - training" to "fine - tuning".
In the pre - training stage (as shown in the figure below), CGformer demonstrated better stability and prediction accuracy. The initial error and fluctuation of CGformer were significantly lower than those of CGCNN. The 10 - fold cross - validation (10 - fold CV) showed that the mean absolute error (MAE) of the training set was 0.1703, a 25.7% improvement in performance compared with CGCNN; the average MAE of the test set was 0.3205, nearly a 10% improvement in performance compared with CGCNN. The comparison with ALIGNN and SchNet further highlighted the excellent performance of CGformer.
Comparison of the effects of CGformer and CGCNN
In terms of the fitting effect, the deviation between the predicted values and the real values of CGformer was smaller, the residuals were more concentrated around 0, and the standard deviation of the residuals was smaller, proving that its prediction of sodium - ion Eb was more reliable.
In the fine - tuning stage (as shown in the figure below), after 200 rounds of fine - tuning of the pre - trained CGformer, the MAE significantly decreased after about 10 rounds. The average MAE of the final 10 - fold cross - validation was only 0.0361. After fine - tuning, the deviation between the predicted values and the real values was further reduced. The residuals were mainly concentrated in the range of - 0.05 to 0.05 and showed a good normal distribution, proving that its prediction of Eb in high - entropy systems had extremely high accuracy, reflecting its application potential in scenarios with data scarcity.
Fine - tuning results of CGformer on the calculated high - entropy sodium Eb dataset
Finally, the researchers experimentally synthesized and electrochemically characterized the 6 optimal HE - NSEs screened by CGformer to verify their structures and performances. The results showed that these materials all exhibited excellent room - temperature ionic conductivity. At a room temperature of 25°C, the sodium - ion conductivity was between 0.093 and 0.256 mS/cm, significantly higher than that of undoped Na₃Zr₂Si₂PO₁₂.
Experimental verification of the selected HE - NSEs
"Artificial Intelligence + Materials" Has Become the Mainstream of Materials Science Development
"Artificial Intelligence + Materials" has become a cutting - edge research direction in the current field of materials science. By integrating artificial intelligence technology with the R & D, design, and application of materials, it demonstrates the powerful development potential and application value of the intersection of the two disciplines. The proposal of CGformer undoubtedly adds a brilliant stroke to the application of artificial intelligence in the field of materials science. Due to its unique and innovative algorithm architecture, it solves the key problems in high - entropy material R & D.
CGformer is just the tip of the iceberg of the exploration of AIMS - Lab in the field of artificial intelligence and materials science. The interdisciplinary research between artificial intelligence and materials science, as one of the main research directions of the laboratory, has long been a vivid footnote of the laboratory and has yielded fruitful results.
Also from this team, last year, with the title "Transformer enables ion transport behavior evolution and conductivity regulation for solid electrolyte", they presented their research results in the international top - tier journal Energy Storage Materials. The research proposed an artificial intelligence model called T - AIMD, which uses the Transformer network architecture. It can not only significantly reduce the computational cost but also quickly and accurately predict the behavior of any ion in any crystal structure. This method has increased the simulation speed of traditional AIMD by more than 100 times, significantly accelerating the material performance evaluation process.