Implementing graph neural networks with TensorFlow-Keras

Graph neural networks are a versatile machine learning architecture that received a lot of attention recently. In this technical report, we present an implementation of convolution and pooling layers for TensorFlow-Keras models, which allows a seamless and flexible integration into standard Keras layers to set up graph models in a functional way. This implies the usage of mini-batches as the first tensor dimension, which can be realized via the new RaggedTensor class of TensorFlow best suited for graphs. We developed the Keras Graph Convolutional Neural Network Python package kgcnn based on TensorFlow-Keras that provides a set of Keras layers for graph networks which focus on a transparent tensor structure passed between layers and an ease-of-use mindset

Graph neural networks in TensorFlow-Keras with RaggedTensor representation (kgcnn)

Graph neural networks are a versatile machine learning architecture that received a lot of attention recently due to its wide range of applications. In this technical report, we present an implementation of graph convolution and graph pooling layers for TensorFlow-Keras models, which allows a seamless and flexible integration into standard Keras layers to set up graph models in a functional way. We developed the Keras Graph Convolutional Neural Network Python package kgcnn based on TensorFlow-Keras which focus on a transparent tensor structure passed between layers and an ease-of-use mindset

Neural networks trained on synthetically generated crystals can extract structural information from ICSD powder X-ray diffractograms

Machine learning techniques have successfully been used to extract structural information such as the crystal space group from powder X-ray diffractograms. However, training directly on simulated diffractograms from databases such as the ICSD is challenging due to its limited size, class-inhomogeneity, and bias toward certain structure types. We propose an alternative approach of generating synthetic crystals with random coordinates by using the symmetry operations of each space group. Based on this approach, we demonstrate online training of deep ResNet-like models on up to a few million unique on-the-fly generated synthetic diffractograms per hour. For our chosen task of space group classification, we achieved a test accuracy of 79.9% on unseen ICSD structure types from most space groups. This surpasses the 56.1% accuracy of the current state-of-the-art approach of training on ICSD crystals directly. Our results demonstrate that synthetically generated crystals can be used to extract structural information from ICSD powder diffractograms, which makes it possible to apply very large state-of-the-art machine learning models in the area of powder X-ray diffraction. We further show first steps toward applying our methodology to experimental data, where automated XRD data analysis is crucial, especially in high-throughput settings. While we focused on the prediction of the space group, our approach has the potential to be extended to related tasks in the future

MEGAN: Multi-Explanation Graph Attention Network

Explainable artificial intelligence (XAI) methods are expected to improve trust during human-AI interactions, provide tools for model analysis and extend human understanding of complex problems. Explanation-supervised training allows to improve explanation quality by training self-explaining XAI models on ground truth or human-generated explanations. However, existing explanation methods have limited expressiveness and interoperability due to the fact that only single explanations in form of node and edge importance are generated. To that end we propose the novel multi-explanation graph attention network (MEGAN). Our fully differentiable, attention-based model features multiple explanation channels, which can be chosen independently of the task specifications. We first validate our model on a synthetic graph regression dataset. We show that for the special single explanation case, our model significantly outperforms existing post-hoc and explanation-supervised baseline methods. Furthermore, we demonstrate significant advantages when using two explanations, both in quantitative explanation measures as well as in human interpretability. Finally, we demonstrate our model's capabilities on multiple real-world datasets. We find that our model produces sparse high-fidelity explanations consistent with human intuition about those tasks and at the same time matches state-of-the-art graph neural networks in predictive performance, indicating that explanations and accuracy are not necessarily a trade-off.Comment: 9 pages main text, 29 pages total, 19 figure

High‐Throughput Synthesis and Machine Learning Assisted Design of Photodegradable Hydrogels

Due to the large chemical space, the design of functional and responsive soft materials poses many challenges but also offers a wide range of opportunities in terms of the scope of possible properties. Herein, an experimental workflow for miniaturized combinatorial high-throughput screening of functional hydrogel libraries is reported. The data created from the analysis of the photodegradation process of more than 900 different types of hydrogel pads are used to train a machine learning model for automated decision making. Through iterative model optimization based on Bayesian optimization, a substantial improvement in response properties is achieved and thus expanded the scope of material properties obtainable within the chemical space of hydrogels in the study. It is therefore demonstrated that the potential of combining miniaturized high-throughput experiments with smart optimization algorithms for cost and time efficient optimization of materials properties

Interpretable delta-learning of GW quasiparticle energies from GGA-DFT

Accurate prediction of the ionization potential and electron affinity energies of small molecules are important for many applications. Density functional theory (DFT) is computationally inexpensive, but can be very inaccurate for frontier orbital energies or ionization energies. The GW method is sufficiently accurate for many relevant applications, but much more expensive than DFT. Here we study how we can learn to predict orbital energies with GW accuracy using machine learning (ML) on molecular graphs and fingerprints using an interpretable delta-learning approach. ML models presented here can be used to predict quasiparticle energies of small organic molecules even beyond the size of the molecules used for training. We furthermore analyze the learned DFT-to-GW corrections by mapping them to specific localized fragments of the molecules, in order to develop an intuitive interpretation of the learned corrections, and thus to better understand DFT errors

Analyzing Dynamical Disorder for Charge Transport in Organic Semiconductors via Machine Learning

Organic semiconductors are indispensable for today’s display technologies in the form of organic light-emitting diodes (OLEDs) and further optoelectronic applications. However, organic materials do not reach the same charge carrier mobility as inorganic semiconductors, limiting the efficiency of devices. To find or even design new organic semiconductors with higher charge carrier mobility, computational approaches, in particular multiscale models, are becoming increasingly important. However, such models are computationally very costly, especially when large systems and long timescales are required, which is the case to compute static and dynamic energy disorder, i.e., the dominant factor to determine charge transport. Here, we overcome this drawback by integrating machine learning models into multiscale simulations. This allows us to obtain unprecedented insight into relevant microscopic materials properties, in particular static and dynamic disorder contributions for a series of application-relevant molecules. We find that static disorder and thus the distribution of shallow traps are highly asymmetrical for many materials, impacting widely considered Gaussian disorder models. We furthermore analyze characteristic energy level fluctuation times and compare them to typical hopping rates to evaluate the importance of dynamic disorder for charge transport. We hope that our findings will significantly improve the accuracy of computational methods used to predict application-relevant materials properties of organic semiconductors and thus make these methods applicable for virtual materials design

Analyzing dynamical disorder for charge transport in organic semiconductors via machine learning

Organic semiconductors are indispensable for today's display technologies in form of organic light emitting diodes (OLEDs) and further optoelectronic applications. However, organic materials do not reach the same charge carrier mobility as inorganic semiconductors, limiting the efficiency of devices. To find or even design new organic semiconductors with higher charge carrier mobility, computational approaches, in particular multiscale models, are becoming increasingly important. However, such models are computationally very costly, especially when large systems and long time scales are required, which is the case to compute static and dynamic energy disorder, i.e. dominant factor to determine charge transport. Here we overcome this drawback by integrating machine learning models into multiscale simulations. This allows us to obtain unprecedented insight into relevant microscopic materials properties, in particular static and dynamic disorder contributions for a series of application-relevant molecules. We find that static disorder and thus the distribution of shallow traps is highly asymmetrical for many materials, impacting widely considered Gaussian disorder models. We furthermore analyse characteristic energy level fluctuation times and compare them to typical hopping rates to evaluate the importance of dynamic disorder for charge transport. We hope that our findings will significantly improve the accuracy of computational methods used to predict application relevant materials properties of organic semiconductors, and thus make these methods applicable for virtual materials design

Accurate GW frontier orbital energies of 134 kilo molecules

HOMO and LUMO energies are critical molecular properties that typically require high accuracy computations for practical applicability. Until now, a comprehensive dataset containing sufficiently accurate HOMO and LUMO energies has been unavailable. In this study, we introduce a new dataset of HOMO/LUMO energies for QM9 compounds, calculated using the GW method. The GW method offers adequate HOMO/LUMO prediction accuracy for diverse applications, exhibiting mean unsigned errors of 100 meV in the GW100 benchmark dataset. This database may serve as a benchmark of HOMO/LUMO prediction, delta-learning, and transfer learning, particularly for larger molecules where GW is the most accurate but still numerically feasible method. We anticipate that this dataset will enable the development of more accurate machine learning models for predicting molecular properties