Demand Forecasting at Low Aggregation Levels using Factored Conditional Restricted Boltzmann Machine.

The electrical demand forecasting problem can be regarded as a non-linear time series prediction problem depending on many complex factors since it is required at various aggregation levels and at high resolution. To solve this challenging problem, various time series and machine learning approaches has been proposed in the literature. As an evolution of neural network-based prediction methods, deep learning techniques are expected to increase the prediction accuracy by being stochastic and allowing bi-directional connections between neurons. In this paper, we investigate a newly developed deep learning model for time series prediction, namely Factored Conditional Restricted Boltzmann Machine (FCRBM), and extend it for demand forecasting. The assessment is made on the EcoGrid EU dataset, consisting of aggregated electric power consumption, price and meteorological data collected from 1900 customers. The households are equipped with local generation and smart appliances capable of responding to real-time pricing signals. The results show that for the energy prediction problem solved here, FCRBM outperforms the benchmark machine learning approach, i.e. Support Vector Machine

Machine-learning of liquids: A multi-scale framework for liquid physics and coarse-graining using deep learning

Liquids are a condensed and non-crystallized phase of matter and their properties are typically governed by many-body effects. The physics and properties of liquids are a mixture of gas and solid phases, but quite different from any of them. The application of theoretical models developed for solid and gas phases to liquids is difficult and development of theoretical models for liquid is hindered by many-body effects. Computer simulation and experimental methods have played a crucial role in understanding of liquids. In fact, most of our knowledge about liquids are obtained using computer simulation, especially molecular dynamics (MD) and Monte Carlo simulations. MD simulations are successful in finding thermodynamic, structural, dynamical, and transport properties of liquids. However, MD simulation requires accurate potential parameters to model physical phenomena. It is noteworthy to state that with known potential parameters structural properties not only describe local arrangement of atoms, but they are enough to calculate various thermodynamic properties such as pressure and isothermal compressibility. The more intriguing part about liquids is the relationship between structural properties and potential parameters. This question can be seen as the inverse problem of liquid-state theory or as a coarse-graining problem, where the objective is to parameterize a force field to reproduce a reference structure. In this study, a deep neural network is used for atom-agnostic parametrization of the Lennard-Jones potential at different thermodynamic states. The DNN demonstrates good performance for two cases – parameterization of LJ particles and development of single-bead CG LJ potentials for simple multi-atom molecules through transfer learning obtained from LJ particles. The transferability and generalizability of the method are investigated by computing the total variation in the radial distribution function and Kullback-Leibler divergence for the coarse-grained model development. Our results indicate that deep learning can compute the solution to the inverse-problem of liquid-state theory (DeepILST) under the assumption of a predetermined pair potential in a coarse-grained model. In a follow-up study, statistical and deep learning-based methods are employed to obtain insights into the quasi-universal properties of simple liquids. In the first part, a statistical model is employed to provide a probabilistic explanation for the similarity in the structure of simple liquids interacting with different pair potential forms, collectively known as simple liquids. The methodology works by sampling the radial distribution function and the number of interacting particles within the cut-off distance and it produces the probability density function of the net force. We show that matching the probability distribution of the net force can be a direct route to parameterize simple liquids pair potentials with a similar structure, as the net force is the main component of the Newtonian equations of motion. The statistical model is assessed and validated against various cases. The physics and quasi-universality of simple liquids are also studied through deep learning by finding structurally-equivalent Lennard-Jones liquids with similar reduced RDFs, i.e., isomorphs. Structurally-equivalent Lennard-Jones liquids identify systems with constant order parameters in the space of non-dimensional temperature and density of Lennard-Jones liquids consistent with the approximate theoretical solution derived in the current study and other theoretical models. Considering various investigations performed in this study, we show the successful employment of statistical and deep learning approaches and coarse-graining methods in the physics of simple liquids. As shown in above examples, MD simulation is a popular and strong computational tool to compute microscopic and macroscopic properties of liquids. However, to determine properties of atomic systems to a good level of accuracy with minimal noise or fluctuation, MD simulations are performed over a long time ranging from a few nanoseconds to several tens to hundreds of nanoseconds depending on the system and the properties of interest. There have been attempts to go around this issue with enhanced sampling and theoretical models. In this study, by considering simple liquids, we explore the feasibility of significantly reducing the MD simulation time to compute various properties of monoatomic systems such as the structure, pressure, and isothermal compressibility. A deep denoising autoencoder network is trained to obtain structural and thermodynamic properties of Lennard-Jones liquids at various thermodynamic states using a single snapshot RDF as input. The algorithm is successful not only in predicting the RDF of a Lennard-Jones pair potential, but also it is generalizable to other simple liquid pair potentials such as exponential, Yukawa, and inverse-power-law potentials. In terms of computational efficiency, the number of snapshots required from MD simulation to obtain the accuracy of DAE predicted RDF is at least hundred snapshots, making the network highly efficient. The pressure and isothermal compressibility from DAE based RDFs are also comparable with those obtained from longtime MD simulation. To expand our frameworks for more complex liquids, we investigate development of coarse-grained (CG) models of water, which is far more complex than simple liquids. In this study, we train a neural network-based force field with two- and three-body interactions, which makes the developed force field interpretable. Within our framework, the requirement for accurate forces and energies is eliminated by using the local search algorithm instead of backpropagation. We successfully develop coarse-grained models of both classical and ab initio water models. We also investigate the dependency of the coarse-grained force field of water on the number of expansions, which shows that the double-well interaction, known as a signature of water-like behavior among spherically symmetric pairwise interactions, vanishes with the inclusion of three-body interactions. We also notice that the two-body interaction fails to reproduce the angular distribution of water, especially over a short range. Based on our findings, we conclude that water-like behavior is better captured using the three-body interaction, which is consistent with the directional dependency of interactions in water. Finally, we employ graph neural network in the phase identification of water. Due to the high dimensionality and uninterpretable nature of atomistic simulation data, researchers have developed a wide variety of order parameters to reduce dimensionality and connect data with the phase and structural properties. Motivated by the importance of water in various areas, water is studied through various order parameters such as bond-order parameter (BOP), tetrahedral order parameter, and local-structure index. Even though these order parameters are widely adapted in various studies ranging from ice nucleation, phase discrimination/identification, free energy calculation, and as collective variables of enhanced sampling simulation, however, they are far from perfect. In several cases, it requires lots of domain expertise and effort to combine multiple order parameters to reach conclusive findings. Our phase classification works by collecting all the pairwise distances of high dimensional data within a cut-off distance, followed by feeding these data into an edge-conditioned convolutional graph neural network. The high accuracy and no need for pre-calculation of other order parameters make our methods more rigorous and generalizable for more complex cases such as confined water