14 research outputs found
Recommended from our members
Perspective: new insights from loss function landscapes of neural networks
Abstract: We investigate the structure of the loss function landscape for neural networks subject to dataset mislabelling, increased training set diversity, and reduced node connectivity, using various techniques developed for energy landscape exploration. The benchmarking models are classification problems for atomic geometry optimisation and hand-written digit prediction. We consider the effect of varying the size of the atomic configuration space used to generate initial geometries and find that the number of stationary points increases rapidly with the size of the training configuration space. We introduce a measure of node locality to limit network connectivity and perturb permutational weight symmetry, and examine how this parameter affects the resulting landscapes. We find that highly-reduced systems have low capacity and exhibit landscapes with very few minima. On the other hand, small amounts of reduced connectivity can enhance network expressibility and can yield more complex landscapes. Investigating the effect of deliberate classification errors in the training data, we find that the variance in testing AUC, computed over a sample of minima, grows significantly with the training error, providing new insight into the role of the variance-bias trade-off when training under noise. Finally, we illustrate how the number of local minima for networks with two and three hidden layers, but a comparable number of variable edge weights, increases significantly with the number of layers, and as the number of training data decreases. This work helps shed further light on neural network loss landscapes and provides guidance for future work on neural network training and optimisation
Machine learning enabled experimental design and parameter estimation for ultrafast spin dynamics
Advanced experimental measurements are crucial for driving theoretical
developments and unveiling novel phenomena in condensed matter and material
physics, which often suffer from the scarcity of facility resources and
increasing complexities. To address the limitations, we introduce a methodology
that combines machine learning with Bayesian optimal experimental design
(BOED), exemplified with x-ray photon fluctuation spectroscopy (XPFS)
measurements for spin fluctuations. Our method employs a neural network model
for large-scale spin dynamics simulations for precise distribution and utility
calculations in BOED. The capability of automatic differentiation from the
neural network model is further leveraged for more robust and accurate
parameter estimation. Our numerical benchmarks demonstrate the superior
performance of our method in guiding XPFS experiments, predicting model
parameters, and yielding more informative measurements within limited
experimental time. Although focusing on XPFS and spin fluctuations, our method
can be adapted to other experiments, facilitating more efficient data
collection and accelerating scientific discoveries
Machine learning for cardiac ultrasound time series data
We consider the problem of identifying frames in a cardiac ultrasound video associated with left ventricular chamber end-systolic (ES, contraction) and end-diastolic (ED, expansion) phases of the cardiac cycle. Our procedure involves a simple application of non-negative matrix factorization (NMF) to a series of frames of a video from a single patient. Rank-2 NMF is performed to compute two end-members. The end members are shown to be close representations of the actual heart morphology at the end of each phase of the heart function. Moreover, the entire time series can be represented as a linear combination of these two end-member states thus providing a very low dimensional representation of the time dynamics of the heart. Unlike previous work, our methods do not require any electrocardiogram (ECG) information in order to select the end-diastolic frame. Results are presented for a data set of 99 patients including both healthy and diseased examples.UCLA through the Physical Sciences Division; Entrepreneurship and Innovation Fund; Department of Mathematics; NSF [DMS-1045536, DMS-1417674]; ONR [N00014-16-1-2119]; Cross-disciplinary Scholars in Science and Technology (CSST) program at UCLACPCI-S(ISTP)1013
Capturing dynamical correlations using implicit neural representations
The observation and description of collective excitations in solids is a
fundamental issue when seeking to understand the physics of a many-body system.
Analysis of these excitations is usually carried out by measuring the dynamical
structure factor, S(Q, ), with inelastic neutron or x-ray scattering
techniques and comparing this against a calculated dynamical model. Here, we
develop an artificial intelligence framework which combines a neural network
trained to mimic simulated data from a model Hamiltonian with automatic
differentiation to recover unknown parameters from experimental data. We
benchmark this approach on a Linear Spin Wave Theory (LSWT) simulator and
advanced inelastic neutron scattering data from the square-lattice spin-1
antiferromagnet LaNiO. We find that the model predicts the unknown
parameters with excellent agreement relative to analytical fitting. In doing
so, we illustrate the ability to build and train a differentiable model only
once, which then can be applied in real-time to multi-dimensional scattering
data, without the need for human-guided peak finding and fitting algorithms.
This prototypical approach promises a new technology for this field to
automatically detect and refine more advanced models for ordered quantum
systems.Comment: 12 pages, 7 figure
Testing the data framework for an AI algorithm in preparation for high data rate X-ray facilities
The advent of next-generation X-ray free electron lasers will be capable of
delivering X-rays at a repetition rate approaching 1 MHz continuously. This
will require the development of data systems to handle experiments at these
type of facilities, especially for high throughput applications, such as
femtosecond X-ray crystallography and X-ray photon fluctuation spectroscopy.
Here, we demonstrate a framework which captures single shot X-ray data at the
LCLS and implements a machine-learning algorithm to automatically extract the
contrast parameter from the collected data. We measure the time required to
return the results and assess the feasibility of using this framework at high
data volume. We use this experiment to determine the feasibility of solutions
for `live' data analysis at the MHz repetition rate
Recommended from our members
Perspective: new insights from loss function landscapes of neural networks
Abstract: We investigate the structure of the loss function landscape for neural networks subject to dataset mislabelling, increased training set diversity, and reduced node connectivity, using various techniques developed for energy landscape exploration. The benchmarking models are classification problems for atomic geometry optimisation and hand-written digit prediction. We consider the effect of varying the size of the atomic configuration space used to generate initial geometries and find that the number of stationary points increases rapidly with the size of the training configuration space. We introduce a measure of node locality to limit network connectivity and perturb permutational weight symmetry, and examine how this parameter affects the resulting landscapes. We find that highly-reduced systems have low capacity and exhibit landscapes with very few minima. On the other hand, small amounts of reduced connectivity can enhance network expressibility and can yield more complex landscapes. Investigating the effect of deliberate classification errors in the training data, we find that the variance in testing AUC, computed over a sample of minima, grows significantly with the training error, providing new insight into the role of the variance-bias trade-off when training under noise. Finally, we illustrate how the number of local minima for networks with two and three hidden layers, but a comparable number of variable edge weights, increases significantly with the number of layers, and as the number of training data decreases. This work helps shed further light on neural network loss landscapes and provides guidance for future work on neural network training and optimisation
Recommended from our members
Perspective: new insights from loss function landscapes of neural networks
Abstract: We investigate the structure of the loss function landscape for neural networks subject to dataset mislabelling, increased training set diversity, and reduced node connectivity, using various techniques developed for energy landscape exploration. The benchmarking models are classification problems for atomic geometry optimisation and hand-written digit prediction. We consider the effect of varying the size of the atomic configuration space used to generate initial geometries and find that the number of stationary points increases rapidly with the size of the training configuration space. We introduce a measure of node locality to limit network connectivity and perturb permutational weight symmetry, and examine how this parameter affects the resulting landscapes. We find that highly-reduced systems have low capacity and exhibit landscapes with very few minima. On the other hand, small amounts of reduced connectivity can enhance network expressibility and can yield more complex landscapes. Investigating the effect of deliberate classification errors in the training data, we find that the variance in testing AUC, computed over a sample of minima, grows significantly with the training error, providing new insight into the role of the variance-bias trade-off when training under noise. Finally, we illustrate how the number of local minima for networks with two and three hidden layers, but a comparable number of variable edge weights, increases significantly with the number of layers, and as the number of training data decreases. This work helps shed further light on neural network loss landscapes and provides guidance for future work on neural network training and optimisation
Bayesian experimental design and parameter estimation for ultrafast spin dynamics
Advanced experimental measurements are crucial for driving theoretical developments and unveiling novel phenomena in condensed matter and materials physics, which often suffer from the scarcity of large-scale facility resources, such as x-ray or neutron scattering centers. To address these limitations, we introduce a methodology that leverages the Bayesian optimal experimental design paradigm to efficiently uncover key quantum spin fluctuation parameters from x-ray photon fluctuation spectroscopy (XPFS) data. Our method is compatible with existing theoretical simulation pipelines and can also be used in combination with fast machine learning surrogate models in the event that real-time simulations are unfeasible. Our numerical benchmarks demonstrate the superior performance in predicting model parameters and in delivering more informative measurements within limited experimental time. Our method can be adapted to many different types of experiments beyond XPFS and spin fluctuation studies, facilitating more efficient data collection and accelerating scientific discoveries
A machine learning photon detection algorithm for coherent x-ray ultrafast fluctuation analysis
X-ray free electron laser experiments have brought unique capabilities and opened new directions in research, such as creating new states of matter or directly measuring atomic motion. One such area is the ability to use finely spaced sets of coherent x-ray pulses to be compared after scattering from a dynamic system at different times. This enables the study of fluctuations in many-body quantum systems at the level of the ultrafast pulse durations, but this method has been limited to a select number of examples and required complex and advanced analytical tools. By applying a new methodology to this problem, we have made qualitative advances in three separate areas that will likely also find application to new fields. As compared to the “droplet-type” models, which typically are used to estimate the photon distributions on pixelated detectors to obtain the coherent x-ray speckle patterns, our algorithm achieves an order of magnitude speedup on CPU hardware and two orders of magnitude improvement on GPU hardware. We also find that it retains accuracy in low-contrast conditions, which is the typical regime for many experiments in structural dynamics. Finally, it can predict photon distributions in high average-intensity applications, a regime which up until now has not been accessible. Our artificial intelligence-assisted algorithm will enable a wider adoption of x-ray coherence spectroscopies, by both automating previously challenging analyses and enabling new experiments that were not otherwise feasible without the developments described in this work
Capturing dynamical correlations using implicit neural representations
Understanding the nature and origin of collective excitations in materials is of fundamental importance for unraveling the underlying physics of a many-body system. Excitation spectra are usually obtained by measuring the dynamical structure factor, S(Q, ω), using inelastic neutron or x-ray scattering techniques and are analyzed by comparing the experimental results against calculated predictions. We introduce a data-driven analysis tool which leverages ‘neural implicit representations’ that are specifically tailored for handling spectrographic measurements and are able to efficiently obtain unknown parameters from experimental data via automatic differentiation. In this work, we employ linear spin wave theory simulations to train a machine learning platform, enabling precise exchange parameter extraction from inelastic neutron scattering data on the square-lattice spin-1 antiferromagnet La2NiO4, showcasing a viable pathway towards automatic refinement of advanced models for ordered magnetic systems