19 research outputs found
On the Compatibility between Neural Networks and Partial Differential Equations for Physics-informed Learning
We shed light on a pitfall and an opportunity in physics-informed neural
networks (PINNs). We prove that a multilayer perceptron (MLP) only with ReLU
(Rectified Linear Unit) or ReLU-like Lipschitz activation functions will always
lead to a vanished Hessian. Such a network-imposed constraint contradicts any
second- or higher-order partial differential equations (PDEs). Therefore, a
ReLU-based MLP cannot form a permissible function space for the approximation
of their solutions. Inspired by this pitfall, we prove that a linear PDE up to
the -th order can be strictly satisfied by an MLP with activation
functions when the weights of its output layer lie on a certain hyperplane, as
called the out-layer-hyperplane. An MLP equipped with the out-layer-hyperplane
becomes "physics-enforced", no longer requiring a loss function for the PDE
itself (but only those for the initial and boundary conditions). Such a
hyperplane exists not only for MLPs but for any network architecture tailed by
a fully-connected hidden layer. To our knowledge, this should be the first PINN
architecture that enforces point-wise correctness of PDEs. We show a
closed-form expression of the out-layer-hyperplane for second-order linear
PDEs, which can be generalised to higher-order nonlinear PDEs.Comment: 12 pages, 3 figure
Zero Coordinate Shift: Whetted Automatic Differentiation for Physics-informed Operator Learning
Automatic differentiation (AD) is a critical step in physics-informed machine
learning, required for computing the high-order derivatives of network output
w.r.t. coordinates of collocation points. In this paper, we present a novel and
lightweight algorithm to conduct AD for physics-informed operator learning,
which we call the trick of Zero Coordinate Shift (ZCS). Instead of making all
sampled coordinates as leaf variables, ZCS introduces only one scalar-valued
leaf variable for each spatial or temporal dimension, simplifying the wanted
derivatives from "many-roots-many-leaves" to "one-root-many-leaves" whereby
reverse-mode AD becomes directly utilisable. It has led to an outstanding
performance leap by avoiding the duplication of the computational graph along
the dimension of functions (physical parameters). ZCS is easy to implement with
current deep learning libraries; our own implementation is achieved by
extending the DeepXDE package. We carry out a comprehensive benchmark analysis
and several case studies, training physics-informed DeepONets to solve partial
differential equations (PDEs) without data. The results show that ZCS has
persistently reduced GPU memory consumption and wall time for training by an
order of magnitude, and such reduction factor scales with the number of
functions. As a low-level optimisation technique, ZCS imposes no restrictions
on data, physics (PDE) or network architecture and does not compromise training
results from any aspect.Comment: Published in Journal of Computational Physics.
https://doi.org/10.1016/j.jcp.2024.11290
Memory-Aware Attentive Control for Community Question Answering With Knowledge-Based Dual Refinement
Benchmarking and scalability of machine-learning methods for photometric redshift estimation
Obtaining accurate photometric redshift (photo-z) estimations is an important aspect of cosmology, remaining a prerequisite of many analyses. In creating novel methods to produce photo-z estimations, there has been a shift towards using machine-learning techniques. However, there has not been as much of a focus on how well different machine-learning methods scale or perform with the ever-increasing amounts of data being produced. Here, we introduce a benchmark designed to analyse the performance and scalability of different supervised machine-learning methods for photo-z estimation. Making use of the Sloan Digital Sky Survey (SDSS – DR12) data set, we analysed a variety of the most used machine-learning algorithms. By scaling the number of galaxies used to train and test the algorithms up to one million, we obtained several metrics demonstrating the algorithms’ performance and scalability for this task. Furthermore, by introducing a new optimization method, time-considered optimization, we were able to demonstrate how a small concession of error can allow for a great improvement in efficiency. From the algorithms tested, we found that the Random Forest performed best with a mean squared error, MSE = 0.0042; however, as other algorithms such as Boosted Decision Trees and k-Nearest Neighbours performed very similarly, we used our benchmarks to demonstrate how different algorithms could be superior in different scenarios. We believe that benchmarks like this will become essential with upcoming surveys, such as the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST), which will capture billions of galaxies requiring photometric redshifts
Recommended from our members
Discovering the building blocks of dark matter halo density profiles with neural networks
The density profiles of dark matter halos are typically modeled using empirical formulas fitted to the density profiles of relaxed halo populations. We present a neural network model that is trained to learn the mapping from the raw density field containing each halo to the dark matter density profile. We show that the model recovers the widely used Navarro-Frenk-White profile out to the virial radius and can additionally describe the variability in the outer profile of the halos. The neural network architecture consists of a supervised encoder-decoder framework, which first compresses the density inputs into a low-dimensional latent representation, and then outputs for any desired value of radius . The latent representation contains all the information used by the model to predict the density profiles. This allows us to interpret the latent representation by quantifying the mutual information between the representation and the halos' ground-truth density profiles. A two-dimensional representation is sufficient to accurately model the density profiles up to the virial radius; however, a three-dimensional representation is required to describe the outer profiles beyond the virial radius. The additional dimension in the representation contains information about the infalling material in the outer profiles of dark matter halos, thus discovering the splashback boundary of halos without prior knowledge of the halos' dynamical history
Keyhole fluctuation and pore formation mechanisms during laser powder bed fusion additive manufacturing
Keyhole porosity is a key concern in laser powder-bed fusion (LPBF), potentially impacting component fatigue life. However, some keyhole porosity formation mechanisms, e.g., keyhole fluctuation, collapse and bubble growth and shrinkage, remain unclear. Using synchrotron X-ray imaging we reveal keyhole and bubble behaviour, quantifying their formation dynamics. The findings support the hypotheses that: (i) keyhole porosity can initiate not only in unstable, but also in the transition keyhole regimes created by high laser power-velocity conditions, causing fast radial keyhole fluctuations (2.5–10 kHz); (ii) transition regime collapse tends to occur part way up the rear-wall; and (iii) immediately after keyhole collapse, bubbles undergo rapid growth due to pressure equilibration, then shrink due to metal-vapour condensation. Concurrent with condensation, hydrogen diffusion into the bubble slows the shrinkage and stabilises the bubble size. The keyhole fluctuation and bubble evolution mechanisms revealed here may guide the development of control systems for minimising porosity
ESA-Ariel Data Challenge NeurIPS 2022: Inferring Physical Properties of Exoplanets From Next-Generation Telescopes
The study of extra-solar planets, or simply, exoplanets, planets outside our
own Solar System, is fundamentally a grand quest to understand our place in the
Universe. Discoveries in the last two decades have re-defined our understanding
of planets, and helped us comprehend the uniqueness of our very own Earth. In
recent years the focus has shifted from planet detection to planet
characterisation, where key planetary properties are inferred from telescope
observations using Monte Carlo-based methods. However, the efficiency of
sampling-based methodologies is put under strain by the high-resolution
observational data from next generation telescopes, such as the James Webb
Space Telescope and the Ariel Space Mission. We are delighted to announce the
acceptance of the Ariel ML Data Challenge 2022 as part of the NeurIPS
competition track. The goal of this challenge is to identify a reliable and
scalable method to perform planetary characterisation. Depending on the chosen
track, participants are tasked to provide either quartile estimates or the
approximate distribution of key planetary properties. To this end, a synthetic
spectroscopic dataset has been generated from the official simulators for the
ESA Ariel Space Mission. The aims of the competition are three-fold. 1) To
offer a challenging application for comparing and advancing conditional density
estimation methods. 2) To provide a valuable contribution towards reliable and
efficient analysis of spectroscopic data, enabling astronomers to build a
better picture of planetary demographics, and 3) To promote the interaction
between ML and exoplanetary science. The competition is open from 15th June and
will run until early October, participants of all skill levels are more than
welcomed