697 research outputs found
Data-driven deep-learning methods for the accelerated simulation of Eulerian fluid dynamics
Deep-learning (DL) methods for the fast inference of the temporal evolution of ļ¬uid-dynamics systems, based on the previous recognition of features underlying large sets of ļ¬uid-dynamics data, have been studied. Speciļ¬cally, models based on convolution neural networks (CNNs) and graph neural networks (GNNs) were proposed and discussed.
A U-Net, a popular fully-convolutional architecture, was trained to infer wave dynamics on liquid surfaces surrounded by walls, given as input the system state at previous time-points. A term for penalising the error of the spatial derivatives was added to the loss function, which resulted in a suppression of spurious oscillations and a more accurate location and length of the
predicted wavefronts. This model proved to accurately generalise to complex wall geometries not seen during training.
As opposed to the image data-structures processed by CNNs, graphs oļ¬er higher freedom on how data is organised and processed. This motivated the use of graphs to represent the state of ļ¬uid-dynamic systems discretised by unstructured sets of nodes, and GNNs to process such graphs. Graphs have enabled more accurate representations of curvilinear geometries and higher resolution placement exclusively in areas where physics is more challenging to resolve. Two novel
GNN architectures were designed for ļ¬uid-dynamics inference: the MuS-GNN, a multi-scale GNN, and the REMuS-GNN, a rotation-equivariant multi-scale GNN. Both architectures work by repeatedly passing messages from each node to its nearest nodes in the graph. Additionally, lower-resolutions graphs, with a reduced number of nodes, are deļ¬ned from the original graph,
and messages are also passed from ļ¬ner to coarser graphs and vice-versa. The low-resolution graphs allowed for eļ¬ciently capturing physics encompassing a range of lengthscales.
Advection and ļ¬uid ļ¬ow, modelled by the incompressible Navier-Stokes equations, were the two types of problems used to assess the proposed GNNs. Whereas a single-scale GNN was suļ¬cient to achieve high generalisation accuracy in advection simulations, ļ¬ow simulation highly beneļ¬ted from an increasing number of low-resolution graphs. The generalisation and long-term accuracy of these simulations were further improved by the REMuS-GNN architecture, which
processes the system state independently of the orientation of the coordinate system thanks to a rotation-invariant representation and carefully designed components. To the best of the authorās knowledge, the REMuS-GNN architecture was the ļ¬rst rotation-equivariant and multi-scale GNN.
The simulations were accelerated between one (in a CPU) and three (in a GPU) orders of magnitude with respect to a CPU-based numerical solver. Additionally, the parallelisation of multi-scale GNNs resulted in a close-to-linear speedup with the number of CPU cores or GPUs.Open Acces
Investigating the metabolomics of treatment response in patients with inflammatory rheumatic diseases
Background:
Rheumatic and musculoskeletal diseases (RMDs) are autoimmune-mediated chronic diseases affecting the joints around the body, involving an inappropriate immune response being launched against the tissues of the joint. These devastating diseases include rheumatoid arthritis (RA) and psoriatic arthritis (PsA). If insufficiently managed ā or indeed in severe cases ā these diseases can substantially impact a patientās quality of life, leading to joint damage, dysfunction, and disability. However, numerous treatments exist for these diseases that control the immune-mediated factors driving disease, described as disease modifying anti-rheumatic drugs (DMARDs). Despite the success of these drugs for patients in achieving remission, they are not effective in all patients, and those who do not respond well to first-line treatments will typically be given an alternative drug on a trial-and-error basis until they respond successfully. Given the rapid and irreversible damage these diseases can induce even in the early stages, the need for early and aggressive treatment is fundamental for reaching a good outcome for the patient. Biomarkers can be employed to identify the most suitable drug to administer on a patient-to-patient basis, using these to predict who will respond to which drug. Incorporating biomarkers into the clinical management of these diseases is expected to be fundamental for precision medicine. These may come from multiple molecular sources. For example, currently used biomarkers include autoantibodies while this project primarily focuses on discovering biomarkers from the metabolome.
Methodology:
This project involved the secondary analyses of metabolomic and transcriptomic datasets generated from patients enrolled on multiple clinical studies. These include data from the Targeting Synovitis in Early Rheumatoid Arthritis (TaSER) (n=72), Treatment in the Rotterdam Early Arthritis Cohort (tREACH) (n=82), Characterising the Centralised Pain Phenotype in Chronic Rheumatic Disease (CENTAUR) (n=50) and Mayo Clinic - Hur et al. (2021) (n=64) ā cohorts. The metabolic findings' translatability across cohorts was evaluated by incorporating datasets from various regions, including the United Kingdom, the Netherlands, and the United States of America.
These multi-omic datasets were analysed using an in-house workflow developed throughout this projectās duration, involving the use of the R environment to perform exploratory data analysis, supervised machine learning and an investigation of the biological relevance of the findings. Other methods were also employed, notably an exploration and evaluation of data integration methods.
Supervised machine learning was included to generate molecular profiles of treatment responses from multiple datasets. Doing so showed the value of combining multiple weakly-associated analytes in a model that could predict patient responses. However, an important component, the validation of these models, could not be performed in this work, although suggestions were made throughout of possible next steps.
Results and Discussion:
The analysis of the TaSER metabolomic data showed metabolites associated with methotrexate response after 3 months of treatment. Tryptophan and argininerelated metabolites were included in the metabolic model predictive of the 3-month response. While the model was not directly validated using subsequent datasets, including the tREACH and Mayo Clinic cohorts, additional features from these pathways were associated with treatment response. Included across cohorts were several tryptophan metabolites, including those derived from indole. Since these are largely produced via the gut microbiome it was suggested that the gut microbiome may influence the effectiveness of RMD treatments. Since RA and PsA were considered in this work as two archetypal RMDs, part of the project intended to investigate whether there were shared metabolic features found in association to treatment response in both diseases. These common metabolites were not clearly identified, although arginine-related metabolites were observed in models generated from the TaSER and CENTAUR cohorts in association with response to treatment in both conditions.
Owing to the limitations of the untargeted metabolomic approach, this work was expected to provide an initial step in understanding the involvement of arginine and tryptophan related pathways in influencing treatment response in RMDs. Not performed in this work, it was expected that targeted metabolomics would provide clearer insights into these metabolites, providing absolute quantification with the identification of these features of interest in the patient samples. It was expected that expanding the cohort sizes and incorporating other omics platforms would provide a greater understanding of the mechanisms of the resolution of RMDs and inform future therapeutic targets.
An important output from this project was the analytical pipeline developed and employed throughout for the omics analysis to inform biomarker discovery. Later work will involve generating a package in the R environment called markerHuntR. The R scripts for the functions with example datasets can be found at https://github.com/cambest202/markerHuntR.git. It is anticipated that the package will soon be described in more detail in a publication. The package will be available for researchers familiar with R to perform similar analyses as those described in this work
Discovering Causal Relations and Equations from Data
Physics is a field of science that has traditionally used the scientific
method to answer questions about why natural phenomena occur and to make
testable models that explain the phenomena. Discovering equations, laws and
principles that are invariant, robust and causal explanations of the world has
been fundamental in physical sciences throughout the centuries. Discoveries
emerge from observing the world and, when possible, performing interventional
studies in the system under study. With the advent of big data and the use of
data-driven methods, causal and equation discovery fields have grown and made
progress in computer science, physics, statistics, philosophy, and many applied
fields. All these domains are intertwined and can be used to discover causal
relations, physical laws, and equations from observational data. This paper
reviews the concepts, methods, and relevant works on causal and equation
discovery in the broad field of Physics and outlines the most important
challenges and promising future lines of research. We also provide a taxonomy
for observational causal and equation discovery, point out connections, and
showcase a complete set of case studies in Earth and climate sciences, fluid
dynamics and mechanics, and the neurosciences. This review demonstrates that
discovering fundamental laws and causal relations by observing natural
phenomena is being revolutionised with the efficient exploitation of
observational data, modern machine learning algorithms and the interaction with
domain knowledge. Exciting times are ahead with many challenges and
opportunities to improve our understanding of complex systems.Comment: 137 page
What's in a Prior? Learned Proximal Networks for Inverse Problems
Proximal operators are ubiquitous in inverse problems, commonly appearing as
part of algorithmic strategies to regularize problems that are otherwise
ill-posed. Modern deep learning models have been brought to bear for these
tasks too, as in the framework of plug-and-play or deep unrolling, where they
loosely resemble proximal operators. Yet, something essential is lost in
employing these purely data-driven approaches: there is no guarantee that a
general deep network represents the proximal operator of any function, nor is
there any characterization of the function for which the network might provide
some approximate proximal. This not only makes guaranteeing convergence of
iterative schemes challenging but, more fundamentally, complicates the analysis
of what has been learned by these networks about their training data. Herein we
provide a framework to develop learned proximal networks (LPN), prove that they
provide exact proximal operators for a data-driven nonconvex regularizer, and
show how a new training strategy, dubbed proximal matching, provably promotes
the recovery of the log-prior of the true data distribution. Such LPN provide
general, unsupervised, expressive proximal operators that can be used for
general inverse problems with convergence guarantees. We illustrate our results
in a series of cases of increasing complexity, demonstrating that these models
not only result in state-of-the-art performance, but provide a window into the
resulting priors learned from data
Neural Network Approximation of Continuous Functions in High Dimensions with Applications to Inverse Problems
The remarkable successes of neural networks in a huge variety of inverse
problems have fueled their adoption in disciplines ranging from medical imaging
to seismic analysis over the past decade. However, the high dimensionality of
such inverse problems has simultaneously left current theory, which predicts
that networks should scale exponentially in the dimension of the problem,
unable to explain why the seemingly small networks used in these settings work
as well as they do in practice. To reduce this gap between theory and practice,
we provide a general method for bounding the complexity required for a neural
network to approximate a H\"older (or uniformly) continuous function defined on
a high-dimensional set with a low-complexity structure. The approach is based
on the observation that the existence of a Johnson-Lindenstrauss embedding
of a given high-dimensional set
into a low dimensional cube implies that for
any H\"older (or uniformly) continuous function , there
exists a H\"older (or uniformly) continuous function
such that for all . Hence, if
one has a neural network which approximates , then a
layer can be added that implements the JL embedding to obtain a neural
network that approximates . By pairing JL embedding results
along with results on approximation of H\"older (or uniformly) continuous
functions by neural networks, one then obtains results which bound the
complexity required for a neural network to approximate H\"older (or uniformly)
continuous functions on high dimensional sets. The end result is a general
theoretical framework which can then be used to better explain the observed
empirical successes of smaller networks in a wider variety of inverse problems
than current theory allows.Comment: 26 pages, 1 figur
Conditional gradients for total variation regularization with PDE constraints: a graph cuts approach
Total variation regularization has proven to be a valuable tool in the
context of optimal control of differential equations. This is particularly
attributed to the observation that TV-penalties often favor piecewise constant
minimizers with well-behaved jumpsets. On the downside, their intricate
properties significantly complicate every aspect of their analysis, from the
derivation of first-order optimality conditions to their discrete approximation
and the choice of a suitable solution algorithm. In this paper, we investigate
a general class of minimization problems with TV-regularization, comprising
both continuous and discretized control spaces, from a convex geometry
perspective. This leads to a variety of novel theoretical insights on
minimization problems with total variation regularization as well as tools for
their practical realization. First, by studying the extremal points of the
respective total variation unit balls, we enable their efficient solution by
geometry exploiting algorithms, e.g. fully-corrective generalized conditional
gradient methods. We give a detailed account on the practical realization of
such a method for piecewise constant finite element approximations of the
control on triangulations of the spatial domain. Second, in the same setting
and for suitable sequences of uniformly refined meshes, it is shown that
minimizers to discretized PDE-constrained optimal control problems approximate
solutions to a continuous limit problem involving an anisotropic total
variation reflecting the fine-scale geometry of the mesh.Comment: 47 pages, 12 figure
Mathematics of biomedical imaging todayāa perspective
Biomedical imaging is a fascinating, rich and dynamic research area, which has huge importance in biomedical research and clinical practice alike. The key technology behind the processing, and automated analysis and quantification of imaging data is mathematics. Starting with the optimisation of the image acquisition and the reconstruction of an image from indirect tomographic measurement data, all the way to the automated segmentation of tumours in medical images and the design of optimal treatment plans based on image biomarkers, mathematics appears in all of these in different flavours. Non-smooth optimisation in the context of sparsity-promoting image priors, partial differential equations for image registration and motion estimation, and deep neural networks for image segmentation, to name just a few. In this article, we present and review mathematical topics that arise within the whole biomedical imaging pipeline, from tomographic measurements to clinical support tools, and highlight some modern topics and open problems. The article is addressed to both biomedical researchers who want to get a taste of where mathematics arises in biomedical imaging as well as mathematicians who are interested in what mathematical challenges biomedical imaging research entails
Point spread function approximation of high rank Hessians with locally supported non-negative integral kernels
We present an efficient matrix-free point spread function (PSF) method for
approximating operators that have locally supported non-negative integral
kernels. The method computes impulse responses of the operator at scattered
points, and interpolates these impulse responses to approximate integral kernel
entries. Impulse responses are computed by applying the operator to Dirac comb
batches of point sources, which are chosen by solving an ellipsoid packing
problem. Evaluation of kernel entries allows us to construct a hierarchical
matrix (H-matrix) approximation of the operator. Further matrix computations
are performed with H-matrix methods. We use the method to build preconditioners
for the Hessian operator in two inverse problems governed by partial
differential equations (PDEs): inversion for the basal friction coefficient in
an ice sheet flow problem and for the initial condition in an
advective-diffusive transport problem. While for many ill-posed inverse
problems the Hessian of the data misfit term exhibits a low rank structure, and
hence a low rank approximation is suitable, for many problems of practical
interest the numerical rank of the Hessian is still large. But Hessian impulse
responses typically become more local as the numerical rank increases, which
benefits the PSF method. Numerical results reveal that the PSF preconditioner
clusters the spectrum of the preconditioned Hessian near one, yielding roughly
5x-10x reductions in the required number of PDE solves, as compared to
regularization preconditioning and no preconditioning. We also present a
numerical study for the influence of various parameters (that control the shape
of the impulse responses) on the effectiveness of the advection-diffusion
Hessian approximation. The results show that the PSF-based preconditioners are
able to form good approximations of high-rank Hessians using a small number of
operator applications
An hpāadaptive multiāelement stochastic collocation method for surrogate modeling with information reāuse
This article introduces an hpāadaptive multiāelement stochastic collocation method, which additionally allows to reāuse existing model evaluations during either hā or pārefinement. The collocation method is based on weighted Leja nodes. After hārefinement, local interpolations are stabilized by adding and sorting Leja nodes on each newly created subāelement in a hierarchical manner. For pārefinement, the local polynomial approximations are based on totalādegree or dimensionāadaptive bases. The method is applied in the context of forward and inverse uncertainty quantification to handle nonāsmooth or strongly localized response surfaces. The performance of the proposed method is assessed in several test cases, also in comparison to competing methods
- ā¦