697 research outputs found

    Data-driven deep-learning methods for the accelerated simulation of Eulerian fluid dynamics

    Get PDF
    Deep-learning (DL) methods for the fast inference of the temporal evolution of ļ¬‚uid-dynamics systems, based on the previous recognition of features underlying large sets of ļ¬‚uid-dynamics data, have been studied. Speciļ¬cally, models based on convolution neural networks (CNNs) and graph neural networks (GNNs) were proposed and discussed. A U-Net, a popular fully-convolutional architecture, was trained to infer wave dynamics on liquid surfaces surrounded by walls, given as input the system state at previous time-points. A term for penalising the error of the spatial derivatives was added to the loss function, which resulted in a suppression of spurious oscillations and a more accurate location and length of the predicted wavefronts. This model proved to accurately generalise to complex wall geometries not seen during training. As opposed to the image data-structures processed by CNNs, graphs oļ¬€er higher freedom on how data is organised and processed. This motivated the use of graphs to represent the state of ļ¬‚uid-dynamic systems discretised by unstructured sets of nodes, and GNNs to process such graphs. Graphs have enabled more accurate representations of curvilinear geometries and higher resolution placement exclusively in areas where physics is more challenging to resolve. Two novel GNN architectures were designed for ļ¬‚uid-dynamics inference: the MuS-GNN, a multi-scale GNN, and the REMuS-GNN, a rotation-equivariant multi-scale GNN. Both architectures work by repeatedly passing messages from each node to its nearest nodes in the graph. Additionally, lower-resolutions graphs, with a reduced number of nodes, are deļ¬ned from the original graph, and messages are also passed from ļ¬ner to coarser graphs and vice-versa. The low-resolution graphs allowed for eļ¬ƒciently capturing physics encompassing a range of lengthscales. Advection and ļ¬‚uid ļ¬‚ow, modelled by the incompressible Navier-Stokes equations, were the two types of problems used to assess the proposed GNNs. Whereas a single-scale GNN was suļ¬ƒcient to achieve high generalisation accuracy in advection simulations, ļ¬‚ow simulation highly beneļ¬ted from an increasing number of low-resolution graphs. The generalisation and long-term accuracy of these simulations were further improved by the REMuS-GNN architecture, which processes the system state independently of the orientation of the coordinate system thanks to a rotation-invariant representation and carefully designed components. To the best of the authorā€™s knowledge, the REMuS-GNN architecture was the ļ¬rst rotation-equivariant and multi-scale GNN. The simulations were accelerated between one (in a CPU) and three (in a GPU) orders of magnitude with respect to a CPU-based numerical solver. Additionally, the parallelisation of multi-scale GNNs resulted in a close-to-linear speedup with the number of CPU cores or GPUs.Open Acces

    Investigating the metabolomics of treatment response in patients with inflammatory rheumatic diseases

    Get PDF
    Background: Rheumatic and musculoskeletal diseases (RMDs) are autoimmune-mediated chronic diseases affecting the joints around the body, involving an inappropriate immune response being launched against the tissues of the joint. These devastating diseases include rheumatoid arthritis (RA) and psoriatic arthritis (PsA). If insufficiently managed ā€“ or indeed in severe cases ā€“ these diseases can substantially impact a patientā€™s quality of life, leading to joint damage, dysfunction, and disability. However, numerous treatments exist for these diseases that control the immune-mediated factors driving disease, described as disease modifying anti-rheumatic drugs (DMARDs). Despite the success of these drugs for patients in achieving remission, they are not effective in all patients, and those who do not respond well to first-line treatments will typically be given an alternative drug on a trial-and-error basis until they respond successfully. Given the rapid and irreversible damage these diseases can induce even in the early stages, the need for early and aggressive treatment is fundamental for reaching a good outcome for the patient. Biomarkers can be employed to identify the most suitable drug to administer on a patient-to-patient basis, using these to predict who will respond to which drug. Incorporating biomarkers into the clinical management of these diseases is expected to be fundamental for precision medicine. These may come from multiple molecular sources. For example, currently used biomarkers include autoantibodies while this project primarily focuses on discovering biomarkers from the metabolome. Methodology: This project involved the secondary analyses of metabolomic and transcriptomic datasets generated from patients enrolled on multiple clinical studies. These include data from the Targeting Synovitis in Early Rheumatoid Arthritis (TaSER) (n=72), Treatment in the Rotterdam Early Arthritis Cohort (tREACH) (n=82), Characterising the Centralised Pain Phenotype in Chronic Rheumatic Disease (CENTAUR) (n=50) and Mayo Clinic - Hur et al. (2021) (n=64) ā€“ cohorts. The metabolic findings' translatability across cohorts was evaluated by incorporating datasets from various regions, including the United Kingdom, the Netherlands, and the United States of America. These multi-omic datasets were analysed using an in-house workflow developed throughout this projectā€™s duration, involving the use of the R environment to perform exploratory data analysis, supervised machine learning and an investigation of the biological relevance of the findings. Other methods were also employed, notably an exploration and evaluation of data integration methods. Supervised machine learning was included to generate molecular profiles of treatment responses from multiple datasets. Doing so showed the value of combining multiple weakly-associated analytes in a model that could predict patient responses. However, an important component, the validation of these models, could not be performed in this work, although suggestions were made throughout of possible next steps. Results and Discussion: The analysis of the TaSER metabolomic data showed metabolites associated with methotrexate response after 3 months of treatment. Tryptophan and argininerelated metabolites were included in the metabolic model predictive of the 3-month response. While the model was not directly validated using subsequent datasets, including the tREACH and Mayo Clinic cohorts, additional features from these pathways were associated with treatment response. Included across cohorts were several tryptophan metabolites, including those derived from indole. Since these are largely produced via the gut microbiome it was suggested that the gut microbiome may influence the effectiveness of RMD treatments. Since RA and PsA were considered in this work as two archetypal RMDs, part of the project intended to investigate whether there were shared metabolic features found in association to treatment response in both diseases. These common metabolites were not clearly identified, although arginine-related metabolites were observed in models generated from the TaSER and CENTAUR cohorts in association with response to treatment in both conditions. Owing to the limitations of the untargeted metabolomic approach, this work was expected to provide an initial step in understanding the involvement of arginine and tryptophan related pathways in influencing treatment response in RMDs. Not performed in this work, it was expected that targeted metabolomics would provide clearer insights into these metabolites, providing absolute quantification with the identification of these features of interest in the patient samples. It was expected that expanding the cohort sizes and incorporating other omics platforms would provide a greater understanding of the mechanisms of the resolution of RMDs and inform future therapeutic targets. An important output from this project was the analytical pipeline developed and employed throughout for the omics analysis to inform biomarker discovery. Later work will involve generating a package in the R environment called markerHuntR. The R scripts for the functions with example datasets can be found at https://github.com/cambest202/markerHuntR.git. It is anticipated that the package will soon be described in more detail in a publication. The package will be available for researchers familiar with R to perform similar analyses as those described in this work

    Discovering Causal Relations and Equations from Data

    Full text link
    Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws and principles that are invariant, robust and causal explanations of the world has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing the world and, when possible, performing interventional studies in the system under study. With the advent of big data and the use of data-driven methods, causal and equation discovery fields have grown and made progress in computer science, physics, statistics, philosophy, and many applied fields. All these domains are intertwined and can be used to discover causal relations, physical laws, and equations from observational data. This paper reviews the concepts, methods, and relevant works on causal and equation discovery in the broad field of Physics and outlines the most important challenges and promising future lines of research. We also provide a taxonomy for observational causal and equation discovery, point out connections, and showcase a complete set of case studies in Earth and climate sciences, fluid dynamics and mechanics, and the neurosciences. This review demonstrates that discovering fundamental laws and causal relations by observing natural phenomena is being revolutionised with the efficient exploitation of observational data, modern machine learning algorithms and the interaction with domain knowledge. Exciting times are ahead with many challenges and opportunities to improve our understanding of complex systems.Comment: 137 page

    What's in a Prior? Learned Proximal Networks for Inverse Problems

    Full text link
    Proximal operators are ubiquitous in inverse problems, commonly appearing as part of algorithmic strategies to regularize problems that are otherwise ill-posed. Modern deep learning models have been brought to bear for these tasks too, as in the framework of plug-and-play or deep unrolling, where they loosely resemble proximal operators. Yet, something essential is lost in employing these purely data-driven approaches: there is no guarantee that a general deep network represents the proximal operator of any function, nor is there any characterization of the function for which the network might provide some approximate proximal. This not only makes guaranteeing convergence of iterative schemes challenging but, more fundamentally, complicates the analysis of what has been learned by these networks about their training data. Herein we provide a framework to develop learned proximal networks (LPN), prove that they provide exact proximal operators for a data-driven nonconvex regularizer, and show how a new training strategy, dubbed proximal matching, provably promotes the recovery of the log-prior of the true data distribution. Such LPN provide general, unsupervised, expressive proximal operators that can be used for general inverse problems with convergence guarantees. We illustrate our results in a series of cases of increasing complexity, demonstrating that these models not only result in state-of-the-art performance, but provide a window into the resulting priors learned from data

    Neural Network Approximation of Continuous Functions in High Dimensions with Applications to Inverse Problems

    Full text link
    The remarkable successes of neural networks in a huge variety of inverse problems have fueled their adoption in disciplines ranging from medical imaging to seismic analysis over the past decade. However, the high dimensionality of such inverse problems has simultaneously left current theory, which predicts that networks should scale exponentially in the dimension of the problem, unable to explain why the seemingly small networks used in these settings work as well as they do in practice. To reduce this gap between theory and practice, we provide a general method for bounding the complexity required for a neural network to approximate a H\"older (or uniformly) continuous function defined on a high-dimensional set with a low-complexity structure. The approach is based on the observation that the existence of a Johnson-Lindenstrauss embedding AāˆˆRdƗDA\in\mathbb{R}^{d\times D} of a given high-dimensional set SāŠ‚RDS\subset\mathbb{R}^D into a low dimensional cube [āˆ’M,M]d[-M,M]^d implies that for any H\"older (or uniformly) continuous function f:Sā†’Rpf:S\to\mathbb{R}^p, there exists a H\"older (or uniformly) continuous function g:[āˆ’M,M]dā†’Rpg:[-M,M]^d\to\mathbb{R}^p such that g(Ax)=f(x)g(Ax)=f(x) for all xāˆˆSx\in S. Hence, if one has a neural network which approximates g:[āˆ’M,M]dā†’Rpg:[-M,M]^d\to\mathbb{R}^p, then a layer can be added that implements the JL embedding AA to obtain a neural network that approximates f:Sā†’Rpf:S\to\mathbb{R}^p. By pairing JL embedding results along with results on approximation of H\"older (or uniformly) continuous functions by neural networks, one then obtains results which bound the complexity required for a neural network to approximate H\"older (or uniformly) continuous functions on high dimensional sets. The end result is a general theoretical framework which can then be used to better explain the observed empirical successes of smaller networks in a wider variety of inverse problems than current theory allows.Comment: 26 pages, 1 figur

    Conditional gradients for total variation regularization with PDE constraints: a graph cuts approach

    Full text link
    Total variation regularization has proven to be a valuable tool in the context of optimal control of differential equations. This is particularly attributed to the observation that TV-penalties often favor piecewise constant minimizers with well-behaved jumpsets. On the downside, their intricate properties significantly complicate every aspect of their analysis, from the derivation of first-order optimality conditions to their discrete approximation and the choice of a suitable solution algorithm. In this paper, we investigate a general class of minimization problems with TV-regularization, comprising both continuous and discretized control spaces, from a convex geometry perspective. This leads to a variety of novel theoretical insights on minimization problems with total variation regularization as well as tools for their practical realization. First, by studying the extremal points of the respective total variation unit balls, we enable their efficient solution by geometry exploiting algorithms, e.g. fully-corrective generalized conditional gradient methods. We give a detailed account on the practical realization of such a method for piecewise constant finite element approximations of the control on triangulations of the spatial domain. Second, in the same setting and for suitable sequences of uniformly refined meshes, it is shown that minimizers to discretized PDE-constrained optimal control problems approximate solutions to a continuous limit problem involving an anisotropic total variation reflecting the fine-scale geometry of the mesh.Comment: 47 pages, 12 figure

    Mathematics of biomedical imaging todayā€”a perspective

    Get PDF
    Biomedical imaging is a fascinating, rich and dynamic research area, which has huge importance in biomedical research and clinical practice alike. The key technology behind the processing, and automated analysis and quantification of imaging data is mathematics. Starting with the optimisation of the image acquisition and the reconstruction of an image from indirect tomographic measurement data, all the way to the automated segmentation of tumours in medical images and the design of optimal treatment plans based on image biomarkers, mathematics appears in all of these in different flavours. Non-smooth optimisation in the context of sparsity-promoting image priors, partial differential equations for image registration and motion estimation, and deep neural networks for image segmentation, to name just a few. In this article, we present and review mathematical topics that arise within the whole biomedical imaging pipeline, from tomographic measurements to clinical support tools, and highlight some modern topics and open problems. The article is addressed to both biomedical researchers who want to get a taste of where mathematics arises in biomedical imaging as well as mathematicians who are interested in what mathematical challenges biomedical imaging research entails

    Point spread function approximation of high rank Hessians with locally supported non-negative integral kernels

    Full text link
    We present an efficient matrix-free point spread function (PSF) method for approximating operators that have locally supported non-negative integral kernels. The method computes impulse responses of the operator at scattered points, and interpolates these impulse responses to approximate integral kernel entries. Impulse responses are computed by applying the operator to Dirac comb batches of point sources, which are chosen by solving an ellipsoid packing problem. Evaluation of kernel entries allows us to construct a hierarchical matrix (H-matrix) approximation of the operator. Further matrix computations are performed with H-matrix methods. We use the method to build preconditioners for the Hessian operator in two inverse problems governed by partial differential equations (PDEs): inversion for the basal friction coefficient in an ice sheet flow problem and for the initial condition in an advective-diffusive transport problem. While for many ill-posed inverse problems the Hessian of the data misfit term exhibits a low rank structure, and hence a low rank approximation is suitable, for many problems of practical interest the numerical rank of the Hessian is still large. But Hessian impulse responses typically become more local as the numerical rank increases, which benefits the PSF method. Numerical results reveal that the PSF preconditioner clusters the spectrum of the preconditioned Hessian near one, yielding roughly 5x-10x reductions in the required number of PDE solves, as compared to regularization preconditioning and no preconditioning. We also present a numerical study for the influence of various parameters (that control the shape of the impulse responses) on the effectiveness of the advection-diffusion Hessian approximation. The results show that the PSF-based preconditioners are able to form good approximations of high-rank Hessians using a small number of operator applications

    An hpā€adaptive multiā€element stochastic collocation method for surrogate modeling with information reā€use

    Get PDF
    This article introduces an hpā€adaptive multiā€element stochastic collocation method, which additionally allows to reā€use existing model evaluations during either hā€ or pā€refinement. The collocation method is based on weighted Leja nodes. After hā€refinement, local interpolations are stabilized by adding and sorting Leja nodes on each newly created subā€element in a hierarchical manner. For pā€refinement, the local polynomial approximations are based on totalā€degree or dimensionā€adaptive bases. The method is applied in the context of forward and inverse uncertainty quantification to handle nonā€smooth or strongly localized response surfaces. The performance of the proposed method is assessed in several test cases, also in comparison to competing methods
    • ā€¦
    corecore