6 research outputs found
DiTTO: Diffusion-inspired Temporal Transformer Operator
Solving partial differential equations (PDEs) using a data-driven approach
has become increasingly common. The recent development of the operator learning
paradigm has enabled the solution of a broader range of PDE-related problems.
We propose an operator learning method to solve time-dependent PDEs
continuously in time without needing any temporal discretization. The proposed
approach, named DiTTO, is inspired by latent diffusion models. While diffusion
models are usually used in generative artificial intelligence tasks, their
time-conditioning mechanism is extremely useful for PDEs. The
diffusion-inspired framework is combined with elements from the Transformer
architecture to improve its capabilities.
We demonstrate the effectiveness of the new approach on a wide variety of
PDEs in multiple dimensions, namely the 1-D Burgers' equation, 2-D
Navier-Stokes equations, and the acoustic wave equation in 2-D and 3-D. DiTTO
achieves state-of-the-art results in terms of accuracy for these problems. We
also present a method to improve the performance of DiTTO by using fast
sampling concepts from diffusion models. Finally, we show that DiTTO can
accurately perform zero-shot super-resolution in time
CrunchGPT: A chatGPT assisted framework for scientific machine learning
Scientific Machine Learning (SciML) has advanced recently across many
different areas in computational science and engineering. The objective is to
integrate data and physics seamlessly without the need of employing elaborate
and computationally taxing data assimilation schemes. However, preprocessing,
problem formulation, code generation, postprocessing and analysis are still
time consuming and may prevent SciML from wide applicability in industrial
applications and in digital twin frameworks. Here, we integrate the various
stages of SciML under the umbrella of ChatGPT, to formulate CrunchGPT, which
plays the role of a conductor orchestrating the entire workflow of SciML based
on simple prompts by the user. Specifically, we present two examples that
demonstrate the potential use of CrunchGPT in optimizing airfoils in
aerodynamics, and in obtaining flow fields in various geometries in interactive
mode, with emphasis on the validation stage. To demonstrate the flow of the
CrunchGPT, and create an infrastructure that can facilitate a broader vision,
we built a webapp based guided user interface, that includes options for a
comprehensive summary report. The overall objective is to extend CrunchGPT to
handle diverse problems in computational mechanics, design, optimization and
controls, and general scientific computing tasks involved in SciML, hence using
it as a research assistant tool but also as an educational tool. While here the
examples focus in fluid mechanics, future versions will target solid mechanics
and materials science, geophysics, systems biology and bioinformatics.Comment: 20 pages, 26 figure
Understanding the Efficacy of U-Net & Vision Transformer for Groundwater Numerical Modelling
This paper presents a comprehensive comparison of various machine learning
models, namely U-Net, U-Net integrated with Vision Transformers (ViT), and
Fourier Neural Operator (FNO), for time-dependent forward modelling in
groundwater systems. Through testing on synthetic datasets, it is demonstrated
that U-Net and U-Net + ViT models outperform FNO in accuracy and efficiency,
especially in sparse data scenarios. These findings underscore the potential of
U-Net-based models for groundwater modelling in real-world applications where
data scarcity is prevalent
Rethinking skip connections in Spiking Neural Networks with Time-To-First-Spike coding
Time-To-First-Spike (TTFS) coding in Spiking Neural Networks (SNNs) offers significant advantages in terms of energy efficiency, closely mimicking the behavior of biological neurons. In this work, we delve into the role of skip connections, a widely used concept in Artificial Neural Networks (ANNs), within the domain of SNNs with TTFS coding. Our focus is on two distinct types of skip connection architectures: (1) addition-based skip connections, and (2) concatenation-based skip connections. We find that addition-based skip connections introduce an additional delay in terms of spike timing. On the other hand, concatenation-based skip connections circumvent this delay but produce time gaps between after-convolution and skip connection paths, thereby restricting the effective mixing of information from these two paths. To mitigate these issues, we propose a novel approach involving a learnable delay for skip connections in the concatenation-based skip connection architecture. This approach successfully bridges the time gap between the convolutional and skip branches, facilitating improved information mixing. We conduct experiments on public datasets including MNIST and Fashion-MNIST, illustrating the advantage of the skip connection in TTFS coding architectures. Additionally, we demonstrate the applicability of TTFS coding on beyond image recognition tasks and extend it to scientific machine-learning tasks, broadening the potential uses of SNNs
A Convolutional Dispersion Relation Preserving Scheme for the Acoustic Wave Equation
We propose an accurate numerical scheme for approximating the solution of the
two dimensional acoustic wave problem. We use machine learning to find a
stencil suitable even in the presence of high wavenumbers. The proposed scheme
incorporates physically informed elements from the field of optimized numerical
schemes into a convolutional optimization machine learning algorithm