Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation

Across a variety of scientific disciplines, sparse inverse covariance estimation is a popular tool for capturing the underlying dependency relationships in multivariate data. Unfortunately, most estimators are not scalable enough to handle the sizes of modern high-dimensional data sets (often on the order of terabytes), and assume Gaussian samples. To address these deficiencies, we introduce HP-CONCORD, a highly scalable optimization method for estimating a sparse inverse covariance matrix based on a regularized pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal gradient method uses a novel communication-avoiding linear algebra algorithm and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving parallel scalability on problems with up to ~819 billion parameters (1.28 million dimensions); even on a single node, HP-CONCORD demonstrates scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to estimate the underlying dependency structure of the brain from fMRI data, and use the result to identify functional regions automatically. The results show good agreement with a clustering from the neuroscience literature.Comment: Main paper: 15 pages, appendix: 24 page

Deep Learning for Inverting Borehole Resistivity Measurements.

139 p.El subsuelo terrestre está formado por diferentes materiales, principalmente por rocas porosas que posiblemente contienen minerales y están rellenas de agua salada y/o hidrocarburos. Por lo general, las formaciones que crean estos materiales son irregulares y con materiales de diferentes propiedades mezclados en el mismo estrato.Uno de los principales objetivos en geofísica es determinar las propiedades petrofísicas del subsuelo de la Tierra. De este modo, las compañías pueden determinar la localización de las reservas de hidrocarburos para maximizar su producción o descubrir localizaciones óptimas para el almacenamiento de hidrógeno o el depósito de CO$_2$. Para este propósito, las compañías registran mediciones electromagnéticas utilizando herramientas de Medición Durante Perforación (LWD por sus siglas en inglés -- Logging While Drilling), las cuales son capaces de recabar datos mientras se lleva a cabo el proceso de prospección. Los datos obtenidos se procesan para producir un mapa del subsuelo de la Tierra. Basándose en el mapa generado, el operador ajusta en tiempo real la trayectoria de la herramienta de prospección para seguir explorando objetivos de explotación, incluidos los yacimientos de petróleo y gas, y maximizar la posterior productividad de las reservas disponibles. Esta técnica de ajuste en tiempo real se denomina geo-navegación.Hoy en día, la geo-navegación desempeña un papel esencial en geofísica. Sin embargo, requiere la resolución de problemas inversos en tiempo real. Esto supone un reto, ya que los problemas inversos suelen estar mal planteados.Existen múltiples métodos tradicionales para resolver los problemas inversos, principalmente, los métodos basados en el gradiente o en la estadística. Sin embargo, estos métodos tienen graves limitaciones. En particular, a menudo necesitan calcular el problema inverso cientos de veces para cada conjunto de mediciones, lo que es computacionalmente caro en problemas tridimensionales (3D).Para superar estas limitaciones, proponemos el uso de técnicas de Aprendizaje Profundo (DL por sus siglas en inglés -- Deep Learning) para resolver los problemas inversos. Aunque la etapa de entrenamiento de una Red Neuronal Profunda (DNN por sus siglas en inglés Deep Neural Network) puede requerir mucho tiempo, una vez que la red está correctamente entrenada puede predecir la solución en una fracción de segundo, facilitando las operaciones de geo-navegación en tiempo real. En la primera parte de esta tesis, investigamos las funciones de pérdida apropiadas para entrenar una DNN cuando se trata de un problema inverso.Además, para entrenar adecuadamente una DNN que se aproxime a la solución inversa, necesitamos un gran conjunto de datos que contenga la solución del problema directo para muchos modelos terrestres diferentes. Para crear dicho conjunto de datos, necesitamos resolver una Ecuación en Derivadas Parciales (PDE por sus siglas en inglés -- Partial Differential Equation) miles de veces. La creación de un conjunto de datos puede llevar mucho tiempo, especialmente para los problemas bidimensionales y tridimensionales, ya que la resolución de la PDE mediante métodos tradicionales, como el Método de Elementos Finitos (FEM por sus siglas en inglés -- Finite Element Method), es computacionalmente caro. Por lo tanto, queremos reducir el coste computacional de la construcción de la base de datos necesaria para entrenar la DNN. Para ello, proponemos el uso de métodos de Análisis Isogeométrico refinado (rIGA por sus siglas en inglés -- refined Isogeometric Analysis).Además, exploramos la posibilidad de utilizar técnicas de DL para resolver PDE, que es la limitación computacional principal al resolver problemas inversos. Nuestro objetivo principal es desarrollar un simulador rápido para resolver PDE paramétricas. Como primer paso, en esta tesis analizamos los problemas de cuadratura que aparecen al resolver PDE utilizando DNN y proponemos diferentes métodos de integración para superar estas limitacionesbca

Rapid solution of potential integral equations in complicated 3-dimensional geometries

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1997.Includes bibliographical references (p. 133-137).by Joel Reuben Phillips.Ph.D

Deep Learning for Inverting Borehole Resistivity Measurements

There exist multiple traditional methods to solve inverse problems, mainly, gradient-based or statistics-based methods. However, these methods have severe limitations. In particular, they often need to compute the forward problem hundreds of times, which is computationally expensive in three-dimensional (3D) problems. In this dissertation, we propose the use of Deep Learning (DL) techniques to solve inverse problems. Although the training stage of a Deep Neural Network (DNN) may be time-consuming, after the network is properly trained it can forecast the solution in a fraction of a second, facilitating real-time operations. In the first part of this dissertation, we investigate appropriate loss functions to train a DNN when dealing with an inverse problem. Additionally, to properly train a DNN that approximates the inverse solution, we require a large dataset containing the solution of the forward problem. To create such dataset, we need to solve aPartial Differential Equation (PDE) thousands of times. Building a dataset may be time-consuming, especially for two and three-dimensional problems since solving PDEs using traditional methods, such as the Finite Element Method (FEM), is computationally expensive. Thus, we want to reduce the computational cost of building the database needed to train the DNN. For this, we propose the use of rIGA methods. In addition, we explore the possibility of using DL techniques to solve PDEs, which is the main computational bottleneck when solving inverse problems. Our main goal is to develop a fast forward simulator for solving parametric PDEs. As a first step, in this dissertation we analyze the quadrature problems that appear while solving PDEs using DNNs and propose different integration methods to overcome these limitations

Tackling Lateral Variability Using Surface Waves: A Tomography-Like Approach

Lateral velocity variations in the near-surface reflect the presence of buried geological or anthropic structures, and their identification is of interest for many fields of application. Surface wave tomography (SWT) is a powerful technique for detecting both smooth and sharp lateral velocity variations at very different scales. A surface-wave inversion scheme derived from SWT is here applied to a 2-D active seismic dataset to characterize the shape of an urban waste deposit in an old landfill, located 15 km South of Vienna (Austria). First, the tomography-derived inverse problem for the 2-D case is defined: under the assumption of straight rays at the surface connecting sources and receivers, the forward problem for one frequency reduces to a linear relationship between observed phase differences at adjacent receivers and wavenumbers (from which phase velocities are straightforwardly derived). A norm damping regularization constraint is applied to ensure a smooth solution in space: the choice of the damping parameter is made through a minimization process, by which only phase variations of the order of the average wavelength are modelled. The inverse problem is solved for each frequency with a weighted least-squares approach, to take into account the data error variances. An independent multi-offset phase analysis (MOPA) is performed using the same dataset, for comparison: pseudo-sections from the tomography-derived linear inversion and MOPA are very consistent, with the former giving a more continuous result both in space and frequency and less artefacts. Local dispersion curves are finally depth inverted and a quasi-2-D shear wave velocity section is retrieved: we identify a well-defined low velocity zone and interpret it as the urban waste deposit body. Results are consistent with both electrical and electromagnetic measurements acquired on the same line

Electronic correlations in inhomogeneous model systems: numerical simulation of spectra and transmission

Many fascinating features in condensed matter systems emerge due to the interaction between electrons. Magnetism is such a paramount consequence, which is explained in terms of the exchange interaction of electrons. Another prime example is the metal-to-Mott-insulator transition, where the energy cost of Coulomb repulsion competes against the kinetic energy, the latter favoring delocalization. While systems of correlated electrons are exciting and show remarkable and technologically promising physical properties, they are difficult to treat theoretically. A single-particle description is insufficient; the quantum many-body problem of interacting electrons has to be solved. In the present thesis, we study physical properties of half-metallic ferromagnets which are used in spintronic devices. Half-metals exhibit a metallic spin channel, while the other spin channel is insulating; they are characterized by a high spin polarization. This thesis contributes to the development of numerical methods and applies them to models of half-metallic ferromagnets. Throughout this work, the single-band Hubbard Hamiltonian is considered, and electronic correlations are treated within dynamical mean-field theory. Instead of directly solving the lattice model, the dynamical mean-field theory amounts to solving a local, effective impurity problem that is determined self-consistently. At finite temperatures, this impurity problem is solved employing continuous-time quantum Monte Carlo algorithms formulated in the action formalism. As these algorithms are formulated in imaginary time, an analytic continuation is required to obtain spectral functions. We formulate a version of the N-point Padé algorithm that calculates the location of the poles in a least-squares sense. To directly obtain spectra for real frequencies, we employ Hamiltonian-based tensor network methods at zero temperature. We also summarize the ideas of the density matrix renormalization group algorithm, and of the time evolution using the time-dependent variational principle, employing a diagrammatic notation. Real materials never display perfect translational symmetry. Thus, realistic models require the inclusion of disorder effects. In this work, we discuss these within a single-site approximation, the coherent potential approximation, and combine it with the dynamical mean-field theory, allowing to treat interacting electrons in multicomponent alloys on a local level. We extend this combined scheme to off-diagonal disorder, that is, disorder in the hopping amplitudes, by employing the Blackman–Esterling–Berk formalism. For this purpose, we illustrate the ideas of this formalism using tensor diagrams and provide an efficient implementation. The structure of the effective medium is discussed, and a concentration scaling is proposed that resolves some of its peculiarities. The limit of vanishing hopping between different components is discussed and solved analytically for the Bethe lattice with a general coordination number. We exemplify the combined algorithm for a Bethe lattice, showing results that exhibit alloy-band-insulator to correlated-metal to Mott-insulator transitions. We study models of half-metallic ferromagnets to elucidate the effects of local electronic correlations on the spectral function. To model half-metallicity, a static spin splitting is used to produce the half-metallic density of states. Applying the Padé analytic continuation to the self-energy instead of the Green’s function produces reliable spectral functions agreeing with the zero-temperature results obtained for real frequencies. To address transport properties, we investigate the interface of a half-metallic layer and a metallic, band insulating, or Mott insulating layer. We observe charge reconstruction which induces metallicity at the interface; quasiparticle states are present in the Mott insulating layer even for a large Hubbard interaction. The transmission through a barrier made of such a single interacting half-metallic layer sandwiched by metallic leads is studied employing the Meir–Wingreen formalism. This allows for a transparent calculation of the transmission in the presence of the Hubbard interaction. For a strong coupling of the central layer to the leads, we identify high intensity bound states which do not contribute to the transmission. For small coupling, on the other hand, we find resonant states which enhance the transmission. In particular, we demonstrate that even for a single half-metallic layer, highly polarized transmissions are achievable

Stratification of canopy magnetic fields in a plage region. Constraints from a spatially-regularized weak-field approximation method

The role of magnetic fields in the chromospheric heating problem remains greatly unconstrained. Most theoretical predictions from numerical models rely on a magnetic configuration, field strength and connectivity whose details have not been well established with observational studies. High-resolution studies of chromospheric magnetic fields in plage are very scarce or non-existent in general. Our aim is to study the stratification of the magnetic field vector in plage regions. We use high-spatial resolution full-Stokes observations acquired with CRISP instrument at the Swedish 1-m Solar Telescope in the Mg I $\lambda$5173, Na I $\lambda$5896 and Ca II $\lambda$8542 lines. We have developed a spatially-regularized weak-field approximation (WFA) method based on the idea of spatial regularization. This method allows for a fast computation of magnetic field maps for an extended field of view. The fidelity of this new technique has been assessed using a snapshot from a realistic 3D magnetohydrodynamics simulation. We have derived the depth-stratification of the line-of-sight component of the magnetic field from the photosphere to the chromosphere in a plage region. The magnetic fields are concentrated in the intergranular lanes in the photosphere and expand horizontally toward the chromosphere, filling all the space and forming a canopy. Our results suggest that the lower boundary of this canopy must be located around 400-600 km from the photosphere. The mean canopy total magnetic field strength in the lower chromosphere ($z\approx760$ km) is 658 G. At $z=1160$ km we estimate $\approx 417$ G. We propose a modification to the WFA that improves its applicability to data with worse signal-to-noise ratio. These methods provide a quick and reliable way of studying multi-layer magnetic field observations without the many difficulties inherent to other inversion methods.Comment: Accepted for publication on 2020-08-2