Mean Field Analysis of Neural Networks: A Law of Large Numbers

Machine learning, and in particular neural network models, have revolutionized fields such as image, text, and speech recognition. Today, many important real-world applications in these areas are driven by neural networks. There are also growing applications in engineering, robotics, medicine, and finance. Despite their immense success in practice, there is limited mathematical understanding of neural networks. This paper illustrates how neural networks can be studied via stochastic analysis, and develops approaches for addressing some of the technical challenges which arise. We analyze one-layer neural networks in the asymptotic regime of simultaneously (A) large network sizes and (B) large numbers of stochastic gradient descent training iterations. We rigorously prove that the empirical distribution of the neural network parameters converges to the solution of a nonlinear partial differential equation. This result can be considered a law of large numbers for neural networks. In addition, a consequence of our analysis is that the trained parameters of the neural network asymptotically become independent, a property which is commonly called "propagation of chaos"

The Impacts of Three Flamelet Burning Regimes in Nonlinear Combustion Dynamics

Axisymmetric simulations of a liquid rocket engine are performed using a delayed detached-eddy-simulation (DDES) turbulence model with the Compressible Flamelet Progress Variable (CFPV) combustion model. Three different pressure instability domains are simulated: completely unstable, semi-stable, and fully stable. The different instability domains are found by varying the combustion chamber and oxidizer post length. Laminar flamelet solutions with a detailed chemical mechanism are examined. The $\beta$ Probability Density Function (PDF) for the mixture fraction and Dirac $\delta$ PDF for both the pressure and the progress variable are used. A coupling mechanism between the Heat Release Rate (HRR) and the pressure in an unstable cycle is demonstrated. Local extinction and reignition is investigated for all the instability domains using the full S-curve approach. A monotonic decrease in the amount of local extinctions and reignitions occurs when pressure oscillation amplitude becomes smaller. The flame index is used to distinguish between the premixed and non-premixed burning mode in different stability domains. An additional simulation of the unstable pressure oscillation case using only the stable flamelet burning branch of the S-curve is performed. Better agreement with experiments in terms of pressure oscillation amplitude is found when the full S-curve is used.Comment: 25 pages, 12 figures. Submitted to Combustion and Flame for a Special Issu

Transient Behavior near Liquid-Gas Interface at Supercritical Pressure

Numerical heat and mass transfer analysis of a configuration where a cool liquid hydrocarbon is suddenly introduced to a hotter gas at supercritical pressure shows that a well-defined phase equilibrium can be established before substantial growth of typical hydrodynamic instabilities. The equilibrium values at the interface quickly reach near-steady values. Sufficiently thick diffusion layers form quickly around the liquid-gas interface (e.g., 3-10 microns for the liquid phase and 10-30 microns for the gas phase in 10-100 microseconds), where density variations become increasingly important with pressure as mixing of species is enhanced. While the hydrocarbon vaporizes and the gas condenses for all analyzed pressures, the net mass flux across the interface reverses as pressure is increased, showing that a clear vaporization-driven problem at low pressures may present condensation at higher pressures. This is achieved while heat still conducts from gas to liquid. Analysis of fundamental thermodynamic laws on a fixed-mass element containing the diffusion layers proves the thermodynamic viability of the obtained results.Comment: Submitted for publication in International Journal of Heat and Mass Transfer. 29 pages, 18 figure

Understanding liquid-jet atomization cascades via vortex dynamics

Temporal instabilities of a planar liquid jet are studied using direct numerical simulation (DNS) of the incompressible Navier-Stokes equations with level-set (LS) and volume-of-fluid (VoF) surface tracking methods. $\lambda_2$ contours are used to relate the vortex dynamics to the surface dynamics at different stages of the jet breakup, namely, lobe formation, lobe perforation, ligament formation, stretching, and tearing. Three distinct breakup mechanisms are identified in the primary breakup, which are well categorized on the parameter space of gas Weber number ($We_g$) versus liquid Reynolds number ($Re_l$). These mechanisms are analyzed here from a vortex dynamics perspective. Vortex dynamics explains the hairpin formation, and the interaction between the hairpins and the Kelvin-Helmholtz (KH) roller explains the perforation of the lobes, which is attributed to the streamwise overlapping of two oppositely-oriented hairpin vortices on top and bottom of the lobe. The formation of corrugations on the lobe front edge at high $Re_l$ is also related to the location and structure of the hairpins with respect to the KH vortex. The lobe perforation and corrugation formation are inhibited at low $Re_l$ and low $We_g$ due to the high surface tension and viscous forces, which damp the small scale corrugations and resist hole formation. Streamwise vorticity generation - resulting in three-dimensional instabilities - is mainly caused by vortex stretching and baroclinic torque at high and low density ratios, respectively. Generation of streamwise vortices and their interaction with spanwise vortices produce the liquid structures seen at various flow conditions. Understanding the liquid sheet breakup and the related vortex dynamics are crucial for controlling the droplet size distribution in primary atomization.Comment: Submitted for publication in Journal of Fluid Mechanics. 56 pages; 52 figure

Length-scale cascade and spread rate of atomizing planar liquid jets

The primary breakup of a planar liquid jet is explored via direct numerical simulation (DNS) of the incompressible Navier-Stokes equation with level-set and volume-of-fluid interface capturing methods. PDFs of the local radius of curvature and the local cross-flow displacement of the liquid-gas interface are evaluated over wide ranges of the Reynolds number ($Re$), Weber number ($We$), density ratio and viscosity ratio. The temporal cascade of liquid-structure length scales and the spread rate of the liquid jet during primary atomization are analyzed. The formation rate of different surface structures, e.g. lobes, ligaments and droplets, are compared for different flow conditions and are explained in terms of the vortex dynamics in each atomization domain that we identified recently. With increasing $We$, the average radius of curvature of the surface decreases, the number of small droplets increases, and the cascade and the surface area growth occur at faster rates. The spray angle is mainly affected by $Re$ and density ratio, and is larger at higher $We$, at higher density ratios, and also at lower $Re$. The change in the spray spread rate versus $Re$ is attributed to the angle of ligaments stretching from the jet core, which increases as $Re$ decreases. Gas viscosity has negligible effect on both the droplet-size distribution and the spray angle. Increasing the wavelength-to-sheet-thickness ratio, however, increases the spray angle and the structure cascade rate, while decreasing the droplet size. The smallest length scale is determined more by surface tension and liquid inertia than by the liquid viscosity, while gas inertia and liquid surface tension are the key parameters in determining the spray angle.Comment: Submitted for publication to International Journal of Multiphase Flow. 37 pages; 33 figure