Advances in Bayesian inference and stable optimization for large-scale machine learning problems

A core task in machine learning, and the topic of this thesis, is developing faster and more accurate methods of posterior inference in probabilistic models. The thesis has two components. The first explores using deterministic methods to improve the efficiency of Markov Chain Monte Carlo (MCMC) algorithms. We propose new MCMC algorithms that can use deterministic methods as a “prior” to bias MCMC proposals to be in areas of high posterior density, leading to highly efficient sampling. In Chapter 2 we develop such methods for continuous distributions, and in Chapter 3 for binary distributions. The resulting methods consistently outperform existing state-of-the-art sampling techniques, sometimes by several orders of magnitude. Chapter 4 uses similar ideas as in Chapters 2 and 3, but in the context of modeling the performance of left-handed players in one-on-one interactive sports. The second part of this thesis explores the use of stable stochastic gradient descent (SGD) methods for computing a maximum a posteriori (MAP) estimate in large-scale machine learning problems. In Chapter 5 we propose two such methods for softmax regression. The first is an implementation of Implicit SGD (ISGD), a stable but difficult to implement SGD method, and the second is a new SGD method specifically designed for optimizing a double-sum formulation of the softmax. Both methods comprehensively outperform the previous state-of-the-art on seven real world datasets. Inspired by the success of ISGD on the softmax, we investigate its application to neural networks in Chapter 6. In this chapter we present a novel layer-wise approximation of ISGD that has efficiently computable updates. Experiments show that the resulting method is more robust to high learning rates and generally outperforms standard backpropagation on a variety of tasks

A statistical correlation of sunquakes based on their seismic and white-light emission

Several mechanisms have been proposed to explain the transient seismic emission, i.e. “sunquakes,” from some solar flares. Some theories associate high-energy electrons and/or white-light emission with sunquakes. High-energy charged particles and their subsequent heating of the photosphere and/or chromosphere could induce acoustic waves in the solar interior. We carried out a correlative study of solar flares with emission in hard X-rays, enhanced continuum emission at 6173 Å, and transient seismic emission. We selected those flares observed by the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI) with a considerable flux above 50 keV between 1 January 2010 and 26 June 2014. We then used data from the Helioseismic and Magnetic Imager onboard the Solar Dynamic Observatory to search for excess visible-continuum emission and new sunquakes not previously reported. We found a total of 18 sunquakes out of 75 flares investigated. All of the sunquakes were associated with an enhancement of the visible continuum during the flare. Finally, we calculated a coefficient of correlation for a set of dichotomic variables related to these observations. We found a strong correlation between two of the standard helioseismic detection techniques, and between sunquakes and visible-continuum enhancements. We discuss the phenomenological connectivity between these physical quantities and the observational difficulties of detecting seismic signals and excess continuum radiation

Early and late systolic wall stress differentially relate to myocardial contraction and relaxation in middle-aged adults: the Asklepios study

Experimental studies implicate late systolic load as a determinant of impaired left ventricular (LV) relaxation. We aimed to assess the relationship between the myocardial loading sequence and left ventricular (LV) contraction and relaxation. Time-resolved central pressure and time-resolved LV geometry were measured with carotid tonometry and speckle-tracking echocardiography, respectively, for computation of time-resolved ejection-phase myocardial wall stress (EP-MWS) among 1,214 middle-aged adults without manifest cardiovascular disease from the general population. Early diastolic annular velocity, systolic annular velocities were measured with tissue Doppler imaging and segmentaveraged longitudinal strain was measured with speckle-tracking echocardiography. After adjustment for age, gender and potential confounders, late EP-MWS was negatively associated with early diastolic mitral annular velocity (e', standardized β=-0.25; P<0.0001) and mitral inflow propagation velocity (Vpe, standardized β=-0.13; P=0.02). In contrast, early EP-MWS was positively associated with e' (standardized β=0.18; P<0.0001) and Vpe (standardized β=0.22; P<0.0001). A higher late EP-MWS predicted a lower systolic mitral annular velocity (S', standardized β=-0.31; P<0.0001) and lesser myocardial longitudinal strain (standardized β=0.32; P<0.0001), whereas a higher early EP-MWS was associated with a higher S' (standardized β=0.16; P=0.002) and greater longitudinal strain (standardized β=-0.24; P=0.002). The loading sequence remained independently associated with e' after adjustment for S' or systolic longitudinal strain. In the context of available experimental data, our findings support the role of the myocardial loading sequence as a determinant of LV systolic and diastolic function. A loading sequence characterized by prominent late systolic wall stress was associated with lower longitudinal systolic function and diastolic relaxation

A simultaneous planar laser-induced fluorescence, particle image velocimetry and particle tracking velocimetry technique for the investigation of thin liquid-film flows

AbstractA simultaneous measurement technique based on planar laser-induced fluorescence imaging (PLIF) and particle image/tracking velocimetry (PIV/PTV) is described for the investigation of the hydrodynamic characteristics of harmonically excited liquid thin-film flows. The technique is applied as part of an extensive experimental campaign that covers four different Kapitza (Ka) number liquids, Reynolds (Re) numbers spanning the range 2.3–320, and inlet-forced/wave frequencies in the range 1–10Hz. Film thicknesses (from PLIF) for flat (viscous and unforced) films are compared to micrometer stage measurements and analytical predictions (Nusselt solution), with a resulting mean deviation being lower than the nominal resolution of the imaging setup (around 20μm). Relative deviations are calculated between PTV-derived interfacial and bulk velocities and analytical results, with mean values amounting to no more than 3.2% for both test cases. In addition, flow rates recovered using LIF/PTV (film thickness and velocity profile) data are compared to direct flowmeter readings. The mean relative deviation is found to be 1.6% for a total of six flat and nine wavy flows. The practice of wave/phase-locked flow-field averaging is also implemented, allowing the generation of highly localized velocity profile, bulk velocity and flow rate data along the wave topology. Based on this data, velocity profiles are extracted from 20 locations along the wave topology and compared to analytically derived ones based on local film thickness measurements and the Nusselt solution. Increasing the waviness by modulating the forcing frequency is found to result in lower absolute deviations between experiments and theoretical predictions ahead of the wave crests, and higher deviations behind the wave crests. At the wave crests, experimentally derived interfacial velocities are overestimated by nearly 100%. Finally, locally non-parabolic velocity profiles are identified ahead of the wave crests; a phenomenon potentially linked to the cross-stream velocity field

Jet engine exhaust emissions of high altitude commercial aircraft projected to 1990

Projected minimum levels of engine exhaust emissions that may be practicably achievable for future commercial aircraft operating at high-altitude cruise conditions are presented. The forecasts are based on:(1) current knowledge of emission characteristics of combustors and augmentors; (2) the status of combustion research in emission reduction technology; and (3) predictable trends in combustion systems and operating conditions as required for projected engine designs that are candidates for advanced subsonic or supersonic commercial aircraft fueled by either JP fuel, liquefied natural gas, or hydrogen. Results are presented for cruise conditions in terms of both an emission index (g constituent/kg fuel) and an emission rate (g constituent/hr)

Advanced combustion techniques for controlling NO sub x emissions of high altitude cruise aircraft

An array of experiments designed to explore the potential of advanced combustion techniques for controlling the emissions of aircraft into the upper atmosphere was discussed. Of particular concern are the oxides of nitrogen (NOx) emissions into the stratosphere. The experiments utilize a wide variety of approaches varying from advanced combustor concepts to fundamental flame tube experiments. Results are presented which indicate that substantial reductions in cruise NOx emissions should be achievable in future aircraft engines. A major NASA program is described which focuses the many fundamental experiments into a planned evolution and demonstration of the prevaporized-premixed combustion technique in a full-scale engine

Wide range operation of advanced low NOx combustors for supersonic high-altitude aircraft gas turbines

An initial rig program tested the Jet Induced Circulation (JIC) and Vortex Air Blast (VAB) systems in small can combustor configurations for NOx emissions at a simulated high altitude, supersonic cruise condition. The VAB combustor demonstrated the capability of meeting the NOx goal of 1.0 g NO2/kg fuel at the cruise condition. In addition, the program served to demonstrate the limited low-emissions range available from the lean, premixed combustor. A follow-on effort was concerned with the problem of operating these lean, premixed combustors with acceptable emissions at simulated engine idle conditions. Various techniques have been demonstrated that allow satisfactory operation on both the JIC and VAB combustors at idle with CO emissions below 20 g/kg fuel. The VAB combustor was limited by flashback/autoignition phenomena at the cruise conditions to a pressure of 8 atmospheres. The JIC combustor was operated up to the full design cruise pressure of 14 atmospheres without encountering an autoignition limitation although the NOx levels, in the 2-3 g NO2/kg fuel range, exceeded the program goal

Aeronautical Engineering: A special bibliography with indexes, supplement 51

This bibliography lists 206 reports, articles, and other documents introduced into the NASA Scientific and Technical Information System in November 1974