Stochastic approximation of score functions for Gaussian processes

We discuss the statistical properties of a recently introduced unbiased stochastic approximation to the score equations for maximum likelihood calculation for Gaussian processes. Under certain conditions, including bounded condition number of the covariance matrix, the approach achieves $O(n)$ storage and nearly $O(n)$ computational effort per optimization step, where $n$ is the number of data sites. Here, we prove that if the condition number of the covariance matrix is bounded, then the approximate score equations are nearly optimal in a well-defined sense. Therefore, not only is the approximation efficient to compute, but it also has comparable statistical properties to the exact maximum likelihood estimates. We discuss a modification of the stochastic approximation in which design elements of the stochastic terms mimic patterns from a $2^n$ factorial design. We prove these designs are always at least as good as the unstructured design, and we demonstrate through simulation that they can produce a substantial improvement over random designs. Our findings are validated by numerical experiments on simulated data sets of up to 1 million observations. We apply the approach to fit a space-time model to over 80,000 observations of total column ozone contained in the latitude band $40^{\circ}$-$50^{\circ}$N during April 2012.Comment: Published in at http://dx.doi.org/10.1214/13-AOAS627 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Techniques for the Analysis and Understanding of Cosmic Evolution

abstract: The Cosmic Microwave Background (CMB) has provided precise information on the evolution of the Universe and the current cosmological paradigm. The CMB has not yet provided definitive information on the origin and strength of any primordial magnetic fields or how they affect the presence of magnetic fields observed throughout the cosmos. This work outlines an alternative method to investigating and identifying the presence of cosmic magnetic fields. This method searches for Faraday Rotation (FR) and specifically uses polarized CMB photons as back-light. I find that current generation CMB experiments may be not sensitive enough to detect FR but next generation experiments should be able to make highly significant detections. Identifying FR with the CMB will provide information on the component of magnetic fields along the line of sight of observation. The 21cm emission from the hyperfine splitting of neutral Hydrogen in the early universe is predicted to provide precise information about the formation and evolution of cosmic structure, complementing the wealth of knowledge gained from the CMB. 21cm cosmology is a relatively new field, and precise measurements of the Epoch of Reionization (EoR) have not yet been achieved. In this work I present 2σ upper limits on the power spectrum of 21cm fluctuations (Δ²(k)) probed at the cosmological wave number k from the Donald C. Backer Precision Array for Probing the Epoch of Reionization (PAPER) 64 element deployment. I find upper limits on Δ²(k) in the range 0.3 < k < 0.6 h/Mpc to be (650 mK)², (450 mK)², (390 mK)², (250 mK)², (280mK)², (250 mK)² at redshifts z = 10.87, 9.93, 8.91, 8.37, 8.13 and 7.48 respectively Building on the power spectrum analysis, I identify a major limiting factor in detecting the 21cm power spectrum. This work is concluded by outlining a metric to evaluate the predisposition of redshifted 21cm interferometers to foreground contamination in power spectrum estimation. This will help inform the construction of future arrays and enable high fidelity imaging and cross-correlation analysis with other high redshift cosmic probes like the CMB and other upcoming all sky surveys. I find future arrays with uniform (u,v) coverage and small spectral evolution of their response in the (u,v,f) cube can minimize foreground leakage while pursuing 21cm imaging.Dissertation/ThesisDoctoral Dissertation Physics 201

Fast Numerical and Machine Learning Algorithms for Spatial Audio Reproduction

Audio reproduction technologies have underwent several revolutions from a purely mechanical, to electromagnetic, and into a digital process. These changes have resulted in steady improvements in the objective qualities of sound capture/playback on increasingly portable devices. However, most mobile playback devices remove important spatial-directional components of externalized sound which are natural to the subjective experience of human hearing. Fortunately, the missing spatial-directional parts can be integrated back into audio through a combination of computational methods and physical knowledge of how sound scatters off of the listener's anthropometry in the sound-field. The former employs signal processing techniques for rendering the sound-field. The latter employs approximations of the sound-field through the measurement of so-called Head-Related Impulse Responses/Transfer Functions (HRIRs/HRTFs). This dissertation develops several numerical and machine learning algorithms for accelerating and personalizing spatial audio reproduction in light of available mobile computing power. First, spatial audio synthesis between a sound-source and sound-field requires fast convolution algorithms between the audio-stream and the HRIRs. We introduce a novel sparse decomposition algorithm for HRIRs based on non-negative matrix factorization that allows for faster time-domain convolution than frequency-domain fast-Fourier-transform variants. Second, the full sound-field over the spherical coordinate domain must be efficiently approximated from a finite collection of HRTFs. We develop a joint spatial-frequency covariance model for Gaussian process regression (GPR) and sparse-GPR methods that supports the fast interpolation and data fusion of HRTFs across multiple data-sets. Third, the direct measurement of HRTFs requires specialized equipment that is unsuited for widespread acquisition. We ``bootstrap'' the human ability to localize sound in listening tests with Gaussian process active-learning techniques over graphical user interfaces that allows the listener to infer his/her own HRTFs. Experiments are conducted on publicly available HRTF datasets and human listeners

The Unified-FFT Method for Fast Solution of Integral Equations as Applied to Shielded-Domain Electromagnetics

Electromagnetic (EM) solvers are widely used within computer-aided design (CAD) to improve and ensure success of circuit designs. Unfortunately, due to the complexity of Maxwell\u27s equations, they are often computationally expensive. While considerable progress has been made in the realm of speed-enhanced EM solvers, these fast solvers generally achieve their results through methods that introduce additional error components by way of geometric approximations, sparse-matrix approximations, multilevel decomposition of interactions, and more. This work introduces the new method, Unified-FFT (UFFT). A derivative of method of moments, UFFT scales as O(N log N), and achieves fast analysis by the unique combination of FFT-enhanced matrix fill operations (MFO) with FFT-enhanced matrix solve operations (MSO). In this work, two versions of UFFT are developed, UFFT-Precorrected (UFFT-P) and UFFT-Grid Totalizing (UFFT-GT). UFFT-P uses precorrected FFT for MSO and allows the use of basis functions that do not conform to a regular grid. UFFT-GT uses conjugate gradient FFT for MSO and features the capability of reducing the error of the solution down to machine precision. The main contribution of UFFT-P is a fast solver, which utilizes FFT for both MFO and MSO. It is demonstrated in this work to not only provide simulation results for large problems considerably faster than state of the art commercial tools, but also to be capable of simulating geometries which are too complex for conventional simulation. In UFFT-P these benefits come at the expense of a minor penalty to accuracy. UFFT-GT contains further contributions as it demonstrates that such a fast solver can be accurate to numerical precision as compared to a full, direct analysis. It is shown to provide even more algorithmic efficiency and faster performance than UFFT-P. UFFT-GT makes an additional contribution in that it is developed not only for planar geometries, but also for the case of multilayered dielectrics and metallization. This functionality is particularly useful for multi-layered printed circuit boards (PCBs) and integrated circuits (ICs). Finally, UFFT-GT contributes a 3D planar solver, which allows for current to be discretized in the z-direction. This allows for similar fast and accurate simulation with the inclusion of some 3D features, such as vias connecting metallization planes

Primordial Gravitational Wave Detectability with Deep Small-sky Cosmic Microwave Background Experiments

We use the Bayesian estimation on direct T - Q - U cosmic microwave background (CMB) polarization maps to forecast errors on the tensor-to-scalar power ratio r, and hence on primordial gravitational waves, as a function of sky coverage f_sky. This map-based likelihood filters the information in the pixel-pixel space into the optimal combinations needed for r detection for cut skies, providing enhanced information over a first-step linear separation into a combination of E, B, and mixed modes, and ignoring the latter. With current computational power and for typical resolutions appropriate for r detection, the large matrix inversions required are accurate and fast. Our simulations explore two classes of experiments, with differing bolometric detector numbers, sensitivities, and observational strategies. One is motivated by a long duration balloon experiment like Spider, with pixel noise ∝ √f_sky for a specified observing period. This analysis also applies to ground-based array experiments. We find that, in the absence of systematic effects and foregrounds, an experiment with Spider-like noise concentrating on f_sky ~ 0.02-0.2 could place a 2σ_r ≈ 0.014 boundary (~95% confidence level), which rises to 0.02 with an ℓ-dependent foreground residual left over from an assumed efficient component separation. We contrast this with a Planck-like fixed instrumental noise as f_sky varies, which gives a Galaxy-masked (f_sky = 0.75) 2σ_r ≈ 0.015, rising to ≈0.05 with the foreground residuals. Using as the figure of merit the (marginalized) one-dimensional Shannon entropy of r, taken relative to the first 2003 WMAP CMB-only constraint, gives –2.7 bits from the 2012 WMAP9+ACT+SPT+LSS data, and forecasts of –6 bits from Spider (+ Planck); this compares with up to –11 bits for CMBPol, COrE, and PIXIE post-Planck satellites and –13 bits for a perfectly noiseless cosmic variance limited experiment. We thus confirm the wisdom of the current strategy for r detection of deeply probed patches covering the f_sky minimum-error trough with balloon and ground experiments

X10 for high-performance scientific computing

High performance computing is a key technology that enables large-scale physical simulation in modern science. While great advances have been made in methods and algorithms for scientific computing, the most commonly used programming models encourage a fragmented view of computation that maps poorly to the underlying computer architecture. Scientific applications typically manifest physical locality, which means that interactions between entities or events that are nearby in space or time are stronger than more distant interactions. Linear-scaling methods exploit physical locality by approximating distant interactions, to reduce computational complexity so that cost is proportional to system size. In these methods, the computation required for each portion of the system is different depending on that portion’s contribution to the overall result. To support productive development, application programmers need programming models that cleanly map aspects of the physical system being simulated to the underlying computer architecture while also supporting the irregular workloads that arise from the fragmentation of a physical system. X10 is a new programming language for high-performance computing that uses the asynchronous partitioned global address space (APGAS) model, which combines explicit representation of locality with asynchronous task parallelism. This thesis argues that the X10 language is well suited to expressing the algorithmic properties of locality and irregular parallelism that are common to many methods for physical simulation. The work reported in this thesis was part of a co-design effort involving researchers at IBM and ANU in which two significant computational chemistry codes were developed in X10, with an aim to improve the expressiveness and performance of the language. The first is a Hartree–Fock electronic structure code, implemented using the novel Resolution of the Coulomb Operator approach. The second evaluates electrostatic interactions between point charges, using either the smooth particle mesh Ewald method or the fast multipole method, with the latter used to simulate ion interactions in a Fourier Transform Ion Cyclotron Resonance mass spectrometer. We compare the performance of both X10 applications to state-of-the-art software packages written in other languages. This thesis presents improvements to the X10 language and runtime libraries for managing and visualizing the data locality of parallel tasks, communication using active messages, and efficient implementation of distributed arrays. We evaluate these improvements in the context of computational chemistry application examples. This work demonstrates that X10 can achieve performance comparable to established programming languages when running on a single core. More importantly, X10 programs can achieve high parallel efficiency on a multithreaded architecture, given a divide-and-conquer pattern parallel tasks and appropriate use of worker-local data. For distributed memory architectures, X10 supports the use of active messages to construct local, asynchronous communication patterns which outperform global, synchronous patterns. Although point-to-point active messages may be implemented efficiently, productive application development also requires collective communications; more work is required to integrate both forms of communication in the X10 language. The exploitation of locality is the key insight in both linear-scaling methods and the APGAS programming model; their combination represents an attractive opportunity for future co-design efforts