Non-Volatile Memory Array Based Quantization- and Noise-Resilient LSTM Neural Networks

In cloud and edge computing models, it is important that compute devices at the edge be as power efficient as possible. Long short-term memory (LSTM) neural networks have been widely used for natural language processing, time series prediction and many other sequential data tasks. Thus, for these applications there is increasing need for low-power accelerators for LSTM model inference at the edge. In order to reduce power dissipation due to data transfers within inference devices, there has been significant interest in accelerating vector-matrix multiplication (VMM) operations using non-volatile memory (NVM) weight arrays. In NVM array-based hardware, reduced bit-widths also significantly increases the power efficiency. In this paper, we focus on the application of quantization-aware training algorithm to LSTM models, and the benefits these models bring in terms of resilience against both quantization error and analog device noise. We have shown that only 4-bit NVM weights and 4-bit ADC/DACs are needed to produce equivalent LSTM network performance as floating-point baseline. Reasonable levels of ADC quantization noise and weight noise can be naturally tolerated within our NVMbased quantized LSTM network. Benchmark analysis of our proposed LSTM accelerator for inference has shown at least 2.4x better computing efficiency and 40x higher area efficiency than traditional digital approaches (GPU, FPGA, and ASIC). Some other novel approaches based on NVM promise to deliver higher computing efficiency (up to 4.7x) but require larger arrays with potential higher error rates.Comment: Published in: 2019 IEEE International Conference on Rebooting Computing (ICRC

The C-13 Suess effect in the world surface oceans and its implications for oceanic uptake of CO2:Analysis of observations at Bermuda

Surface ocean water delta(13)C measurements near Bermuda are examined in an attempt to find the annual decrease caused by the addition of anthropogenic CO2 to the atmosphere. We refer to this trend as the surface ocean C-13 Suess effect. Interannual variability, which may be related to the El Nino - Southern Oscillation in the Atlantic Ocean, is apparent. We try to correct the data for this variability so as to better determine the trend. The trend has implications for the uptake of anthropogenic CO2 by the oceans. We employ a three-dimensional model of ocean chemistry to relate the trend at Bermuda to the average ocean trend, then use the average ocean trend to estimate the vertical diffusivity K in a one-dimensional ocean model, and finally use this model to calculate the oceanic uptake of CO2. Uncertainties associated with the estimation of the Suess effect at Bermuda and in the analysis procedure preclude a firm estimate of the oceanic uptake of CO2. Results are, in general, consistent with the low side of the Intergovernmental Panel on Climate Control estimation of 2.0 +/- 0.8 GtC yr(-1). With a longer record at Bermuda and delta(13)C observations at additional locations, we believe this approach will lead to a useful estimate of oceanic uptake

Nitrous oxide variability at sub-kilometre resolution in the Atlantic sector of the Southern Ocean

The Southern Ocean is an important region for global nitrous oxide (N2O) cycling. The contribution of different source and sink mechanisms is, however, not very well constrained due to a scarcity of seawater data from the area. Here we present high-resolution surface N2O measurements from the Atlantic sector of the Southern Ocean, taking advantage of a relatively new underway setup allowing for collection of data during transit across mesoscale features such as frontal systems and eddies. Covering a range of different environments and biogeochemical settings, N2O saturations and sea-to-air fluxes were highly variable: Saturations ranged from 96.5% at the sea ice edge in the Weddell Sea to 126.1% across the Polar Frontal Zone during transit to South Georgia. Negative sea-to-air fluxes (N2O uptake) of up to −1.3 µmol m−2 d−1 were observed in the Subantarctic Zone and highest positive fluxes (N2O emission) of 14.5 µmol m−2 d−1 in Stromness Bay, coastal South Georgia. Although N2O saturations were high in areas of high productivity, no correlation between saturations and chlorophyll a (as a proxy for productivity) was observed. Nevertheless, there is a clear effect of islands and shallow bathymetry on N2O production as inferred from supersaturations

Stoichiometry of the degradation of dissolved and particulate biogenic organic matter in the NW Iberian upwelling

The average composition of the dissolved and particulate products of early degradation of marine phytoplankton has been established for the first time in a coastal upwelling system using a mixing analysis along isopycnal surfaces combined with a stoichiometric model. About 17–18% of the mineralized organic matter is derived from the decomposition of organic particulates, and 16–35% is from the dissolved organic matter. The remaining 50–70% is derived probably from large fast sinking particles. On average, the mineralized material on large particles has the closest composition to the Redfield formula. The ratio of dissolved saccharides to dissolved organic matter respiration is >40% higher than expected from a material of Redfield composition. Finally, the ratio of lipid to particulate organic matter respiration is >80% larger than expected from a material of Redfield composition. Regarding the decomposition of hard structures, biogenic silica dissolves predominantly in the inner shelf, where organic carbon oxidation is more intense, and diatom deposition occurs preferentially

The Atacama Cosmology Telescope: a measurement of the primordial power spectrum

We present constraints on the primordial power spectrum of adiabatic fluctuations using data from the 2008 Southern Survey of the Atacama Cosmology Telescope (ACT). The angular resolution of ACT provides sensitivity to scales beyond \ell = 1000 for resolution of multiple peaks in the primordial temperature power spectrum, which enables us to probe the primordial power spectrum of adiabatic scalar perturbations with wavenumbers up to k \simeq 0.2 Mpc^{-1}. We find no evidence for deviation from power-law fluctuations over two decades in scale. Matter fluctuations inferred from the primordial temperature power spectrum evolve over cosmic time and can be used to predict the matter power spectrum at late times; we illustrate the overlap of the matter power inferred from CMB measurements (which probe the power spectrum in the linear regime) with existing probes of galaxy clustering, cluster abundances and weak lensing constraints on the primordial power. This highlights the range of scales probed by current measurements of the matter power spectrum.Comment: 10 pages, 5 figures, submitted to Ap

Planck 2015 results. XIII. Cosmological parameters

We present results based on full-mission Planck observations of temperature and polarization anisotropies of the CMB. These data are consistent with the six-parameter inflationary LCDM cosmology. From the Planck temperature and lensing data, for this cosmology we find a Hubble constant, H0= (67.8 +/- 0.9) km/s/Mpc, a matter density parameter Omega_m = 0.308 +/- 0.012 and a scalar spectral index with n_s = 0.968 +/- 0.006. (We quote 68% errors on measured parameters and 95% limits on other parameters.) Combined with Planck temperature and lensing data, Planck LFI polarization measurements lead to a reionization optical depth of tau = 0.066 +/- 0.016. Combining Planck with other astrophysical data we find N_ eff = 3.15 +/- 0.23 for the effective number of relativistic degrees of freedom and the sum of neutrino masses is constrained to < 0.23 eV. Spatial curvature is found to be |Omega_K| < 0.005. For LCDM we find a limit on the tensor-to-scalar ratio of r <0.11 consistent with the B-mode constraints from an analysis of BICEP2, Keck Array, and Planck (BKP) data. Adding the BKP data leads to a tighter constraint of r < 0.09. We find no evidence for isocurvature perturbations or cosmic defects. The equation of state of dark energy is constrained to w = -1.006 +/- 0.045. Standard big bang nucleosynthesis predictions for the Planck LCDM cosmology are in excellent agreement with observations. We investigate annihilating dark matter and deviations from standard recombination, finding no evidence for new physics. The Planck results for base LCDM are in agreement with BAO data and with the JLA SNe sample. However the amplitude of the fluctuations is found to be higher than inferred from rich cluster counts and weak gravitational lensing. Apart from these tensions, the base LCDM cosmology provides an excellent description of the Planck CMB observations and many other astrophysical data sets

Measurements of Secondary Cosmic Microwave Background Anisotropies with the South Pole Telescope

The South Pole Telescope is a 10m millimeter-wavelength telescope for finding galaxy clusters via the thermal Sunyaev-Zel'dovich (tSZ) effect. This thesis is divided into two parts. The first part describes the development of the kilopixel SPT-SZ receiver and the frequency-domain multiplexor (fMUX). The second part describes the first SPT power spectrum measurement and the first detection of the tSZ power spectrum. The SPT-SZ focal plane consists of 960 spiderweb coupled transition-edge sensors. Due to strong electro-thermal feedback, these devices have good sensitivity and linearity, though risk spontaneous oscillations. Adding heat capacity to these devices can ensure stability, so long as the loopgain, [special characters omitted], is less than Gint/G 0, the ratio between the thermal conductances linking the TES to the heat capacity and linking the heat capacity to the bath. I describe as experimental technique for measuring the internal thermal structure of these devices, allowing for rapid sensor evaluation. The fMUX readout system reduces wiring complexity in this receiver by AC-biasing each sensor at a unique frequency and sending signals from multiple bolometers along one pair of wires. The Series SQUID Arrays (SSAs) used to read changes in bolometer current are notably non-linear and extremely sensitive to ambient magnetic fields. The SSAs are housed in compact magnetic shielding modules which reduces their effective area to 80 mΦ0/gauss. The SSA are fedback with a flux-locked loop to improve their linearity and dynamic range, and decrease their input reactance. The FLL is bandwidth of 1 MHz with a measured loopgain of 10. In the current implementation, this bandwidth is limited between the SQUID input coil and other reactances, which I study in Chapter 4. In the second part of the thesis I present power spectrum measurements for the first 100 deg2 field observed by the SPT. On angular scales where the primary CMB anisotropy is dominant, ℓ [special characters omitted] 3000, the SPT power spectrum is consistent with the standard ΛCDM cosmology. On smaller scales, we see strong evidence for a point source contribution, consistent with a population of dusty, star-forming galaxies. I combine the 150 and 220 GHz data to remove the majority of the point source power, and use the point source subtracted spectrum to detect Sunyaev-Zel'dovich (SZ) power at 2.6 σ. At ℓ = 3000, the SZ power in the subtracted bandpowers is 4.2±1.5 μK2, which is significantly lower than the power predicted by a fiducial model using WMAP5 cosmological parameters

Measurements of Secodary Cosmic Microwave Background Anisotropies with the South Pole Telescope

The South Pole Telescope is a 10m millimeter-wavelength telescope for finding galaxy clusters via the thermal Sunyaev-Zel'dovich (tSZ) effect. This thesis is divided into two parts. The first part describes the development of the kilopixel SPT-SZ receiver and the frequency-domain multiplexor (fMUX). The second part describes the first SPT power spectrum measurement and the first detection of the tSZ power spectrum. The SPT-SZ focal plane consists of 960 spiderweb coupled transition-edge sensors. Due to strong electro-thermal feedback, these devices have good sensitivity and linearity, though risk spontaneous oscillations. Adding heat capacity to these devices can ensure stability, so long as the loopgain, $\mathcal{L}$, is less than $G_\textrm{int}/G_0$, the ratio between the thermal conductances linking the TES to the heat capacity and linking the heat capacity to the bath. I describe as experimental technique for measuring the internal thermal structure of these devices, allowing for rapid sensor evaluation. The fMUX readout system reduces wiring complexity in this receiver by AC-biasing each sensor at a unique frequency and sending signals from multiple bolometers along one pair of wires. The Series SQUID Arrays (SSAs) used to read changes in bolometer current are notably non-linear and extremely sensititve to ambient magnetic fields. The SSAs are housed in compact magnetic shielding modules which reduces their effective area to 80 $\textrm{m}\Phi_0/\textrm{gauss}$. The SSA are fedback with a flux-locked loop to improve their linearity and dynamic range, and decrease their input reactance. The FLL is bandwidth of 1 MHz with a measured loopgain of 10. In the current implementation, this bandwidth is limited between the SQUID input coil and other reactances, which I study in Chapter \ref{chap:fllstab}.In the second part of the thesis I present power spectrum measurements for the first 100~deg$^2$ field observed by the SPT. On angular scales where the primary CMB anisotropy is dominant, $\ell \lesssim 3000$, the SPT power spectrum is consistent with the standard $\Lambda$CDM cosmology. On smaller scales, we see strong evidence for a point source contribution, consisteThe South Pole Telescope is a 10m millimeter-wavelength telescope for finding galaxy clusters via the thermal Sunyaev-Zel'dovich (tSZ) effect. This thesis is divided into two parts. The first part describes the development of the kilopixel SPT-SZ receiver and the frequency-domain multiplexor (fMUX). The second part describes the first SPT power spectrum measurement and the first detection of the tSZ power spectrum. The SPT-SZ focal plane consists of 960 spiderweb coupled transition-edge sensors. Due to strong electro-thermal feedback, these devices have good sensitivity and linearity, though risk spontaneous oscillations. Adding heat capacity to these devices can ensure stability, so long as the loopgain, $\mathcal{L}$, is less than $G_\textrm{int}/G_0$, the ratio between the thermal conductances linking the TES to the heat capacity and linking the heat capacity to the bath. I describe as experimental technique for measuring the internal thermal structure of these devices, allowing for rapid sensor evaluation. The fMUX readout system reduces wiring complexity in this receiver by AC-biasing each sensor at a unique frequency and sending signals from multiple bolometers along one pair of wires. The Series SQUID Arrays (SSAs) used to read changes in bolometer current are notably non-linear and extremely sensititve to ambient magnetic fields. The SSAs are housed in compact magnetic shielding modules which reduces their effective area to 80 $\textrm{m}\Phi_0/\textrm{gauss}$. The SSA are fedback with a flux-locked loop to improve their linearity and dynamic range, and decrease their input reactance. The FLL is bandwidth of 1 MHz with a measured loopgain of 10. In the current implementation, this bandwidth is limited between the SQUID input coil and other reactances, which I study in Chapter \ref{chap:fllstab}.In the second part of the thesis I present power spectrum measurements for the first 100~deg$^2$ field observed by the SPT. On angular scales where the primary CMB anisotropy is dominant, $\ell \lesssim 3000$, the SPT power spectrum is consistent with the standard $\Lambda$CDM cosmology. On smaller scales, we see strong evidence for a point source contribution, consistent with a population of dusty, star-forming galaxies. I combine the 150 and 220$\,$GHz data to remove the majority of the point source power, and use the point source subtracted spectrum to detect Sunyaev-Zel'dovich (SZ) power at $2.6\,\sigma$. At $\ell=3000$, the SZ power in the subtracted bandpowers is $4.2\,$$\pm$$\,1.5\, \mu\rm{K}^2$, which is significantly lower than the power predicted by a fiducial model using WMAP5 cosmological parameters. t with a population of dusty, star-forming galaxies. I combine the 150 and 220$\,$GHz data to remove the majority of the point source power, and use the point source subtracted spectrum to detect Sunyaev-Zel'dovich (SZ) power at $2.6\,\sigma$. At $\ell=3000$, the SZ power in the subtracted bandpowers is $4.2\,$$\pm$$\,1.5\, \mu\rm{K}^2$, which is significantly lower than the power predicted by a fiducial model using WMAP5 cosmological parameters