A Study of Deep Learning Robustness Against Computation Failures

For many types of integrated circuits, accepting larger failure rates in computations can be used to improve energy efficiency. We study the performance of faulty implementations of certain deep neural networks based on pessimistic and optimistic models of the effect of hardware faults. After identifying the impact of hyperparameters such as the number of layers on robustness, we study the ability of the network to compensate for computational failures through an increase of the network size. We show that some networks can achieve equivalent performance under faulty implementations, and quantify the required increase in computational complexity

Modeling and Energy Optimization of LDPC Decoder Circuits with Timing Violations

This paper proposes a "quasi-synchronous" design approach for signal processing circuits, in which timing violations are permitted, but without the need for a hardware compensation mechanism. The case of a low-density parity-check (LDPC) decoder is studied, and a method for accurately modeling the effect of timing violations at a high level of abstraction is presented. The error-correction performance of code ensembles is then evaluated using density evolution while taking into account the effect of timing faults. Following this, several quasi-synchronous LDPC decoder circuits based on the offset min-sum algorithm are optimized, providing a 23%-40% reduction in energy consumption or energy-delay product, while achieving the same performance and occupying the same area as conventional synchronous circuits.Comment: To appear in IEEE Transactions on Communication

Analyse de la croissance et des changements structurels dans l'emploi des agglomérations et des régions métropolitaines de recensement du Canada : une approche par le biais de l'économie géographique

L'objectif de ce mémoire est de mieux comprendre les changements survenus dans la structure industrielle au Canada entre les années 1971 et 2001. Nous tenterons aussi d'extraire les effets de certaines variables sur le niveau d'emploi dans les agglomérations de recensements (AR) du Canada. Pour ce faire, nous utilisons des données portant sur l'emploi dans les AR entre les années 1971 et 2001. L'emploi y est divisé par industrie et par catégorie pour chacune des agglomérations. Nous retrouvons dans la partie statistique des indices d'inégalités qui démontrent les inégalités de la distribution de l'emploi présent dans la structure industrielle. Nous construisons aussi un indice indiquant le niveau de diversification des industries pour chaque AR. L'un des phénomènes remarqués est que les villes de plus grande taille sont davantage diversifiées que celles de plus petite taille. Ensuite, nous avons aussi construit un indice servant à mesurer le niveau de dispersion des industries et des types de fonctions dans l'espace. Dans un second temps, nous réalisons une analyse économétrique des variables ayant un impact sur le niveau de l'emploi. Nous trouvons que la population et la proximité des États-Unis ont un impact positif sur le niveau de l'emploi. Nous nous intéressons aussi aux raisons qui influencent le type d'industrie présent dans une AR. Ensuite, nous nous penchons sur les déterminants de la présence de certaines catégories de fonctions d'emploi dans les AR. Enfin, la dernière analyse nous permet d'expliquer les variations de la dispersion des industries dans l'espace. Elle nous permet de constater que les industries des services ont un indice d'inégalité de la dispersion 5 pour cent plus bas que les industries du secteur des matières premières

VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing

The hardware implementation of deep neural networks (DNNs) has recently received tremendous attention: many applications in fact require high-speed operations that suit a hardware implementation. However, numerous elements and complex interconnections are usually required, leading to a large area occupation and copious power consumption. Stochastic computing has shown promising results for low-power area-efficient hardware implementations, even though existing stochastic algorithms require long streams that cause long latencies. In this paper, we propose an integer form of stochastic computation and introduce some elementary circuits. We then propose an efficient implementation of a DNN based on integral stochastic computing. The proposed architecture has been implemented on a Virtex7 FPGA, resulting in 45% and 62% average reductions in area and latency compared to the best reported architecture in literature. We also synthesize the circuits in a 65 nm CMOS technology and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared to the binary radix implementation at the same misclassification rate. Due to fault-tolerant nature of stochastic architectures, we also consider a quasi-synchronous implementation which yields 33% reduction in energy consumption w.r.t. the binary radix implementation without any compromise on performance.Comment: 11 pages, 12 figure

Northwestern Pacific typhoon intensity controlled by changes in ocean temperatures.

Dominant climatic factors controlling the lifetime peak intensity of typhoons are determined from six decades of Pacific typhoon data. We find that upper ocean temperatures in the low-latitude northwestern Pacific (LLNWP) and sea surface temperatures in the central equatorial Pacific control the seasonal average lifetime peak intensity by setting the rate and duration of typhoon intensification, respectively. An anomalously strong LLNWP upper ocean warming has favored increased intensification rates and led to unprecedentedly high average typhoon intensity during the recent global warming hiatus period, despite a reduction in intensification duration tied to the central equatorial Pacific surface cooling. Continued LLNWP upper ocean warming as predicted under a moderate [that is, Representative Concentration Pathway (RCP) 4.5] climate change scenario is expected to further increase the average typhoon intensity by an additional 14% by 2100

Layerwise Noise Maximisation to Train Low-Energy Deep Neural Networks

Deep neural networks (DNNs) depend on the storage of a large number of parameters, which consumes an important portion of the energy used during inference. This paper considers the case where the energy usage of memory elements can be reduced at the cost of reduced reliability. A training algorithm is proposed to optimize the reliability of the storage separately for each layer of the network, while incurring a negligible complexity overhead compared to a conventional stochastic gradient descent training. For an exponential energy-reliability model, the proposed training approach can decrease the memory energy consumption of a DNN with binary parameters by 3.3$\times$ at isoaccuracy, compared to a reliable implementation.Comment: To be presented at AICAS 202

Sharpness-Aware Training for Accurate Inference on Noisy DNN Accelerators

Energy-efficient deep neural network (DNN) accelerators are prone to non-idealities that degrade DNN performance at inference time. To mitigate such degradation, existing methods typically add perturbations to the DNN weights during training to simulate inference on noisy hardware. However, this often requires knowledge about the target hardware and leads to a trade-off between DNN performance and robustness, decreasing the former to increase the latter. In this work, we show that applying sharpness-aware training by optimizing for both the loss value and the loss sharpness significantly improves robustness to noisy hardware at inference time while also increasing DNN performance. We further motivate our results by showing a high correlation between loss sharpness and model robustness. We show superior performance compared to injecting noise during training and aggressive weight clipping on multiple architectures, optimizers, datasets, and training regimes without relying on any assumptions about the target hardware. This is observed on a generic noise model as well as on accurate noise simulations from real hardware.Comment: Preprin

La composition interactive immersive : une approche participative à la composition électroacoustique

Mémoire en recherche-créationL’évolution constante des moyens technologiques à la disposition des artistes facilite aujourd'hui la création de systèmes interactifs faisant appel à la participation du public. L’interactivité peut être utilisée pour accroitre le sentiment d’immersion d’une œuvre lorsqu’elle s’intègre de façon transparente avec les autres éléments, qu’ils soient sonores, narratifs ou scénographiques. Ce mémoire aborde premièrement les concepts d’interactivité et d’immersion en musique, pour ensuite traiter des deux principaux projets réalisés dans le cadre de cette recherche-création. Le premier, Evo[1], est une pièce électroacoustique de concert qui requiert la participation du public afin de lui donner forme. En interagissant avec un site web, les utilisateurs contrôlent les objets sonores qui forgent la forme de la pièce. La seconde œuvre proposée, La démonstration, est une pièce-installation pour un seul participant. L’utilisateur interagit avec des objets scénographiques qui influencent la forme et le contenu narratif. L'univers sonore y est renouvelé à chaque participation grâce à l’intégration d’algorithmes génératifs et d’une importante banque de pistes musicales précomposées. Ces deux projets ont permis de mettre à l’épreuve différents types processus participatifs au sein de compositions électroacoustiques.The constant evolution of the technological means available to artists allows for the creation of interactive systems that integrate the audience’s participation. Interactivity can also improve the immersive quality of a piece when it is integrated in a transparent fashion with the other elements in play, be they related to sound, narrativity or scenography. This thesis first discusses the concepts related to interactivity and immersion in music, before reviewing the two main projects created for this research-creation. The first one, Evo[1], is an electroacoustic concert piece requiring the participation of the audience in order to give it its intended shape. By interacting with a website, the users can control the sound objects that make up the piece. The second work, La démonstration, is a piece-installation for a single participant. The user interacts with scenographic objects which influence the narrative shape and the content. This sonic universe is renewed with each performance owing to the integration of generative algorithms, along with an important pre-composed musical library. These two projects allowed for the experimentation of different types of interactive processes in electroacoustic compositions

Multiple equilibria and low-frequency variability of wind-driven ocean models

Thesis (Ph. D.)--Joint Program in Physical Oceanography (Massachusetts Institute of Technology, Dept. of Earth, Atmospheric, and Planetary Sciences; and the Woods Hole Oceanographic Institution), 1998.Includes bibliographical references (leaves 156-158).by François W. Primeau.Ph.D

Learning Energy-Efficient Hardware Configurations for Massive MIMO Beamforming

Hybrid beamforming (HBF) and antenna selection are promising techniques for improving the energy efficiency~(EE) of massive multiple-input multiple-output~(mMIMO) systems. However, the transmitter architecture may contain several parameters that need to be optimized, such as the power allocated to the antennas and the connections between the antennas and the radio frequency chains. Therefore, finding the optimal transmitter architecture requires solving a non-convex mixed integer problem in a large search space. In this paper, we consider the problem of maximizing the EE of fully digital precoder~(FDP) and hybrid beamforming~(HBF) transmitters. First, we propose an energy model for different beamforming structures. Then, based on the proposed energy model, we develop an unsupervised deep learning method to maximize the EE by designing the transmitter configuration for FDP and HBF. The proposed deep neural networks can provide different trade-offs between spectral efficiency and energy consumption while adapting to different numbers of active users. Finally, to ensure that the proposed method can be implemented in practice, we investigate the ability of the model to be trained exclusively using imperfect channel state information~(CSI), both for the input to the deep learning model and for the calculation of the loss function. Simulation results show that the proposed solutions can outperform conventional methods in terms of EE while being trained with imperfect CSI. Furthermore, we show that the proposed solutions are less complex and more robust to noise than conventional methods.Comment: This preprint comprises 15 pages and features 15 figures. Copyright may be transferred without notic