Number Systems for Deep Neural Network Architectures: A Survey

Deep neural networks (DNNs) have become an enabling component for a myriad of artificial intelligence applications. DNNs have shown sometimes superior performance, even compared to humans, in cases such as self-driving, health applications, etc. Because of their computational complexity, deploying DNNs in resource-constrained devices still faces many challenges related to computing complexity, energy efficiency, latency, and cost. To this end, several research directions are being pursued by both academia and industry to accelerate and efficiently implement DNNs. One important direction is determining the appropriate data representation for the massive amount of data involved in DNN processing. Using conventional number systems has been found to be sub-optimal for DNNs. Alternatively, a great body of research focuses on exploring suitable number systems. This article aims to provide a comprehensive survey and discussion about alternative number systems for more efficient representations of DNN data. Various number systems (conventional/unconventional) exploited for DNNs are discussed. The impact of these number systems on the performance and hardware design of DNNs is considered. In addition, this paper highlights the challenges associated with each number system and various solutions that are proposed for addressing them. The reader will be able to understand the importance of an efficient number system for DNN, learn about the widely used number systems for DNN, understand the trade-offs between various number systems, and consider various design aspects that affect the impact of number systems on DNN performance. In addition, the recent trends and related research opportunities will be highlightedComment: 28 page

Feasibility study of an Integrated Program for Aerospace vehicle Design (IPAD). Volume 4: IPAD system design

The computing system design of IPAD is described and the requirements which form the basis for the system design are discussed. The system is presented in terms of a functional design description and technical design specifications. The functional design specifications give the detailed description of the system design using top-down structured programming methodology. Human behavioral characteristics, which specify the system design at the user interface, security considerations, and standards for system design, implementation, and maintenance are also part of the technical design specifications. Detailed specifications of the two most common computing system types in use by the major aerospace companies which could support the IPAD system design are presented. The report of a study to investigate migration of IPAD software between the two candidate 3rd generation host computing systems and from these systems to a 4th generation system is included

A computer-aided design for digital filter implementation

Imperial Users onl

Modern Computer Arithmetic (version 0.5.1)

This is a draft of a book about algorithms for performing arithmetic, and their implementation on modern computers. We are concerned with software more than hardware - we do not cover computer architecture or the design of computer hardware. Instead we focus on algorithms for efficiently performing arithmetic operations such as addition, multiplication and division, and their connections to topics such as modular arithmetic, greatest common divisors, the Fast Fourier Transform (FFT), and the computation of elementary and special functions. The algorithms that we present are mainly intended for arbitrary-precision arithmetic. They are not limited by the computer word size, only by the memory and time available for the computation. We consider both integer and real (floating-point) computations. The book is divided into four main chapters, plus an appendix. Our aim is to present the latest developments in a concise manner. At the same time, we provide a self-contained introduction for the reader who is not an expert in the field, and exercises at the end of each chapter. Chapter titles are: 1, Integer Arithmetic; 2, Modular Arithmetic and the FFT; 3, Floating-Point Arithmetic; 4, Elementary and Special Function Evaluation; 5 (Appendix), Implementations and Pointers. The book also contains a bibliography of 236 entries, index, summary of notation, and summary of complexities.Comment: Preliminary version of a book to be published by Cambridge University Press. xvi+247 pages. Cite as "Modern Computer Arithmetic, Version 0.5.1, 5 March 2010". For further details, updates and errata see http://wwwmaths.anu.edu.au/~brent/pub/pub226.html or http://www.loria.fr/~zimmerma/mca/pub226.htm

An Introduction to Programming for Bioscientists: A Python-based Primer

Computing has revolutionized the biological sciences over the past several decades, such that virtually all contemporary research in the biosciences utilizes computer programs. The computational advances have come on many fronts, spurred by fundamental developments in hardware, software, and algorithms. These advances have influenced, and even engendered, a phenomenal array of bioscience fields, including molecular evolution and bioinformatics; genome-, proteome-, transcriptome- and metabolome-wide experimental studies; structural genomics; and atomistic simulations of cellular-scale molecular assemblies as large as ribosomes and intact viruses. In short, much of post-genomic biology is increasingly becoming a form of computational biology. The ability to design and write computer programs is among the most indispensable skills that a modern researcher can cultivate. Python has become a popular programming language in the biosciences, largely because (i) its straightforward semantics and clean syntax make it a readily accessible first language; (ii) it is expressive and well-suited to object-oriented programming, as well as other modern paradigms; and (iii) the many available libraries and third-party toolkits extend the functionality of the core language into virtually every biological domain (sequence and structure analyses, phylogenomics, workflow management systems, etc.). This primer offers a basic introduction to coding, via Python, and it includes concrete examples and exercises to illustrate the language's usage and capabilities; the main text culminates with a final project in structural bioinformatics. A suite of Supplemental Chapters is also provided. Starting with basic concepts, such as that of a 'variable', the Chapters methodically advance the reader to the point of writing a graphical user interface to compute the Hamming distance between two DNA sequences.Comment: 65 pages total, including 45 pages text, 3 figures, 4 tables, numerous exercises, and 19 pages of Supporting Information; currently in press at PLOS Computational Biolog

ARITHMETIC LOGIC UNIT ARCHITECTURES WITH DYNAMICALLY DEFINED PRECISION

Modern central processing units (CPUs) employ arithmetic logic units (ALUs) that support statically defined precisions, often adhering to industry standards. Although CPU manufacturers highly optimize their ALUs, industry standard precisions embody accuracy and performance compromises for general purpose deployment. Hence, optimizing ALU precision holds great potential for improving speed and energy efficiency. Previous research on multiple precision ALUs focused on predefined, static precisions. Little previous work addressed ALU architectures with customized, dynamically defined precision. This dissertation presents approaches for developing dynamic precision ALU architectures for both fixed-point and floating-point to enable better performance, energy efficiency, and numeric accuracy. These new architectures enable dynamically defined precision, including support for vectorization. The new architectures also prevent performance and energy loss due to applying unnecessarily high precision on computations, which often happens with statically defined standard precisions. The new ALU architectures support different precisions through the use of configurable sub-blocks, with this dissertation including demonstration implementations for floating point adder, multiply, and fused multiply-add (FMA) circuits with 4-bit sub-blocks. For these circuits, the dynamic precision ALU speed is nearly the same as traditional ALU approaches, although the dynamic precision ALU is nearly twice as large

Primary Structure and Solution Conditions Determine Conformational Ensemble Properties of Intrinsically Disordered Proteins

Intrinsically disordered proteins (IDPs) are a class of proteins that do not exhibit well-defined three-dimensional structures. The absence of structure is intrinsic to their amino acid sequences, which are characterized by low hydrophobicity and high net charge per residue compared to folded proteins. Contradicting the classic structure-function paradigm, IDPs are capable of interacting with high specificity and affinity, often acquiring order in complex with protein and nucleic acid binding partners. This phenomenon is evident during cellular activities involving IDPs, which include transcriptional and translational regulation, cell cycle control, signal transduction, molecular assembly, and molecular recognition. Although approximately 30% of eukaryotic proteomes are intrinsically disordered, the nature of IDP conformational ensembles remains unclear. In this dissertation, we describe relationships connecting characteristics of IDP conformational ensembles to their primary structures and solution conditions. Using molecular simulations and fluorescence experiments on a set of base-rich IDPs, we find that net charge per residue segregates conformational ensembles along a globule-to-coil transition. Speculatively generalizing this result, we propose a phase diagram that predicts an IDP\u27s average size and shape based on sequence composition and use it to generate hypotheses for a broad set of intrinsically disordered regions (IDRs). Simulations reveal that acid-rich IDRs, unlike their oppositely charged base-rich counterparts, exhibit disordered globular ensembles despite intra-chain repulsive electrostatic interactions. This apparent asymmetry is sensitive to simulation parameters for representing alkali and halide salt ions, suggesting that solution conditions modulate IDP conformational ensembles. We refine the ion parameters using a calibration procedure that relies exclusively on crystal lattice properties. Simulations with these parameters recover swollen coil behavior for acid-rich IDRs, but also uncover a dependence on sequence patterning for polyampholytic IDPs. These contributions initiate an endeavor to elucidate general principles that enable prediction of an IDP\u27s conformational ensemble based on primary structure and solution conditions, a goal analogous to structure prediction for folded proteins. Such principles would provide a molecular basis for understanding the roles of IDPs in physiology and pathophysiology, guide development of agents that modulate their behavior, and enable their rational design from chosen specifications

Application of the residue number system to the matrix multiplication problem

Due to the character of the original source materials and the nature of batch digitization, quality control issues may be present in this document. Please report any quality issues you encounter to [email protected], referencing the URI of the item.Includes bibliographical references.Not availabl