Number Systems for Deep Neural Network Architectures: A Survey

Deep neural networks (DNNs) have become an enabling component for a myriad of artificial intelligence applications. DNNs have shown sometimes superior performance, even compared to humans, in cases such as self-driving, health applications, etc. Because of their computational complexity, deploying DNNs in resource-constrained devices still faces many challenges related to computing complexity, energy efficiency, latency, and cost. To this end, several research directions are being pursued by both academia and industry to accelerate and efficiently implement DNNs. One important direction is determining the appropriate data representation for the massive amount of data involved in DNN processing. Using conventional number systems has been found to be sub-optimal for DNNs. Alternatively, a great body of research focuses on exploring suitable number systems. This article aims to provide a comprehensive survey and discussion about alternative number systems for more efficient representations of DNN data. Various number systems (conventional/unconventional) exploited for DNNs are discussed. The impact of these number systems on the performance and hardware design of DNNs is considered. In addition, this paper highlights the challenges associated with each number system and various solutions that are proposed for addressing them. The reader will be able to understand the importance of an efficient number system for DNN, learn about the widely used number systems for DNN, understand the trade-offs between various number systems, and consider various design aspects that affect the impact of number systems on DNN performance. In addition, the recent trends and related research opportunities will be highlightedComment: 28 page

Use of RNS Based Pseudo Noise Sequence in DS-CDMA and 3G WCDMA

Code Division Multiple Access (CDMA) based on Spread Signal (SS) has emerged as one of the most important multiple access technologies for Second Generation (2G) and Third Generation (3G) wireless communication systems by its wide applications in many important mobile cellular standards. CDMA technique relies on spreading codes to separate dierent users or channels and its properties will govern the performance of the system. So many of the problems of communication systems based on CDMA technology stem from the spreading codes/sequences, which includes two sub-categories, one being the orthogonal codes, such as Walsh Hadamard (WH) codes and Orthogonal Variable Spreading Factor (OVSF) codes, and the other being pseudo-noise or Pseudo Random (PN) sequences, such as Gold sequences, Kasami sequences, m-sequences, etc. In this thesis a PN sequence generation based on Residue Arithmetic is investigated with an eort to improve the performance of existing interference-limited CDMA technology for mobile cellular systems. This interference-limited performance is due to the fact that all the existing CDMA codes used in mobile cellular standards does not consider external interferences, multipath propagation, Doppler eect etc. So the non-ideal correlation properties of the pseudo-random CDMA codes results in MAI when used in a multi-user system. The PN codes appear random yet they are completely deterministic in nature with a small set of initial conditions. Consequently this work focuses on CDMA code design approach based on Residue Number System (RNS) which should take into account as many real operational conditions as possible and to maintain a suciently large code set size.First, the thesis reviews RNS, DS-CDMA and CDMA codes that are already implemented in various mobile cellular standards. Then the new PN Sequencegenerator design based on RNS is discussed. Comparison of the generated PN sequence with respect to other standard sequence is done in terms of number of codes and correlation properties. Monte-Carlo simulations with the generated sequence are carried out for performance analysis under multi-path environment. The system has been evaluated in AWGN, Rayleigh Fading channel and dierent Stationary Multipath Channels for dierent cross-correlation threshold. It is known that orthogonal Codes are used to multiplex more than one signal for downlink transmission over cellular networks. This downlink transmission is prone to self interference caused by the loss of orthogonality between spreading codes due to multipath propagation. This issue is investigated in detail with respect to WCDMA standards, which is very good representative for CDMA based 3G mobile cellular systems where the channelization code is OVSF code. The code assignment blocking (CAB) (If a particular code in the tree is used in a cell, then all its parent codes and child codes should not be used in the same cell to maintain orthogonality among the users) problem of OVSF codes restricts the number of available codes for a given cell. Since the 3rd generation WCDMA mobile communication systems apply the same multiple access technique, the generated sequence can also be the channelization code for downlink WCDMA system to mitigate the the same. The performance of the system is compared with Walsh Hadamard code over multipath AWGN and dierent Fading channels. This thesis work shows that RNS based PN sequence has enhanced performance to that of other CDMA codes by comparing the bit error probability in multi- user and multipath environment thus contributing a little towards the evolution of next generation CDMA technology

Perception-motivated parallel algorithms for haptics

Negli ultimi anni l\u2019utilizzo di dispositivi aptici, atti cio\ue8 a riprodurre l\u2019interazione fisica con l\u2019ambiente remoto o virtuale, si sta diffondendo in vari ambiti della robotica e dell\u2019informatica, dai videogiochi alla chirurgia robotizzata eseguita in teleoperazione, dai cellulari alla riabilitazione. In questo lavoro di tesi abbiamo voluto considerare nuovi punti di vista sull\u2019argomento, allo scopo di comprendere meglio come riportare l\u2019essere umano, che \ue8 l\u2019unico fruitore del ritorno di forza, tattile e di telepresenza, al centro della ricerca sui dispositivi aptici. Allo scopo ci siamo focalizzati su due aspetti: una manipolazione del segnale di forza mutuata dalla percezione umana e l\u2019utilizzo di architetture multicore per l\u2019implementazione di algoritmi aptici e robotici. Con l\u2019aiuto di un setup sperimentale creato ad hoc e attraverso l\u2019utilizzo di un joystick con ritorno di forza a 6 gradi di libert\ue0, abbiamo progettato degli esperimenti psicofisici atti all\u2019identificazione di soglie differenziali di forze/coppie nel sistema mano-braccio. Sulla base dei risultati ottenuti abbiamo determinato una serie di funzioni di scalatura del segnale di forza, una per ogni grado di libert\ue0, che permettono di aumentare l\u2019abilit\ue0 umana nel discriminare stimoli differenti. L\u2019utilizzo di tali funzioni, ad esempio in teleoperazione, richiede la possibilit\ue0 di variare il segnale di feedback e il controllo del dispositivo sia in relazione al lavoro da svolgere, sia alle peculiari capacit\ue0 dell\u2019utilizzatore. La gestione del dispositivo deve quindi essere in grado di soddisfare due obbiettivi tendenzialmente in contrasto, e cio\ue8 il raggiungimento di alte prestazioni in termini di velocit\ue0, stabilit\ue0 e precisione, abbinato alla flessibilit\ue0 tipica del software. Una soluzione consiste nell\u2019affidare il controllo del dispositivo ai nuovi sistemi multicore che si stanno sempre pi\uf9 prepotentemente affacciando sul panorama informatico. Per far ci\uf2 una serie di algoritmi consolidati deve essere portata su sistemi paralleli. In questo lavoro abbiamo dimostrato che \ue8 possibile convertire facilmente vecchi algoritmi gi\ue0 implementati in hardware, e quindi intrinsecamente paralleli. Un punto da definire rimane per\uf2 quanto costa portare degli algoritmi solitamente descritti in VLSI e schemi in un linguaggio di programmazione ad alto livello. Focalizzando la nostra attenzione su un problema specifico, la pseudoinversione di matrici che \ue8 presente in molti algoritmi di dinamica e cinematica, abbiamo mostrato che un\u2019attenta progettazione e decomposizione del problema permette una mappatura diretta sulle unit\ue0 di calcolo disponibili. In aggiunta, l\u2019uso di parallelismo a livello di dati su macchine SIMD permette di ottenere buone prestazioni utilizzando semplici operazioni vettoriali come addizioni e shift. Dato che di solito tali istruzioni fanno parte delle implementazioni hardware la migrazione del codice risulta agevole. Abbiamo testato il nostro approccio su una Sony PlayStation 3 equipaggiata con un processore IBM Cell Broadband Engine.In the last years the use of haptic feedback has been used in several applications, from mobile phones to rehabilitation, from video games to robotic aided surgery. The haptic devices, that are the interfaces that create the stimulation and reproduce the physical interaction with virtual or remote environments, have been studied, analyzed and developed in many ways. Every innovation in the mechanics, electronics and technical design of the device it is valuable, however it is important to maintain the focus of the haptic interaction on the human being, who is the only user of force feedback. In this thesis we worked on two main topics that are relevant to this aim: a perception based force signal manipulation and the use of modern multicore architectures for the implementation of the haptic controller. With the help of a specific experimental setup and using a 6 dof haptic device we designed a psychophysical experiment aimed at identifying of the force/torque differential thresholds applied to the hand-arm system. On the basis of the results obtained we determined a set of task dependent scaling functions, one for each degree of freedom of the three-dimensional space, that can be used to enhance the human abilities in discriminating different stimuli. The perception based manipulation of the force feedback requires a fast, stable and configurable controller of the haptic interface. Thus a solution is to use new available multicore architectures for the implementation of the controller, but many consolidated algorithms have to be ported to these parallel systems. Focusing on specific problem, i.e. the matrix pseudoinversion, that is part of the robotics dynamic and kinematic computation, we showed that it is possible to migrate code that was already implemented in hardware, and in particular old algorithms that were inherently parallel and thus not competitive on sequential processors. The main question that still lies open is how much effort is required in order to write these algorithms, usually described in VLSI or schematics, in a modern programming language. We show that a careful task decomposition and design permit a mapping of the code on the available cores. In addition, the use of data parallelism on SIMD machines can give good performance when simple vector instructions such as add and shift operations are used. Since these instructions are present also in hardware implementations the migration can be easily performed. We tested our approach on a Sony PlayStation 3 game console equipped with IBM Cell Broadband Engine processor