Doctor of Philosophy

dissertationDeep Neural Networks (DNNs) are the state-of-art solution in a growing number of tasks including computer vision, speech recognition, and genomics. However, DNNs are computationally expensive as they are carefully trained to extract and abstract features from raw data using multiple layers of neurons with millions of parameters. In this dissertation, we primarily focus on inference, e.g., using a DNN to classify an input image. This is an operation that will be repeatedly performed on billions of devices in the datacenter, in self-driving cars, in drones, etc. We observe that DNNs spend a vast majority of their runtime to runtime performing matrix-by-vector multiplications (MVM). MVMs have two major bottlenecks: fetching the matrix and performing sum-of-product operations. To address these bottlenecks, we use in-situ computing, where the matrix is stored in programmable resistor arrays, called crossbars, and sum-of-product operations are performed using analog computing. In this dissertation, we propose two hardware units, ISAAC and Newton.In ISAAC, we show that in-situ computing designs can outperform DNN digital accelerators, if they leverage pipelining, smart encodings, and can distribute a computation in time and space, within crossbars, and across crossbars. In the ISAAC design, roughly half the chip area/power can be attributed to the analog-to-digital conversion (ADC), i.e., it remains the key design challenge in mixed-signal accelerators for deep networks. In spite of the ADC bottleneck, ISAAC is able to out-perform the computational efficiency of the state-of-the-art design (DaDianNao) by 8x. In Newton, we take advantage of a number of techniques to address ADC inefficiency. These techniques exploit matrix transformations, heterogeneity, and smart mapping of computation to the analog substrate. We show that Newton can increase the efficiency of in-situ computing by an additional 2x. Finally, we show that in-situ computing, unfortunately, cannot be easily adapted to handle training of deep networks, i.e., it is only suitable for inference of already-trained networks. By improving the efficiency of DNN inference with ISAAC and Newton, we move closer to low-cost deep learning that in turn will have societal impact through self-driving cars, assistive systems for the disabled, and precision medicine

High Temperature Silicon Carbide Mixed-signal Circuits for Integrated Control and Data Acquisition

Wide bandgap semiconductor materials such as gallium nitride (GaN) and silicon carbide have grown in popularity as a substrate for power devices for high temperature and high voltage applications over the last two decades. Recent research has been focused on the design of integrated circuits for protection and control in these wide bandgap materials. The ICs developed in SiC and GaN can not only complement the power devices in high voltage and high frequency applications, but can also be used for standalone high temperature control and data acquisition circuitry. This dissertation work aims to explore the possibilities in high temperature and wide bandgap circuit design by developing a host of mixed-signal circuits that can be used for control and data acquisition. These include a family of current-mode signal processing circuits, general purpose amplifiers and comparators, and 8-bit data converters. The signal processing circuits along with amplifiers and comparators are then used to develop an integrated mixed-signal controller for a DC-DC flyback converter in a microinverter application. The 8-bit SAR ADC and the 8-bit R-2R ladder DAC open up the possibility of a remote data acquisition and control system in high temperature environments. The circuits and systems presented here offer a gateway to great opportunities in high temperature and power electronics ICs in SiC

The simulation and optimization of steady state process circuits by means of artificial neural networks

Dissertation (Ph.D.) -- University of Stellenbosch, 1993.ENGLISH ABASTRACT: Since the advent of modern process industries engineers engaged in the modelling and simulation of chemical and metallurgical processes have had to contend with two important dilemmas. The first concerns the ill-defined nature of the processes they have to describe, while the second relates to the limitations of prevailing computational resources. Current process simulation procedures are based on explicit process models in one form or another. Many chemical and metallurgical processes are not amenable to this kind of modelling however, and can not be incorporated effectively into current commercial process simulators. As a result many process operations do not benefit from the use of predictive models and simulation routines and plants are often poorly designed and run, ultimately leading to considerable losses in revenue. In addition to this dilemma, process simulation is in a very real way constrained by available computing resources. The construction of adequate process models is essentially meaningless if these models can not be solved efficiently - a situation occurring all too often. In the light of these problems, it is thus not surprising that connectionist systems or neural network methods are singularly attractive to process engineers, since they provide a powerful means of addressing both these dilemmas. These nets can form implicit process models through learning by example, and also serve as a vehicle for parallel supercomputing devices. In this dissertation the use of artificial neural networks for the steady state modelling and optimization of chemical and metallurgical process circuits is consequently investigated. The first chapter is devoted to a brief overview of the simulation of chemical and metallurgical plants by conventional methods, as well as the evolution and impact of computer technology and artificial intelligence on the process industries. Knowledge of the variance covariance matrices of process data is of paramount importance to data reconciliation and gross error detection problems, and although various methods can be employed to estimate these often unknown variances, it is shown in the second chapter that the use of feedforward neural nets can be more efficient than conventional strategies. In the following chapter the important problem of gross error detection in process data is addressed. Existing procedures are statistical and work well for systems subject to linear constraints. Non-linear constraints are not handled well by these methods and it is shown that back propagation neural nets can be trained to detect errors in process systems, regardless of the nature of the constraints. In the fourth chapter the exploitation of the massively parallel information processing structures of feedback neural nets in the optimization of process data reconciliation problems is investigated. Although effective and sophisticated algorithms are available for these procedures, there is an ever present demand for computational devices or routines that can accommodate progressively larger or more complex problems. Simulations indicate that neural nets can be efficient instruments for the implementation of parallel strategies for the optimization of such problems. In the penultimate chapter a gold reduction plant and a leach plant are modelled with neural nets and the models shown to be considerably better than the linear regression models used in practice. The same technique is also demonstrated with the modelling of an apatite flotation plant. Neural nets can also be used in conjunction with other methods and in the same chapter the steady state simulation and optimization of a gravity separation circuit with the use of two linear programming models and a neural net are described.AFRIKAANSE OPSOMMING: Sedert die ontstaan van prosesingenieurswese, het ingenieurs gemoeid met die modellering en simulasie van chemiese en metallurgiese prosesse met twee belangrike dilemmas te kampe gehad. Die eerste het te make met die swakgedefinieerde aard van chemiese prosesse, wat die beskrywing en dus ook die beheer daarvan kompliseer, terwyl die tweede verband hou met die beperkinge van huidige berekeningsmiddele. Die prosesse wat tans gebruik word om chemiese prosesse te simuleer is gebaseer op eksplisiete prosesmodelle van een of ander aard. Baie chemiese en metallurgiese prosesse kan egter nie op 'n eksplisiete wyse gemodelleer word nie, en kan gevolglik ook nie doeltreffendheid deur kommersiële prosessimulators beskryf word nie. Die bedryf van baie prosesse vind derhalwe nie baat by die gebruik van voorspellende modelle en simulasie-algoritmes nie en aanlegte word dikwels suboptimaal ontwerp en bedryf, wat uiteindelik tot aansienlike geldelike verliese kan lei. Prosessimulasie word op die koop toe ook beperk deur die beskikbaarheid van berekeningsfasiliteite. Die konstruksie van geskikte prosesmodelle hou geen voordeel in as hierdie modelle nie doeltreffendheid opgelos kan word nie. Teen die agtergrond van hierdie probleme is dit nie verrassend dat neurale netwerke 'n besondere bekoring vir prosesingenieurs inhou nie, aangesien hulle beide hierdie dilemmas aanspreek. Hierdie nette kan implisiete prosesmodelle konstrueer deur te leer van voorbeelde en dien ook as 'n raamwerk vir parallelle superrekenaars. In hierdie proefskrif word die gebruik van kunsmatige neurale netwerke vir gestadigde toestandsmodellering en optimering van chemiese en metallurgiese prosesse gevolglik ondersoek. Die eerste hoofstuk word gewy aan 'n kort oorsig oor die simulasie van chemiese en metallurgiese aanlegte met konvensionele tegnieke, asook die ontwikkeling en impak van rekenaartegnologie en skynintelligensie in die prosesnywerhede. Kennis van die variansie-kovariansie-matrikse van prosesdata is van kardinale belang vir datarekonsiliasie en die identifikasie en eliminasie van sistematiese foute en alhoewel verskeie metodes aangewend kan word om hierdie onbekende variansies te beraam, word daar in die tweede hoofstuk getoon dat die gebruik van neurale netwerke meer doeltreffend is as konvensionele strategieë. In die volgende hoofstuk word die belangrike probleem van sistematiese foutopsporing in prosesdata ondersoek. Bestaande prosedures is statisties van aard en werk goed vir stelsels onderworpe aan lineêre beperkinge. Nie-lineêre beperkinge kan nie doeltreffend deur hierdie prosedures hanteer word nie en daar word gewys dat terugwaarts-propagerende nette geleer kan word om sulke foute in prosessisteme op te spoor, ongeag die aard van die beperkinge. In die vierde hoofstuk word die rekonsiliasie van prosesdata met behulp van massiewe parallelle dataverwerkingstrukture soos verteenwoordig deur terugvoerende neurale nette, ondersoek. Alhoewel doeltreffende en gesofistikeerde algoritmes beskikbaar is vir die optimering van die tipe probleme, is daar 'n onversadigbare aanvraag na rekenaars wat groter en meer komplekse stelsels kan akkommodeer. Simulasie dui aan dat neurale nette effektief aangewend kan word vir die implementering van parallelle strategieë vir dié tipe optimeringsprobleme. In die voorlaaste hoof stuk word die konneksionistiese modellering van 'n goudreduksie- en 'n logingsaanleg beskryf en daar word aangetoon dat die neurale netwerk-modelle aansienlik beter resultate lewer as die linneêre regressie modelle wat in die praktyk gebruik word. Dieselfde tegnieke vir die modellering van 'n flottasie-aanleg vir apatiet word ook bespreek. Neural nette kan ook saam met ander metodes aangewend word en in dieselfde hoofstuk word die gebruik van twee lineêre programmeringsmodelle en 'n neural net om 'n gravitasieskeidingsbaan onder gestadigde toestande te simuleer en te optimeer, beskryf