Deep in-memory computing

There is much interest in embedding data analytics into sensor-rich platforms such as wearables, biomedical devices, autonomous vehicles, robots, and Internet-of-Things to provide these with decision-making capabilities. Such platforms often need to implement machine learning (ML) algorithms under stringent energy constraints with battery-powered electronics. Especially, energy consumption in memory subsystems dominates such a system's energy efficiency. In addition, the memory access latency is a major bottleneck for overall system throughput. To address these issues in memory-intensive inference applications, this dissertation proposes deep in-memory accelerator (DIMA), which deeply embeds computation into the memory array, employing two key principles: (1) accessing and processing multiple rows of memory array at a time, and (2) embedding pitch-matched low-swing analog processing at the periphery of bitcell array. The signal-to-noise ratio (SNR) is budgeted by employing low-swing operations in both memory read and processing to exploit the application level's error immunity for aggressive energy efficiency. This dissertation first describes the system rationale underlying the DIMA's processing stages by identifying the common functional flow across a diverse set of inference algorithms. Based on the analysis, this dissertation presents a multi-functional DIMA to support four algorithms: support vector machine (SVM), template matching (TM), k-nearest neighbor (k-NN), and matched filter. The circuit and architectural level design techniques and guidelines are provided to address the challenges in achieving multi-functionality. A prototype integrated circuit (IC) of a multi-functional DIMA was fabricated with a 16 KB SRAM array in a 65 nm CMOS process. Measurement results show up to 5.6X and 5.8X energy and delay reductions leading to 31X energy delay product (EDP) reduction with negligible (<1%) accuracy degradation as compared to the conventional 8-b fixed-point digital implementation optimally designed for each algorithm. Then, DIMA also has been applied to more complex algorithms: (1) convolutional neural network (CNN), (2) sparse distributed memory (SDM), and (3) random forest (RF). System-level simulations of CNN using circuit behavioral models in a 45 nm SOI CMOS demonstrate that high probability (>0.99) of handwritten digit recognition can be achieved using the MNIST database, along with a 24.5X reduced EDP, a 5.0X reduced energy, and a 4.9X higher throughput as compared to the conventional system. The DIMA-based SDM architecture also achieves up to 25X and 12X delay and energy reductions, respectively, over conventional SDM with negligible accuracy degradation (within 0.4%) for 16X16 binary-pixel image classification. A DIMA-based RF was realized as a prototype IC with a 16 KB SRAM array in a 65 nm process. To the best of our knowledge, this is the first IC realization of an RF algorithm. The measurement results show that the prototype achieves a 6.8X lower EDP compared to a conventional design at the same accuracy (94%) for an eight-class traffic sign recognition problem. The multi-functional DIMA and extension to other algorithms naturally motivated us to consider a programmable DIMA instruction set architecture (ISA), namely MATI. This dissertation explores a synergistic combination of the instruction set, architecture and circuit design to achieve the programmability without losing DIMA's energy and throughput benefits. Employing silicon-validated energy, delay and behavioral models of deep in-memory components, we demonstrate that MATI is able to realize nine ML benchmarks while incurring negligible overhead in energy (< 0.1%), and area (4.5%), and in throughput, over a fixed four-function DIMA. In this process, MATI is able to simultaneously achieve enhancements in both energy (2.5X to 5.5X) and throughput (1.4X to 3.4X) for an overall EDP improvement of up to 12.6X over fixed-function digital architectures

Brain-machine interface coupled cognitive sensory fusion with a Kohonen and reservoir computing scheme

Artificial Intelligence (AI) has been a source of great intrigue and has spawned many questions regarding the human condition and the core of what it means to be a sentient entity. The field has bifurcated into so-called “weak” and “strong” artificial intelligence. In weak artificial intelligence reside the forms of automation and data mining that we interact with on a daily basis. Strong artificial intelligence can be best defined as a “synthetic” being with cognitive abilities and the capacity for presence of mind that we would normally associate with humankind. We feel that this distinction is misguided. First, we begin with the statement that intelligence lies on a spectrum, even in artificial systems. The fact that our systems currently can be considered weak artificial intelligence does not preclude our ability to develop an understanding that can lead us to more complex behavior. In this research, we utilized neural feedback via electroencephalogram (EEG) data to develop an emotional landscape for linguistic interaction via the android's sensory fields which we consider to be part and parcel of embodied cognition. We have also given the iCub child android the instinct to babble the words it has learned. This is a skill that we leveraged for low-level linguistic acquisition in the latter part of this research, the slightly stronger artificial intelligence goal. This research is motivated by two main questions regarding intelligence: Is intelligence an emergent phenomenon? And, if so, can multi-modal sensory information and a term coined called “co-intelligence” which is a shared sensory experience via coupling EEG input, assist in the development of representations in the mind that we colloquially refer to as language? Given that it is not reasonable to program all of the activities needed to foster intelligence in artificial systems, our hope is that these types of forays will set the stage for further development of stronger artificial intelligence constructs. We have incorporated self-organizing processes - i.e. Kohonen maps, hidden Markov models for the speech, language development and emotional information via neural data - to help lay the substrate for emergence. Next, homage is given to the central and unique role played in intellectual study by language. We have also developed rudimentary associative memory for the iCub that is derived from the aforementioned sensory input that was collected. We formalized this process only as needed, but that is based on the assumption that mind, brain and language can be represented using the mathematics and logic of the day without contradiction. We have some reservations regarding this statement, but unfortunately a proof is a task beyond the scope of this Ph.D. Finally, this data from the coupling of the EEG and the other sensory modes of embodied cognition is used to interact with a reservoir computing recurrent neural network in an attempt to produce simple language interaction, e.g. babbling, from the child android

Hypertext Semiotics in the Commercialized Internet

Die Hypertext Theorie verwendet die selbe Terminologie, welche seit Jahrzehnten in der semiotischen Forschung untersucht wird, wie z.B. Zeichen, Text, Kommunikation, Code, Metapher, Paradigma, Syntax, usw. Aufbauend auf jenen Ergebnissen, welche in der Anwendung semiotischer Prinzipien und Methoden auf die Informatik erfolgreich waren, wie etwa Computer Semiotics, Computational Semiotics und Semiotic Interface Engineering, legt diese Dissertation einen systematischen Ansatz für all jene Forscher dar, die bereit sind, Hypertext aus einer semiotischen Perspektive zu betrachten. Durch die Verknüpfung existierender Hypertext-Modelle mit den Resultaten aus der Semiotik auf allen Sinnesebenen der textuellen, auditiven, visuellen, taktilen und geruchlichen Wahrnehmung skizziert der Autor Prolegomena einer Hypertext-Semiotik-Theorie, anstatt ein völlig neues Hypertext-Modell zu präsentieren. Eine Einführung in die Geschichte der Hypertexte, von ihrer Vorgeschichte bis zum heutigen Entwicklungsstand und den gegenwärtigen Entwicklungen im kommerzialisierten World Wide Web bilden den Rahmen für diesen Ansatz, welcher als Fundierung des Brückenschlages zwischen Mediensemiotik und Computer-Semiotik angesehen werden darf. Während Computer-Semiotiker wissen, dass der Computer eine semiotische Maschine ist und Experten der künstlichen Intelligenz-Forschung die Rolle der Semiotik in der Entwicklung der nächsten Hypertext-Generation betonen, bedient sich diese Arbeit einer breiteren methodologischen Basis. Dementsprechend reichen die Teilgebiete von Hypertextanwendungen, -paradigmen, und -strukturen, über Navigation, Web Design und Web Augmentation zu einem interdisziplinären Spektrum detaillierter Analysen, z.B. des Zeigeinstrumentes der Web Browser, des Klammeraffen-Zeichens und der sogenannten Emoticons. Die Bezeichnung ''Icon'' wird als unpassender Name für jene Bildchen, welche von der graphischen Benutzeroberfläche her bekannt sind und in Hypertexten eingesetzt werden, zurückgewiesen und diese Bildchen durch eine neue Generation mächtiger Graphic Link Markers ersetzt. Diese Ergebnisse werden im Kontext der Kommerzialisierung des Internet betrachtet. Neben der Identifizierung der Hauptprobleme des eCommerce aus der Perspektive der Hypertext Semiotik, widmet sich der Autor den Informationsgütern und den derzeitigen Hindernissen für die New Economy, wie etwa der restriktiven Gesetzeslage in Sachen Copyright und Intellectual Property. Diese anachronistischen Beschränkungen basieren auf der problematischen Annahme, dass auch der Informationswert durch die Knappheit bestimmt wird. Eine semiotische Analyse der iMarketing Techniken, wie z.B. Banner Werbung, Keywords und Link Injektion, sowie Exkurse über den Browser Krieg und den Toywar runden die Dissertation ab

Adaptive Methods for Robust Document Image Understanding

A vast amount of digital document material is continuously being produced as part of major digitization efforts around the world. In this context, generic and efficient automatic solutions for document image understanding represent a stringent necessity. We propose a generic framework for document image understanding systems, usable for practically any document types available in digital form. Following the introduced workflow, we shift our attention to each of the following processing stages in turn: quality assurance, image enhancement, color reduction and binarization, skew and orientation detection, page segmentation and logical layout analysis. We review the state of the art in each area, identify current defficiencies, point out promising directions and give specific guidelines for future investigation. We address some of the identified issues by means of novel algorithmic solutions putting special focus on generality, computational efficiency and the exploitation of all available sources of information. More specifically, we introduce the following original methods: a fully automatic detection of color reference targets in digitized material, accurate foreground extraction from color historical documents, font enhancement for hot metal typesetted prints, a theoretically optimal solution for the document binarization problem from both computational complexity- and threshold selection point of view, a layout-independent skew and orientation detection, a robust and versatile page segmentation method, a semi-automatic front page detection algorithm and a complete framework for article segmentation in periodical publications. The proposed methods are experimentally evaluated on large datasets consisting of real-life heterogeneous document scans. The obtained results show that a document understanding system combining these modules is able to robustly process a wide variety of documents with good overall accuracy

Tracking the Temporal-Evolution of Supernova Bubbles in Numerical Simulations

The study of low-dimensional, noisy manifolds embedded in a higher dimensional space has been extremely useful in many applications, from the chemical analysis of multi-phase flows to simulations of galactic mergers. Building a probabilistic model of the manifolds has helped in describing their essential properties and how they vary in space. However, when the manifold is evolving through time, a joint spatio-temporal modelling is needed, in order to fully comprehend its nature. We propose a first-order Markovian process that propagates the spatial probabilistic model of a manifold at fixed time, to its adjacent temporal stages. The proposed methodology is demonstrated using a particle simulation of an interacting dwarf galaxy to describe the evolution of a cavity generated by a Supernov

Aspects of Linguistic Variation

This volume brings together papers on linguistic variation. It takes a broad perspective, covering not only crosslinguistic and diachronic but also intralinguistic and interspeaker variation, and examines phenomena ranging from negation and TAM over connectives and the lexicon to definite articles and comparative concepts in well- and lesser-known languages. The collection thus contributes to our understanding of variation in general