Modeling and simulating in-memory memristive deep learning systems: an overview of current efforts

Deep Learning (DL) systems have demonstrated unparalleled performance in many challenging engineering applications. As the complexity of these systems inevitably increase, they require increased processing capabilities and consume larger amounts of power, which are not readily available in resource-constrained processors, such as Internet of Things (IoT) edge devices. Memristive In-Memory Computing (IMC) systems for DL, entitled Memristive Deep Learning Systems (MDLSs), that perform the computation and storage of repetitive operations in the same physical location using emerging memory devices, can be used to augment the performance of traditional DL architectures; massively reducing their power consumption and latency. However, memristive devices, such as Resistive Random-Access Memory (RRAM) and Phase-Change Memory (PCM), are difficult and cost-prohibitive to fabricate in small quantities, and are prone to various device non-idealities that must be accounted for. Consequently, the popularity of simulation frameworks, used to simulate MDLS prior to circuit-level realization, is burgeoning. In this paper, we provide a survey of existing simulation frameworks and related tools used to model large-scale MDLS. Moreover, we perform direct performance comparisons of modernized open-source simulation frameworks, and provide insights into future modeling and simulation strategies and approaches. We hope that this treatise is beneficial to the large computers and electrical engineering community, and can help readers better understand available tools and techniques for MDLS development

Quantitative Analysis of Rapid-Scan Phased Array Weather Radar Benefits and Data Quality Under Various Scan Conditions

Currently, NEXRAD provides weather radar coverage for the contiguous United States. It is believed that a replacement system for NEXRAD will be in place by the year 2040, where a major goal of such a system is to provide improved temporal resolution compared to the 5-10-min updates of NEXRAD. In this dissertation, multiple projects are undertaken to help achieve the goals of improved temporal resolution, and to understand possible scanning strategies and radar designs that can meet the goal of improved temporal resolution while either maintaining (or improving) data quality. Chapter 2 of this dissertation uses a radar simulator to simulate the effect of various scanning strategies on data quality. It is found that while simply reducing the number of pulses per radial decreases data quality, other methods such as beam multiplexing and radar imaging/digital beamforming offer significant promise for improving data quality and/or temporal resolution. Beam multiplexing is found to offer a speedup factor of 1.7-2.9, while transmit beam spoiling by 10 degrees in azimuth can offer speedup factors up to ~4 in some regions. Due to various limitations, it is recommended that these two methods be used judiciously for rapid-scan applications. Chapter 3 attempts to quantify the benefits of a rapid-scan weather radar system for tornado detection. The first goal of Chapter 3 is to track the development of a common tornado signature (tornadic debris signature, or TDS) and relate it to developments in tornado strength. This is the first study to analyze the evolution of common tornado signatures at very high temporal resolution (6 s updates) by using a storm-scale tornado model and a radar emulator. This study finds that the areal extent of the TDS is correlated with both debris availability and with tornado strength. We also find that significant changes in the radar moment variables occur on short (sub-1-min) timescales. Chapter 3 also shows that the calculated improvement in tornado detection latency time (137-207 s) is greater than that provided by theory alone (107 s). Together, the two results from Chapter 3 emphasize the need for sub-1-min updates in some applications such as tornado detection. The ability to achieve these rapid updates in certain situations will likely require a combination of advanced scanning strategies (such as those mentioned in Chapter 2) and adaptive scanning. Chapter 4 creates an optimization-based model to adaptively reallocate radar resources for the purpose of improving data quality. This model is primarily meant as a proof of concept to be expanded to other applications in the future. The result from applying this model to two real-world cases is that data quality is successfully improved in multiple areas of enhanced interest, at the expense of worsening data quality in regions where data quality is not as important. This model shows promise for using adaptive scanning in future radar applications. Together, these results can help the meteorological community understand the needs, challenges, and possible solutions to designing a replacement system for NEXRAD. All of the techniques studied herein either rely upon (or are most easily achieved by) phased array radar (PAR), which further emphasizes the utility of PAR for achieving rapid updates with sufficient data quality. It is hoped that the results in this dissertation will help guide future decisions about requirements and design specifications for the replacement system for NEXRAD

Multivariable control of a flotation plant simulator

Includes bibliography.This dissertation describes the application of two multivariable frequency domain techniques in order to design controllers for a “flotation plant”. A flotation plant simulator was designed and constructed at the University of Cape Town. The design of the multivariable controllers was based on a linear time invariant model (in s-domain) developed for the simulator. The two frequency domain techniques, Characteristic Loci (CL) and Inverse Nyquist Array (INA), were implemented in the form of CAD packages. The INA CAD package had already been written at the university but the CL CAD package had to be developed before the design of the controllers could proceed

The Telecommunications and Data Acquisition Report

Reports on developments in programs managed by JPL's Office of Telecommunications and Data Acquisition are presented. Emphasis is placed on activities of the Deep Space Network and its associated ground facilities

Doctor of Philosophy

dissertationHigh-performance supercomputers on the Top500 list are commonly designed around commodity CPUs. Most of the codes executed on these machines are message-passing codes using the message-passing toolkit (MPI). Thus it makes sense to look at these machines from a holistic systems architecture perspective and consider optimizations to commodity processors that make them more efficient in message-passing architectures. Described herein is a new User-Level Notification (ULN) architecture that significantly improves message-passing performance. The architecture integrates a simultaneous multithreaded (SMT) processor with a user-level network interface (NI) that can directly control the execution scheduling of threads on the processor. By allowing the network interface to control the execution of message handling code at the user level, the operating system (OS) related overhead for handling interrupts and user code dispatch related to notifications is eliminated. By using an SMT processor, message handling can be performed in one thread concurrent to user computation in other threads, thus most of the overhead of executing message handlers can be hidden. This dissertation presents measurements showing the OS overheads related to message-passing are significant in modern architectures and describes a new architecture that significantly reduces these overheads. On a communication-intensive real-world application, the ULN architecture provides a 50.9% performance improvement over a more traditional OS-based NIC and a 5.29-31.9% improvement over a best-of-class user-level NIC due to the user-level notifications

Simulation and implementation of novel deep learning hardware architectures for resource constrained devices

Corey Lammie designed mixed signal memristive-complementary metal–oxide–semiconductor (CMOS) and field programmable gate arrays (FPGA) hardware architectures, which were used to reduce the power and resource requirements of Deep Learning (DL) systems; both during inference and training. Disruptive design methodologies, such as those explored in this thesis, can be used to facilitate the design of next-generation DL systems

A Survey on Industrial Control System Testbeds and Datasets for Security Research

The increasing digitization and interconnection of legacy Industrial Control Systems (ICSs) open new vulnerability surfaces, exposing such systems to malicious attackers. Furthermore, since ICSs are often employed in critical infrastructures (e.g., nuclear plants) and manufacturing companies (e.g., chemical industries), attacks can lead to devastating physical damages. In dealing with this security requirement, the research community focuses on developing new security mechanisms such as Intrusion Detection Systems (IDSs), facilitated by leveraging modern machine learning techniques. However, these algorithms require a testing platform and a considerable amount of data to be trained and tested accurately. To satisfy this prerequisite, Academia, Industry, and Government are increasingly proposing testbed (i.e., scaled-down versions of ICSs or simulations) to test the performances of the IDSs. Furthermore, to enable researchers to cross-validate security systems (e.g., security-by-design concepts or anomaly detectors), several datasets have been collected from testbeds and shared with the community. In this paper, we provide a deep and comprehensive overview of ICSs, presenting the architecture design, the employed devices, and the security protocols implemented. We then collect, compare, and describe testbeds and datasets in the literature, highlighting key challenges and design guidelines to keep in mind in the design phases. Furthermore, we enrich our work by reporting the best performing IDS algorithms tested on every dataset to create a baseline in state of the art for this field. Finally, driven by knowledge accumulated during this survey's development, we report advice and good practices on the development, the choice, and the utilization of testbeds, datasets, and IDSs

Integrated System Architectures for High-Performance Internet Servers

Ph.D.Computer Science and EngineeringUniversity of Michiganhttp://deepblue.lib.umich.edu/bitstream/2027.42/90845/1/binkert-thesis.pd

Genetic Programming Techniques in Engineering Applications

2012/2013Machine learning is a suite of techniques that allow developing algorithms for performing tasks by generalizing from examples. Machine learning systems, thus, may automatically synthesize programs from data. This approach is often feasible and cost-effective where manual programming or manual algorithm design is not. In the last decade techniques based on machine learning have spread in a broad range of application domains. In this thesis, we will present several novel applications of a specific machine Learning technique, called Genetic Programming, to a wide set of engineering applications grounded in real world problems. The problems treated in this work range from the automatic synthesis of regular expressions, to the generation of electricity price forecast, to the synthesis of a model for the tracheal pressure in mechanical ventilation. The results demonstrate that Genetic Programming is indeed a suitable tool for solving complex problems of practical interest. Furthermore, several results constitute a significant improvement over the existing state-of-the-art. The main contribution of this thesis is the design and implementation of a framework for the automatic inference of regular expressions from examples based on Genetic Programming. First, we will show the ability of such a framework to cope with the generation of regular expressions for solving text-extraction tasks from examples. We will experimentally assess our proposal comparing our results with previous proposals on a collection of real-world datasets. The results demonstrate a clear superiority of our approach. We have implemented the approach in a web application that has gained considerable interest and has reached peaks of more 10000 daily accesses. Then, we will apply the framework to a popular "regex golf" challenge, a competition for human players that are required to generate the shortest regular expression solving a given set of problems. Our results rank in the top 10 list of human players worldwide and outperform those generated by the only existing algorithm specialized to this purpose. Hence, we will perform an extensive experimental evaluation in order to compare our proposal to the state-of-the-art proposal in a very close and long-established research field: the generation of a Deterministic Finite Automata (DFA) from a labelled set of examples. Our results demonstrate that the existing state-of-the-art in DFA learning is not suitable for text extraction tasks. We will also show a variant of our framework designed for solving text processing tasks of the search-and-replace form. A common way to automate search-and-replace is to describe the region to be modified and the desired changes through a regular expression and a replacement expression. We will propose a solution to automatically produce both those expressions based only on examples provided by user. We will experimentally assess our proposal on real-word search-and-replace tasks. The results indicate that our proposal is indeed feasible. Finally, we will study the applicability of our framework to the generation of schema based on a sample of the eXtensible Markup Language documents. The eXtensible Markup Language documents are largely used in machine-to-machine interactions and such interactions often require that some constraints are applied to the contents of the documents. These constraints are usually specified in a separate document which is often unavailable or missing. In order to generate a missing schema, we will apply and will evaluate experimentally our framework to solve this problem. In the final part of this thesis we will describe two significant applications from different domains. We will describe a forecasting system for producing estimates of the next day electricity price. The system is based on a combination of a predictor based on Genetic Programming and a classifier based on Neural Networks. Key feature of this system is the ability of handling outliers-i.e., values rarely seen during the learning phase. We will compare our results with a challenging baseline representative of the state-of-the-art. We will show that our proposal exhibits smaller prediction error than the baseline. Finally, we will move to a biomedical problem: estimating tracheal pressure in a patient treated with high-frequency percussive ventilation. High-frequency percussive ventilation is a new and promising non-conventional mechanical ventilatory strategy. In order to avoid barotrauma and volutrauma in patience, the pressure of air insufflated must be monitored carefully. Since measuring the tracheal pressure is difficult, a model for accurately estimating the tracheal pressure is required. We will propose a synthesis of such model by means of Genetic Programming and we will compare our results with the state-of-the-art.XXVI Ciclo198

Segmentation and Dimension Reduction: Exploratory and Model-Based Approaches

Representing the information in a data set in a concise way is an important part of data analysis. A variety of multivariate statistical techniques have been developed for this purpose, such as k-means clustering and principal components analysis. These techniques are often based on the principles of segmentation (partitioning the observations into distinct groups) and dimension reduction (constructing a low-dimensional representation of a data set). However, such techniques typically make no statistical assumptions on the process that generates the data; as a result, the statistical significance of the results is often unknown. In this thesis, we incorporate the modeling principles of segmentation and dimension reduction into statistical models. We thus develop new models that can summarize and explain the information in a data set in a simple way. The focus is on dimension reduction using bilinear parameter structures and techniques for clustering both modes of a two-mode data matrix. To illustrate the usefulness of the techniques, the thesis includes a variety of empirical applications in marketing, psychometrics, and political science. An important application is modeling the response behavior in surveys with rating scales, which provides novel insight into what kinds of response styles exist, and how substantive opinions vary among respondents. We find that our modeling approaches yield new techniques for data analysis that can be useful in a variety of applied fields