Approximate Computing Survey, Part I: Terminology and Software & Hardware Approximation Techniques

The rapid growth of demanding applications in domains applying multimedia processing and machine learning has marked a new era for edge and cloud computing. These applications involve massive data and compute-intensive tasks, and thus, typical computing paradigms in embedded systems and data centers are stressed to meet the worldwide demand for high performance. Concurrently, the landscape of the semiconductor field in the last 15 years has constituted power as a first-class design concern. As a result, the community of computing systems is forced to find alternative design approaches to facilitate high-performance and/or power-efficient computing. Among the examined solutions, Approximate Computing has attracted an ever-increasing interest, with research works applying approximations across the entire traditional computing stack, i.e., at software, hardware, and architectural levels. Over the last decade, there is a plethora of approximation techniques in software (programs, frameworks, compilers, runtimes, languages), hardware (circuits, accelerators), and architectures (processors, memories). The current article is Part I of our comprehensive survey on Approximate Computing, and it reviews its motivation, terminology and principles, as well it classifies and presents the technical details of the state-of-the-art software and hardware approximation techniques.Comment: Under Review at ACM Computing Survey

TrTr: A Versatile Pre-Trained Large Traffic Model based on Transformer for Capturing Trajectory Diversity in Vehicle Population

Understanding trajectory diversity is a fundamental aspect of addressing practical traffic tasks. However, capturing the diversity of trajectories presents challenges, particularly with traditional machine learning and recurrent neural networks due to the requirement of large-scale parameters. The emerging Transformer technology, renowned for its parallel computation capabilities enabling the utilization of models with hundreds of millions of parameters, offers a promising solution. In this study, we apply the Transformer architecture to traffic tasks, aiming to learn the diversity of trajectories within vehicle populations. We analyze the Transformer's attention mechanism and its adaptability to the goals of traffic tasks, and subsequently, design specific pre-training tasks. To achieve this, we create a data structure tailored to the attention mechanism and introduce a set of noises that correspond to spatio-temporal demands, which are incorporated into the structured data during the pre-training process. The designed pre-training model demonstrates excellent performance in capturing the spatial distribution of the vehicle population, with no instances of vehicle overlap and an RMSE of 0.6059 when compared to the ground truth values. In the context of time series prediction, approximately 95% of the predicted trajectories' speeds closely align with the true speeds, within a deviation of 7.5144m/s. Furthermore, in the stability test, the model exhibits robustness by continuously predicting a time series ten times longer than the input sequence, delivering smooth trajectories and showcasing diverse driving behaviors. The pre-trained model also provides a good basis for downstream fine-tuning tasks. The number of parameters of our model is over 50 million.Comment: 16 pages, 6 figures, under reviewed by Transportation Research Board Annual Meeting, work in updat

Nonvolatile CMOS memristor, reconfigurable array and its application in power load forecasting

© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. This is the accepted manuscript version of a conference paper which has been published in final form at https://doi.org/10.1109/TII.2023.3341256The high cost, low yield, and low stability of nano-materials significantly hinder the application and development of memristors. To promote the application of memristors, researchers proposed a variety of memristor emulators to simulate memristor functions and apply them in various fields. However these emulators lack nonvolatile characteristics, limiting their scope of application. This paper proposes an innovative nonvolatile memristor circuit based on complementary metal-oxide-semiconductor (CMOS) technology, expanding the horizons of memristor emulators. The proposed memristor is fabricated in a reconfigurable array architecture using the standard CMOS process, allowing the connection between memristors to be altered by configuring the on-off state of switches. Compared to nano-material memristors, the CMOS nonvolatile memristor circuit proposed in this paper offers advantages of low manufacturing cost and easy mass production, which can promote the application of memristors. The application of the reconfigurable array is further studied by constructing an Echo State Network (ESN) for short-term load forecasting in the power system.Peer reviewe

A Cognitive Routing framework for Self-Organised Knowledge Defined Networks

This study investigates the applicability of machine learning methods to the routing protocols for achieving rapid convergence in self-organized knowledge-defined networks. The research explores the constituents of the Self-Organized Networking (SON) paradigm for 5G and beyond, aiming to design a routing protocol that complies with the SON requirements. Further, it also exploits a contemporary discipline called Knowledge-Defined Networking (KDN) to extend the routing capability by calculating the “Most Reliable” path than the shortest one. The research identifies the potential key areas and possible techniques to meet the objectives by surveying the state-of-the-art of the relevant fields, such as QoS aware routing, Hybrid SDN architectures, intelligent routing models, and service migration techniques. The design phase focuses primarily on the mathematical modelling of the routing problem and approaches the solution by optimizing at the structural level. The work contributes Stochastic Temporal Edge Normalization (STEN) technique which fuses link and node utilization for cost calculation; MRoute, a hybrid routing algorithm for SDN that leverages STEN to provide constant-time convergence; Most Reliable Route First (MRRF) that uses a Recurrent Neural Network (RNN) to approximate route-reliability as the metric of MRRF. Additionally, the research outcomes include a cross-platform SDN Integration framework (SDN-SIM) and a secure migration technique for containerized services in a Multi-access Edge Computing environment using Distributed Ledger Technology. The research work now eyes the development of 6G standards and its compliance with Industry-5.0 for enhancing the abilities of the present outcomes in the light of Deep Reinforcement Learning and Quantum Computing

Neuromorphic System Design and Application

With the booming of large scale data related applications, cognitive systems that leverage modern data processing technologies, e.g., machine learning and data mining, are widely used in various industry fields. These application bring challenges to conventional computer systems on both semiconductor manufacturing and computing architecture. The invention of neuromorphic computing system (NCS) is inspired by the working mechanism of human-brain. It is a promising architecture to combat the well-known memory bottleneck in Von Neumann architecture. The recent breakthrough on memristor devices and crossbar structure made an important step toward realizing a low-power, small-footprint NCS on-a-chip. However, the currently low manufacturing reliability of nano-devices and circuit level constrains, .e.g., the voltage IR-drop along metal wires and analog signal noise from the peripheral circuits, bring challenges on scalability, precision and robustness of memristor crossbar based NCS. In this dissertation, we quantitatively analyzed the robustness of memristor crossbar based NCS when considering the device process variations, signal fluctuation and IR-drop. Based on our analysis, we will explore deep understanding on hardware training methods, e.g., on-device training and off-device training. Then, new technologies, e.g., noise-eliminating training, variation-aware training and adaptive mapping, specifically designed to improve the training quality on memristor crossbar hardware will be proposed in this dissertation. A digital initialization step for hardware training is also introduced to reduce training time. The circuit level constrains will also limit the scalability of a single memristor crossbar, which will decrease the efficiency of implementation of NCS. We also leverage system reduction/compression techniques to reduce the required crossbar size for certain applications. Besides, running machine learning algorithms on embedded systems bring new security concerns to the service providers and the users. In this dissertation, we will first explore the security concerns by using examples from real applications. These examples will demonstrate how attackers can access confidential user data, replicate a sensitive data processing model without any access to model details and how expose some key features of training data by using the service as a normal user. Based on our understanding of these security concerns, we will use unique property of memristor device to build a secured NCS

Technology enablers for the implementation of Industry 4.0 to traditional manufacturing sectors: A review

The traditional manufacturing sectors (footwear, textiles and clothing, furniture and toys, among others) are based on small and medium enterprises with limited capacity on investing in modern production technologies. Although these sectors rely heavily on product customization and short manufacturing cycles, they are still not able to take full advantage of the fourth industrial revolution. Industry 4.0 surfaced to address the current challenges of shorter product life-cycles, highly customized products and stiff global competition. The new manufacturing paradigm supports the development of modular factory structures within a computerized Internet of Things environment. With Industry 4.0, rigid planning and production processes can be revolutionized. However, the computerization of manufacturing has a high degree of complexity and its implementation tends to be expensive, which goes against the reality of SMEs that power the traditional sectors. This paper reviews the main scientific-technological advances that have been developed in recent years in traditional sectors with the aim of facilitating the transition to the new industry standard.This research was supported by the Spanish Research Agency (AEI) and the European Regional Development Fund (ERDF) under the project CloudDriver4Industry TIN2017-89266-R

Modern computing: Vision and challenges

Over the past six decades, the computing systems field has experienced significant transformations, profoundly impacting society with transformational developments, such as the Internet and the commodification of computing. Underpinned by technological advancements, computer systems, far from being static, have been continuously evolving and adapting to cover multifaceted societal niches. This has led to new paradigms such as cloud, fog, edge computing, and the Internet of Things (IoT), which offer fresh economic and creative opportunities. Nevertheless, this rapid change poses complex research challenges, especially in maximizing potential and enhancing functionality. As such, to maintain an economical level of performance that meets ever-tighter requirements, one must understand the drivers of new model emergence and expansion, and how contemporary challenges differ from past ones. To that end, this article investigates and assesses the factors influencing the evolution of computing systems, covering established systems and architectures as well as newer developments, such as serverless computing, quantum computing, and on-device AI on edge devices. Trends emerge when one traces technological trajectory, which includes the rapid obsolescence of frameworks due to business and technical constraints, a move towards specialized systems and models, and varying approaches to centralized and decentralized control. This comprehensive review of modern computing systems looks ahead to the future of research in the field, highlighting key challenges and emerging trends, and underscoring their importance in cost-effectively driving technological progress