Direct Feedback Alignment with Sparse Connections for Local Learning

Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent. Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks. Commonly referred to as the weight transport problem, each neuron's dependence on the weights and errors located deeper in the network require exhaustive data movement which presents a key problem in enhancing the performance and energy-efficiency of machine-learning hardware. In this work, we propose a bio-plausible alternative to backpropagation drawing from advances in feedback alignment algorithms in which the error computation at a single synapse reduces to the product of three scalar values. Using a sparse feedback matrix, we show that a neuron needs only a fraction of the information previously used by the feedback alignment algorithms. Consequently, memory and compute can be partitioned and distributed whichever way produces the most efficient forward pass so long as a single error can be delivered to each neuron. Our results show orders of magnitude improvement in data movement and $2\times$ improvement in multiply-and-accumulate operations over backpropagation. Like previous work, we observe that any variant of feedback alignment suffers significant losses in classification accuracy on deep convolutional neural networks. By transferring trained convolutional layers and training the fully connected layers using direct feedback alignment, we demonstrate that direct feedback alignment can obtain results competitive with backpropagation. Furthermore, we observe that using an extremely sparse feedback matrix, rather than a dense one, results in a small accuracy drop while yielding hardware advantages. All the code and results are available under https://github.com/bcrafton/ssdfa.Comment: 15 pages, 8 figure

A Multi-objective Design Approach to Include Material, Manufacturing and Assembly Costs in the Early Design Phase

AbstractConceptual design is a crucial activity in the product development process. The design freedom must consider a trade-off analysis among several aspects such as assembly, manufacturing, and costs. The goal of this approach is to define a multi-objective design approach for the determination of feasible design options. The approach is grounded on the concept of functional basis for the analysis of product modules and the theory of Multi Criteria Decision Making (MCDM) approach for the assessment of the best design option. A complex product (tool-holder carousel of a machine tool) is used as a case study to validate the approach

Self-Addressable Memory-Based FSM: A Scalable Intrusion Detection Engine

One way to detect and thwart a network attack is to compare each incoming packet with predefined patterns, also called an attack pattern database, and raise an alert upon detecting a match. This article presents a novel pattern-matching engine that exploits a memory-based, programmable state machine to achieve deterministic processing rates that are independent of packet and pattern characteristics. Our engine is a self-addressable memory-based finite state machine (SAMFSM), whose current state coding exhibits all its possible next states. Moreover, it is fully reconfigurable in that new attack patterns can be updated easily. A methodology was developed to program the memory and logic. Specifically, we merge non-equivalent states by introducing super characters on their inputs to further enhance memory efficiency without adding labels. SAM-FSM is one of the most storage-efficient machines and reduces the memory requirement by 60 times. Experimental results are presented to demonstrate the validity of SAM-FSM

Product Variants Platform Customization Strategies and Performance of Reconfigurable Manufacturing Systems (RMS)

Customers’ demands and needs are changing over time. As a result, manufacturers are seeking new ways to respond to market changes effectively and efficiently. They include offering customers a wide range of product varieties in a reasonable time while reducing associated costs. One of the prime techniques adopted by manufacturers is mass customization and its enablers, such as product family and product platforms. The main objective of this research is to help manufacturers manage a high level of variety by implementing the most suitable manufacturing strategy and product platform design. Customized Platform To Order (CPTO) has been introduced and compared with existing manufacturing/production strategies, such as assemble to order (ATO). CPTO is a hybrid assemble-to-stock (ATS)/assemble-to-order (ATO) strategy that uses a platform customization approach to increase the efficiency and productivity of manufacturers. The platform(s) design is based on customers’ historical demand rather than on commonality between product variants. In this thesis, the CPTO approach was compared to the ATO and hybrid ATS/ATO strategies. A discrete-event simulation model of the learning factory iFactory in the Intelligent Manufacturing System Centre (IMSC) is developed. The results were then compared with a physical implementation conducted in the (IMS) Centre. The results of this investigation indicated that the CPTO approach provides manufacturers the ability to be more responsive by reducing the lead time by 30% and assembly time by 27% as well as lowering inventory and assembly costs by 24% and 18% respectively for the considered case study. This approach is applicable to products with modular and flexible platforms and both flexible and reconfigurable manufacturing systems

Hardware-based text-to-braille translation

Braille, as a special written method of communication for the blind, has been globally accepted for years. It gives blind people another chance to learn and communicate more efficiently with the rest of the world. It also makes possible the translation of printed languages into a written language which is recognisable for blind people. Recently, Braille is experiencing a decreasing popularity due to the use of alternative technologies, like speech synthesis. However, as a form of literacy, Braille is still playing a significant role in the education of people with visual impairments. With the development of electronic technology, Braille turned out to be well suited to computer-aided production because of its coded forms. Software based text-to-Braille translation has been proved to be a successful solution in Assistive Technology (AT). However, the feasibility and advantages of the algorithm reconfiguration based on hardware implementation have rarely been substantially discussed. A hardware-based translation system with algorithm reconfiguration is able to supply greater throughput than a software-based system. Further, it is also expected as a single component integrated in a multi-functional Braille system on a chip.Therefore, this thesis presents the development of a system for text-to-Braille translation implemented in hardware. Differing from most commercial methods, this translator is able to carry out the translation in hardware instead of using software. To find a particular translation algorithm which is suitable for a hardware-based solution, the history of, and previous contributions to Braille translation are introduced and discussed. It is concluded that Markov systems, a formal language theory, were highly suitable for application to hardware based Braille translation. Furthermore, the text-to-Braille algorithm is reconfigured to achieve parallel processing to accelerate the translation speed. Characteristics and advantages of Field Programmable Gate Arrays (FPGAs), and application of Very High Speed Integrated Circuit Hardware Description Language (VHDL) are introduced to explain how the translating algorithm can be transformed to hardware. Using a Xilinx hardware development platform, the algorithm for text-to-Braille translation is implemented and the structure of the translator is described hierarchically