High-level automation of custom hardware design for high-performance computing

This dissertation focuses on efficient generation of custom processors from high-level language descriptions. Our work exploits compiler-based optimizations and transformations in tandem with high-level synthesis (HLS) to build high-performance custom processors. The goal is to offer a common multiplatform high-abstraction programming interface for heterogeneous compute systems where the benefits of custom reconfigurable (or fixed) processors can be exploited by the application developers. The research presented in this dissertation supports the following thesis: In an increasingly heterogeneous compute environment it is important to leverage the compute capabilities of each heterogeneous processor efficiently. In the case of FPGA and ASIC accelerators this can be achieved through HLS-based flows that (i) extract parallelism at coarser than basic block granularities, (ii) leverage common high-level parallel programming languages, and (iii) employ high-level source-to-source transformations to generate high-throughput custom processors. First, we propose a novel HLS flow that extracts instruction level parallelism beyond the boundary of basic blocks from C code. Subsequently, we describe FCUDA, an HLS-based framework for mapping fine-grained and coarse-grained parallelism from parallel CUDA kernels onto spatial parallelism. FCUDA provides a common programming model for acceleration on heterogeneous devices (i.e. GPUs and FPGAs). Moreover, the FCUDA framework balances multilevel granularity parallelism synthesis using efficient techniques that leverage fast and accurate estimation models (i.e. do not rely on lengthy physical implementation tools). Finally, we describe an advanced source-to-source transformation framework for throughput-driven parallelism synthesis (TDPS), which appropriately restructures CUDA kernel code to maximize throughput on FPGA devices. We have integrated the TDPS framework into the FCUDA flow to enable automatic performance porting of CUDA kernels designed for the GPU architecture onto the FPGA architecture

Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions

In the past decade, Convolutional Neural Networks (CNNs) have demonstrated state-of-the-art performance in various Artificial Intelligence tasks. To accelerate the experimentation and development of CNNs, several software frameworks have been released, primarily targeting power-hungry CPUs and GPUs. In this context, reconfigurable hardware in the form of FPGAs constitutes a potential alternative platform that can be integrated in the existing deep learning ecosystem to provide a tunable balance between performance, power consumption and programmability. In this paper, a survey of the existing CNN-to-FPGA toolflows is presented, comprising a comparative study of their key characteristics which include the supported applications, architectural choices, design space exploration methods and achieved performance. Moreover, major challenges and objectives introduced by the latest trends in CNN algorithmic research are identified and presented. Finally, a uniform evaluation methodology is proposed, aiming at the comprehensive, complete and in-depth evaluation of CNN-to-FPGA toolflows.Comment: Accepted for publication at the ACM Computing Surveys (CSUR) journal, 201

System-on-chip Computing and Interconnection Architectures for Telecommunications and Signal Processing

This dissertation proposes novel architectures and design techniques targeting SoC building blocks for telecommunications and signal processing applications. Hardware implementation of Low-Density Parity-Check decoders is approached at both the algorithmic and the architecture level. Low-Density Parity-Check codes are a promising coding scheme for future communication standards due to their outstanding error correction performance. This work proposes a methodology for analyzing effects of finite precision arithmetic on error correction performance and hardware complexity. The methodology is throughout employed for co-designing the decoder. First, a low-complexity check node based on the P-output decoding principle is designed and characterized on a CMOS standard-cells library. Results demonstrate implementation loss below 0.2 dB down to BER of 10^{-8} and a saving in complexity up to 59% with respect to other works in recent literature. High-throughput and low-latency issues are addressed with modified single-phase decoding schedules. A new "memory-aware" schedule is proposed requiring down to 20% of memory with respect to the traditional two-phase flooding decoding. Additionally, throughput is doubled and logic complexity reduced of 12%. These advantages are traded-off with error correction performance, thus making the solution attractive only for long codes, as those adopted in the DVB-S2 standard. The "layered decoding" principle is extended to those codes not specifically conceived for this technique. Proposed architectures exhibit complexity savings in the order of 40% for both area and power consumption figures, while implementation loss is smaller than 0.05 dB. Most modern communication standards employ Orthogonal Frequency Division Multiplexing as part of their physical layer. The core of OFDM is the Fast Fourier Transform and its inverse in charge of symbols (de)modulation. Requirements on throughput and energy efficiency call for FFT hardware implementation, while ubiquity of FFT suggests the design of parametric, re-configurable and re-usable IP hardware macrocells. In this context, this thesis describes an FFT/IFFT core compiler particularly suited for implementation of OFDM communication systems. The tool employs an accuracy-driven configuration engine which automatically profiles the internal arithmetic and generates a core with minimum operands bit-width and thus minimum circuit complexity. The engine performs a closed-loop optimization over three different internal arithmetic models (fixed-point, block floating-point and convergent block floating-point) using the numerical accuracy budget given by the user as a reference point. The flexibility and re-usability of the proposed macrocell are illustrated through several case studies which encompass all current state-of-the-art OFDM communications standards (WLAN, WMAN, xDSL, DVB-T/H, DAB and UWB). Implementations results are presented for two deep sub-micron standard-cells libraries (65 and 90 nm) and commercially available FPGA devices. Compared with other FFT core compilers, the proposed environment produces macrocells with lower circuit complexity and same system level performance (throughput, transform size and numerical accuracy). The final part of this dissertation focuses on the Network-on-Chip design paradigm whose goal is building scalable communication infrastructures connecting hundreds of core. A low-complexity link architecture for mesochronous on-chip communication is discussed. The link enables skew constraint looseness in the clock tree synthesis, frequency speed-up, power consumption reduction and faster back-end turnarounds. The proposed architecture reaches a maximum clock frequency of 1 GHz on 65 nm low-leakage CMOS standard-cells library. In a complex test case with a full-blown NoC infrastructure, the link overhead is only 3% of chip area and 0.5% of leakage power consumption. Finally, a new methodology, named metacoding, is proposed. Metacoding generates correct-by-construction technology independent RTL codebases for NoC building blocks. The RTL coding phase is abstracted and modeled with an Object Oriented framework, integrated within a commercial tool for IP packaging (Synopsys CoreTools suite). Compared with traditional coding styles based on pre-processor directives, metacoding produces 65% smaller codebases and reduces the configurations to verify up to three orders of magnitude

Embedded electronic systems driven by run-time reconfigurable hardware

Abstract This doctoral thesis addresses the design of embedded electronic systems based on run-time reconfigurable hardware technology –available through SRAM-based FPGA/SoC devices– aimed at contributing to enhance the life quality of the human beings. This work does research on the conception of the system architecture and the reconfiguration engine that provides to the FPGA the capability of dynamic partial reconfiguration in order to synthesize, by means of hardware/software co-design, a given application partitioned in processing tasks which are multiplexed in time and space, optimizing thus its physical implementation –silicon area, processing time, complexity, flexibility, functional density, cost and power consumption– in comparison with other alternatives based on static hardware (MCU, DSP, GPU, ASSP, ASIC, etc.). The design flow of such technology is evaluated through the prototyping of several engineering applications (control systems, mathematical coprocessors, complex image processors, etc.), showing a high enough level of maturity for its exploitation in the industry.Resumen Esta tesis doctoral abarca el diseño de sistemas electrónicos embebidos basados en tecnología hardware dinámicamente reconfigurable –disponible a través de dispositivos lógicos programables SRAM FPGA/SoC– que contribuyan a la mejora de la calidad de vida de la sociedad. Se investiga la arquitectura del sistema y del motor de reconfiguración que proporcione a la FPGA la capacidad de reconfiguración dinámica parcial de sus recursos programables, con objeto de sintetizar, mediante codiseño hardware/software, una determinada aplicación particionada en tareas multiplexadas en tiempo y en espacio, optimizando así su implementación física –área de silicio, tiempo de procesado, complejidad, flexibilidad, densidad funcional, coste y potencia disipada– comparada con otras alternativas basadas en hardware estático (MCU, DSP, GPU, ASSP, ASIC, etc.). Se evalúa el flujo de diseño de dicha tecnología a través del prototipado de varias aplicaciones de ingeniería (sistemas de control, coprocesadores aritméticos, procesadores de imagen, etc.), evidenciando un nivel de madurez viable ya para su explotación en la industria.Resum Aquesta tesi doctoral està orientada al disseny de sistemes electrònics empotrats basats en tecnologia hardware dinàmicament reconfigurable –disponible mitjançant dispositius lògics programables SRAM FPGA/SoC– que contribueixin a la millora de la qualitat de vida de la societat. S’investiga l’arquitectura del sistema i del motor de reconfiguració que proporcioni a la FPGA la capacitat de reconfiguració dinàmica parcial dels seus recursos programables, amb l’objectiu de sintetitzar, mitjançant codisseny hardware/software, una determinada aplicació particionada en tasques multiplexades en temps i en espai, optimizant així la seva implementació física –àrea de silici, temps de processat, complexitat, flexibilitat, densitat funcional, cost i potència dissipada– comparada amb altres alternatives basades en hardware estàtic (MCU, DSP, GPU, ASSP, ASIC, etc.). S’evalúa el fluxe de disseny d’aquesta tecnologia a través del prototipat de varies aplicacions d’enginyeria (sistemes de control, coprocessadors aritmètics, processadors d’imatge, etc.), demostrant un nivell de maduresa viable ja per a la seva explotació a la indústria

Field-programmable gate arrays and quantum Monte Carlo: Power efficient coprocessing for scalable high-performance computing

Massively parallel architectures offer the potential to significantly accelerate an application relative to their serial counterparts. However, not all applications exhibit an adequate level of data and/or task parallelism to exploit such platforms. Furthermore, the power consumption associated with these forms of computation renders "scaling out" for exascale levels of performance incompatible with modern sustainable energy policies. In this work, we investigate the potential for field-programmable gate arrays (FPGAs) to feature in future exascale platforms, and their capacity to improve performance per unit power measurements for the purposes of scientific computing. We have focussed our efforts on Variational Monte Carlo, and report on the benefits of co-processing with an FPGA relative to a purely multicore system.Royal Society Horizon 2020 Hartree Centr

Design Techniques of Parallel Accelerator Architectures for Real-Time Processing of Learning Algorithms

H παρούσα διδακτορική διατριβή έχει ως βασικό αντικείμενο μελέτης τα Συνελικτικά Νευρωνικά Δίκτυα (Convolutional Neural Networks - CNNs) για εφαρμογές υπολογιστικής όρασης (computer vision) και συγκεκριμένα εστιάζει στην εκτέλεση της διαδικασίας της εξαγωγής συμπερασμάτων των CNNs (CNN inference) σε ενσωματωμένους επιταχυντές κατάλληλους για εφαρμογές της υπολογιστικής των παρυφών (edge computing). Ο σκοπός της διατριβής είναι να αντιμετωπίσει τις τρέχουσες προκλήσεις σχετικά με τη βελτιστοποίηση των CNNs προκειμένου αυτά να υλοποιηθούν σε edge computing πλατφόρμες, καθώς και τις προκλήσεις στο πεδίο των τεχνικών σχεδίασης αρχιτεκτονικών επιταχυντών για CNNs. Προς αυτή την κατεύθυνση, η παρούσα διατριβή επικεντρώνεται σε διαφορετικές εφαρμογές βαθιάς μάθησης (deep learning), συμπεριλαμβανομένης της επεξεργασίας εικόνων σε δορυφόρους και της πρόβλεψης ηλιακής ακτινοβολίας από εικόνες. Στις παραπάνω εφαρμογές, η διατριβή συμβάλλει σε τέσσερα διακριτά προβλήματα στα πεδία της βελτιστοποίησης CNNs και της σχεδίασης επιταχυντών CNNs. Αρχικά, η διατριβή συνεισφέρει στην υπάρχουσα βιβλιογραφία σχετικά με τεχνικές επεξεργασίας εικόνας, βασισμένες στα CNNs, για την εκτίμηση και πρόβλεψη ηλιακής ακτινοβολίας. Στα πλαίσια της διατριβής, προτείνεται μια μέθοδος επεξεργασίας εικόνας η οποία βασίζεται στον ακριβή εντοπισμό του Ήλιου σε εικόνες του ουρανού, χρησιμοποιώντας τις συντεταγμένες του Ήλιου και τις εξισώσεις του fisheye φακού της κάμερας λήψης εικόνων του ουρανού. Όταν η προτεινόμενη μέθοδος εφαρμόζεται σε φωτογραφίες του ουρανού πριν από την επεξεργασία τους από τα CNNs, τα αποτελέσματα από την εκτεταμένη μελέτη που διενεργεί η διατριβή, δείχνουν πως μπορεί να βελτιώσει την ακρίβεια των τιμών ακτινοβολίας που παράγουν τα CNNs σε όλες τις περιπτώσεις και με μικρή μόνο αύξηση στο πλήθος των υπολογισμών των CNNs. Στη συνέχεια, η διδακτορική διατριβή επικεντρώνεται στην κατάτμηση εικόνων βασισμένη στη βαθιά μάθηση, με στόχο τον εντοπισμό σύννεφων από δορυφορικές εικόνες σε εφαρμογές επεξεργασίας δεδομένων σε δορυφόρους. Πιο συγκεκριμένα, στα πλαίσια της διατριβής προτείνεται μια αρχιτεκτονική μοντέλου CNN περιορισμένων υπολογιστικών απαιτήσεων, βασισμένη στην αρχιτεκτονική U-Net, η οποία στοχεύει σε μια βελτιωμένη αναλογία ανάμεσα στο μέγεθος του μοντέλου και στις επιδόσεις του στη δυαδική κατάτμηση της εικόνας. Το προτεινόμενο μοντέλο εκμεταλλεύεται πλήθος τεχνικών CNNs προκειμένου να μειώσει το πλήθος των παραμέτρων και πράξεων που απαιτείται για την εκτέλεση του μοντέλου, αλλά ταυτόχρονα να πετυχαίνει ικανοποιητική ακρίβεια αποτελεσμάτων. Η διατριβή διενεργεί μια μελέτη ανάμεσα σε CNN μοντέλα της βιβλιογραφίας για εντοπισμό σύννεφων που έχουν αξιολογηθεί στα ίδια δεδομένα με το προτεινόμενο μοντέλο, και έτσι αναδεικνύει τα προτερήματά του. Επιπλέον, η διδακτορική διατριβή στοχεύει στην αποδοτική υλοποίηση του inference των CNNs επεξεργασίας εικόνας σε ενσωματωμένους επιταχυντές κατάλληλους για εφαρμογές edge computing. Για τον σκοπό αυτό, η διατριβή επιλέγει τα Field-Programmable Gate Arrays (FPGAs) για την επιτάχυνση των CNNs και συνεισφέρει τις λεπτομέρειες της μεθοδολογίας ανάπτυξης που υιοθετήθηκε και η οποία βασίζεται στο εργαλείο Xilinx Vitis AI. Πέρα από τη μελέτη των δυνατοτήτων του Vitis AI, όπως των προχωρημένων τεχνικών κβάντισης των μοντέλων, η διατριβή παρουσιάζει επιπλέον και μια προσέγγιση επιτάχυνσης για την επιτάχυνση των επιμέρους διεργασιών μιας ολοκληρωμένης εργασίας μηχανικής όρασης η οποία εκμεταλλεύεται τους ετερογενείς πόρους του FPGA. Τα αποτελέσματα χρόνων εκτέλεσης και διεκπεραιωτικότητας (throughput) των CNNs τόσο για τη δυαδική κατάτμηση εικόνων για εντοπισμό σύννεφων όσο και για την εκτίμηση ηλιακής ακτινοβολίας από εικόνες, στο FPGA, αναδεικνύουν τις δυνατότητες επεξεργασίας σε πραγματικό χρόνο του επιταχυντή. Τέλος, η διδακτορική διατριβή συνεισφέρει τη σχεδίαση ενός συστήματος διεπαφής, υψηλών επιδόσεων και με ανοχή στα σφάλματα, για την αμφίδρομη μεταφορά εικόνων ανάμεσα σε ενσωματωμένους επιταχυντές βαθιάς μάθησης, στα πλαίσια υπολογιστικών αρχιτεκτονικών για επεξεργασία δεδομένων σε δορυφόρους. Το σύστημα διεπαφής αναπτύχθηκε για την επικοινωνία ανάμεσα σε ένα FPGA και τον επιταχυντή Intel Movidius Myriad 2 και η εκτεταμένη διαδικασία επαλήθευσης του συστήματος, τόσο σε εμπορικά διαθέσιμες όσο και σε πρωτότυπες πλατφόρμες, έδειξε πως αυτό μπορεί να επιτύχει μέχρι και 2.4 Gbps αμφίδρομους ρυθμούς μετάδοσης δεδομένων εικόνων.The current doctoral thesis focuses on Convolutional Neural Networks (CNNs) for computer vision applications and particularly on the deployment of the inference process of CNNs to embedded accelerators suitable for edge computing. The objective of the thesis is to address several challenges regarding the optimization techniques of CNNs towards their edge deployment as well as challenges in the field of CNN accelerator architectures design techniques. In this direction, the thesis focuses on different deep learning applications, including on-board payload data processing as well as solar irradiance forecasting, and makes distinct contributions to four different challenges in the fields of CNN optimization and CNN accelerators design. First, the thesis contributes to the existing literature regarding image processing techniques and deep learning-based image regression for solar irradiance estimation and forecasting. It proposes an image processing method which is based on accurate sun localization in sky images and which utilizes the solar angles and the mapping functions of the lens of the sky imager camera. When the proposed method is applied to the sky images before these are processed by the image regression CNNs, the results from the extensive study that the thesis conducts, show that the method can improve the accuracy of the irradiance values that the CNNs produce in all cases by introducing only minimal computational overhead. Next, the thesis focuses on the task of deep learning-based semantic segmentation in order to enable cloud detection from satellite imagery in on-board payload data processing applications. In particular, the thesis proposes a lightweight CNN model architecture, based on the U-Net architecture, which aims at providing an improved trade-off between model size and binary semantic segmentation performance. The proposed model utilizes several CNN techniques in order to reduce the number of parameters and operations required for the inference but at the same time maintain satisfying performance. The thesis conducts a study among CNN models for cloud detection, which are evaluated on the same test dataset as the proposed model, and thus showcases the advantages of the proposed model. Then, the thesis targets the efficient porting of the inference process of image processing CNNs to edge-oriented embedded accelerator devices. The thesis opts for CNN acceleration based on Field-Programmable Gate Arrays (FPGAs) and contributes the adopted development flow which utilizes the Xilinx Vitis AI framework. Apart from exploring the capabilities of Vitis AI, including its advanced quantization solutions, the thesis also showcases an acceleration approach for accelerating different processes of a single computer vision task by taking advantage of the heterogeneous resources of the FPGA. The execution time and throughput results of the CNN models, for the tasks of binary semantic segmentation for cloud detection as well as image regression for irradiance estimation, on the FPGA, showcase the real-time processing capabilities of the accelerator. Finally, the thesis contributes the design details of a bi-directional interfacing system for high-throughput and fault-tolerant image transfers between deep learning embedded accelerators, in the context of on-board payload data processing architectures. The interfacing system is developed for interfacing an FPGA with the Intel Movidius Myriad 2 and the extensive testing campaign based on both commercial as well as prototype hardware platforms, shows that it can achieve a bit-rate of up to 2.4 Gbps duplex image data transfers