1,684 research outputs found
A DevOps approach to integration of software components in an EU research project
We present a description of the development and deployment infrastructure being created to support the integration effort of HARNESS, an EU FP7 project. HARNESS is a multi-partner research project intended to bring the power of heterogeneous resources to the cloud. It consists of a number of different services and technologies that interact with the OpenStack cloud computing platform at various levels. Many of these components are being developed independently by different teams at different locations across Europe, and keeping the work fully integrated is a challenge. We use a combination of Vagrant based virtual machines, Docker containers, and Ansible playbooks to provide a consistent and up-to-date environment to each developer. The same playbooks used to configure local virtual machines are also used to manage a static testbed with heterogeneous compute and storage devices, and to automate ephemeral larger-scale deployments to Grid5000. Access to internal projects is managed by GitLab, and automated testing of services within Docker-based environments and integrated deployments within virtual-machines is provided by Buildbot
LEGaTO: first steps towards energy-efficient toolset for heterogeneous computing
LEGaTO is a three-year EU H2020 project which started in December 2017. The LEGaTO project will leverage task-based programming models to provide a software ecosystem for Made-in-Europe heterogeneous hardware composed of CPUs, GPUs, FPGAs and dataflow engines. The aim is to attain one order of magnitude energy savings from the edge to the converged cloud/HPC.Peer ReviewedPostprint (author's final draft
λ³λ ¬ λ° λΆμ° μλ² λλ μμ€ν μ μν λͺ¨λΈ κΈ°λ° μ½λ μμ± νλ μμν¬
νμλ
Όλ¬Έ(λ°μ¬)--μμΈλνκ΅ λνμ :곡과λν μ»΄ν¨ν°κ³΅νλΆ,2020. 2. νμν.μννΈμ¨μ΄ μ€κ³ μμ°μ± λ° μ μ§λ³΄μμ±μ ν₯μμν€κΈ° μν΄ λ€μν μννΈμ¨μ΄ κ°λ° λ°©λ²λ‘ μ΄ μ μλμμ§λ§, λλΆλΆμ μ°κ΅¬λ μμ© μννΈμ¨μ΄λ₯Ό νλμ νλ‘μΈμμμ λμμν€λ λ°μ μ΄μ μ λ§μΆκ³ μλ€. λν, μλ² λλ μμ€ν
μ κ°λ°νλ λ°μ νμν μ§μ°μ΄λ μμ μꡬ μ¬νμ λν λΉκΈ°λ₯μ μꡬ μ¬νμ κ³ λ €νμ§ μκ³ μκΈ° λλ¬Έμ μΌλ°μ μΈ μννΈμ¨μ΄ κ°λ° λ°©λ²λ‘ μ μλ² λλ μννΈμ¨μ΄λ₯Ό κ°λ°νλ λ°μ μ μ©νλ κ²μ μ ν©νμ§ μλ€.
μ΄ λ
Όλ¬Έμμλ λ³λ ¬ λ° λΆμ° μλ² λλ μμ€ν
μ λμμΌλ‘ νλ μννΈμ¨μ΄λ₯Ό λͺ¨λΈλ‘ νννκ³ , μ΄λ₯Ό μννΈμ¨μ΄ λΆμμ΄λ κ°λ°μ νμ©νλ κ°λ° λ°©λ²λ‘ μ μκ°νλ€. μ°λ¦¬μ λͺ¨λΈμμ μμ© μννΈμ¨μ΄λ κ³μΈ΅μ μΌλ‘ ννν μ μλ μ¬λ¬ κ°μ νμ€ν¬λ‘ μ΄λ£¨μ΄μ Έ μμΌλ©°, νλμ¨μ΄ νλ«νΌκ³Ό λ
립μ μΌλ‘ λͺ
μΈνλ€. νμ€ν¬ κ°μ ν΅μ λ° λκΈ°νλ λͺ¨λΈμ΄ μ μν κ·μ½μ΄ μ ν΄μ Έ μκ³ , μ΄λ¬ν κ·μ½μ ν΅ν΄ μ€μ νλ‘κ·Έλ¨μ μ€ννκΈ° μ μ μννΈμ¨μ΄ μλ¬λ₯Ό μ μ λΆμμ ν΅ν΄ νμΈν μ μκ³ , μ΄λ μμ©μ κ²μ¦ 볡μ‘λλ₯Ό μ€μ΄λ λ°μ κΈ°μ¬νλ€. μ§μ ν νλμ¨μ΄ νλ«νΌμμ λμνλ νλ‘κ·Έλ¨μ νμ€ν¬λ€μ νλ‘μΈμμ 맀νν μ΄νμ μλμ μΌλ‘ ν©μ±ν μ μλ€.
μμ λͺ¨λΈ κΈ°λ° μννΈμ¨μ΄ κ°λ° λ°©λ²λ‘ μμ μ¬μ©νλ νλ‘κ·Έλ¨ ν©μ±κΈ°λ₯Ό λ³Έ λ
Όλ¬Έμμ μ μνμλλ°, λͺ
μΈν νλ«νΌ μꡬ μ¬νμ λ°νμΌλ‘ λ³λ ¬ λ° λΆμ° μλ² λλ μμ€ν
μμμ λμνλ μ½λλ₯Ό μμ±νλ€. μ¬λ¬ κ°μ μ νμ λͺ¨λΈλ€μ κ³μΈ΅μ μΌλ‘ νννμ¬ μμ©μ λμ ννλ₯Ό λνκ³ , ν©μ±κΈ°λ μ¬λ¬ λͺ¨λΈλ‘ ꡬμ±λ κ³μΈ΅μ μΈ λͺ¨λΈλ‘λΆν° λ³λ ¬μ±μ κ³ λ €νμ¬ νμ€ν¬λ₯Ό μ€νν μ μλ€. λν, νλ‘κ·Έλ¨ ν©μ±κΈ°μμ λ€μν νλ«νΌμ΄λ λ€νΈμν¬λ₯Ό μ§μν μ μλλ‘ μ½λλ₯Ό κ΄λ¦¬νλ λ°©λ²λ 보μ¬μ£Όκ³ μλ€. λ³Έ λ
Όλ¬Έμμ μ μνλ μννΈμ¨μ΄ κ°λ° λ°©λ²λ‘ μ 6κ°μ νλμ¨μ΄ νλ«νΌκ³Ό 3 μ’
λ₯μ λ€νΈμν¬λ‘ ꡬμ±λμ΄ μλ μ€μ κ°μ μννΈμ¨μ΄ μμ€ν
μμ© μμ μ μ΄μ’
λ©ν° νλ‘μΈμλ₯Ό νμ©νλ μ격 λ₯ λ¬λ μμ λ₯Ό μννμ¬ κ°λ° λ°©λ²λ‘ μ μ μ© κ°λ₯μ±μ μννμλ€. λν, νλ‘κ·Έλ¨ ν©μ±κΈ°κ° μλ‘μ΄ νλ«νΌμ΄λ λ€νΈμν¬λ₯Ό μ§μνκΈ° μν΄ νμλ‘ νλ κ°λ° λΉμ©λ μ€μ μΈ‘μ λ° μμΈ‘νμ¬ μλμ μΌλ‘ μ μ λ
Έλ ₯μΌλ‘ μλ‘μ΄ νλ«νΌμ μ§μν μ μμμ νμΈνμλ€.
λ§μ μλ² λλ μμ€ν
μμ μμμΉ λͺ»ν νλμ¨μ΄ μλ¬μ λν΄ κ²°ν¨μ κ°λ΄νλ κ²μ νμλ‘ νκΈ° λλ¬Έμ κ²°ν¨ κ°λ΄μ λν μ½λλ₯Ό μλμΌλ‘ μμ±νλ μ°κ΅¬λ μ§ννμλ€. λ³Έ κΈ°λ²μμ κ²°ν¨ κ°λ΄ μ€μ μ λ°λΌ νμ€ν¬ κ·Έλνλ₯Ό μμ νλ λ°©μμ νμ©νμμΌλ©°, κ²°ν¨ κ°λ΄μ λΉκΈ°λ₯μ μꡬ μ¬νμ μμ© κ°λ°μκ° μ½κ² μ μ©ν μ μλλ‘ νμλ€. λν, κ²°ν¨ κ°λ΄ μ§μνλ κ²κ³Ό κ΄λ ¨νμ¬ μ€μ μλμΌλ‘ ꡬννμ κ²½μ°μ λΉκ΅νμκ³ , κ²°ν¨ μ£Όμ
λꡬλ₯Ό μ΄μ©νμ¬ κ²°ν¨ λ°μ μλ리μ€λ₯Ό μ¬ννκ±°λ, μμλ‘ κ²°ν¨μ μ£Όμ
νλ μ€νμ μννμλ€.
λ§μ§λ§μΌλ‘ κ²°ν¨ κ°λ΄λ₯Ό μ€νν λμ νμ©ν κ²°ν¨ μ£Όμ
λꡬλ λ³Έ λ
Όλ¬Έμ λ λ€λ₯Έ κΈ°μ¬ μ¬ν μ€ νλλ‘ λ¦¬λ
μ€ νκ²½μΌλ‘ λμμΌλ‘ μμ© μμ λ° μ»€λ μμμ κ²°ν¨μ μ£Όμ
νλ λꡬλ₯Ό κ°λ°νμλ€. μμ€ν
μ κ²¬κ³ μ±μ κ²μ¦νκΈ° μν΄ κ²°ν¨μ μ£Όμ
νμ¬ κ²°ν¨ μλ리μ€λ₯Ό μ¬ννλ κ²μ λ리 μ¬μ©λλ λ°©λ²μΌλ‘, λ³Έ λ
Όλ¬Έμμ κ°λ°λ κ²°ν¨ μ£Όμ
λꡬλ μμ€ν
μ΄ λμνλ λμ€μ μ¬ν κ°λ₯ν κ²°ν¨μ μ£Όμ
ν μ μλ λꡬμ΄λ€. 컀λ μμμμμ κ²°ν¨ μ£Όμ
μ μν΄ λ μ’
λ₯μ κ²°ν¨ μ£Όμ
λ°©λ²μ μ 곡νλ©°, νλλ 컀λ GNU λλ²κ±°λ₯Ό μ΄μ©ν λ°©λ²μ΄κ³ , λ€λ₯Έ νλλ ARM νλμ¨μ΄ λΈλ μ΄ν¬ν¬μΈνΈλ₯Ό νμ©ν λ°©λ²μ΄λ€. μμ© μμμμ κ²°ν¨μ μ£Όμ
νκΈ° μν΄ GDB κΈ°λ° κ²°ν¨ μ£Όμ
λ°©λ²μ μ΄μ©νμ¬ λμΌ μμ€ν
νΉμ μ격 μμ€ν
μ μμ©μ κ²°ν¨μ μ£Όμ
ν μ μλ€. κ²°ν¨ μ£Όμ
λꡬμ λν μ€νμ ODROID-XU4 보λμμ μ§ννμλ€.While various software development methodologies have been proposed to increase the design productivity and maintainability of software, they usually focus on the development of application software running on a single processing element, without concern about the non-functional requirements of an embedded system such as latency and resource requirements.
In this thesis, we present a model-based software development method for parallel and distributed embedded systems. An application is specified as a set of tasks that follow a set of given rules for communication and synchronization in a hierarchical fashion, independently of the hardware platform. Having such rules enables us to perform static analysis to check some software errors at compile time to reduce the verification difficulty. Platform-specific program is synthesized automatically after mapping of tasks onto processing elements is determined.
The program synthesizer is also proposed to generate codes which satisfies platform requirements for parallel and distributed embedded systems. As multiple models which can express dynamic behaviors can be depicted hierarchically, the synthesizer supports to manage multiple task graphs with a different hierarchy to run tasks with parallelism. Also, the synthesizer shows methods of managing codes for heterogeneous platforms and generating various communication methods. The viability of the proposed software development method is verified with a real-life surveillance application that runs on six processing elements with three remote communication methods, and remote deep learning example is conducted to use heterogeneous multiprocessing components on distributed systems. Also, supporting a new platform and network requires a small effort by measuring and estimating development costs.
Since tolerance to unexpected errors is a required feature of many embedded systems, we also support an automatic fault-tolerant code generation. Fault tolerance can be applied by modifying the task graph based on the selected fault tolerance configurations, so the non-functional requirement of fault tolerance can be easily adopted by an application developer. To compare the effort of supporting fault tolerance, manual implementation of fault tolerance is performed. Also, the fault tolerance method is tested with the fault injection tool to emulate fault scenarios and inject faults randomly.
Our fault injection tool, which has used for testing our fault-tolerance method, is another work of this thesis. Emulating fault scenarios by intentionally injecting faults is commonly used to test and verify the robustness of a system. To emulate faults on an embedded system, we present a run-time fault injection framework that can inject a fault on both a kernel and application layer of Linux-based systems. For injecting faults on a kernel layer, two complementary fault injection techniques are used. One is based on Kernel GNU Debugger, and the other is using a hardware breakpoint supported by the ARM architecture. For application-level fault injection, the GDB-based fault injection method is used to inject a fault on a remote application. The viability of the proposed fault injection tool is proved by real-life experiments with an ODROID-XU4 system.Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Contribution 6
1.3 Dissertation Organization 8
Chapter 2 Background 9
2.1 HOPES: Hope of Parallel Embedded Software 9
2.1.1 Software Development Procedure 9
2.1.2 Components of HOPES 12
2.2 Universal Execution Model 13
2.2.1 Task Graph Specification 13
2.2.2 Dataflow specification of an Application 15
2.2.3 Task Code Specification and Generic APIs 21
2.2.4 Meta-data Specification 23
Chapter 3 Program Synthesis for Parallel and Distributed Embedded Systems 24
3.1 Motivational Example 24
3.2 Program Synthesis Overview 26
3.3 Program Synthesis from Hierarchically-mixed Models 30
3.4 Platform Code Synthesis 33
3.5 Communication Code Synthesis 36
3.6 Experiments 40
3.6.1 Development Cost of Supporting New Platforms and Networks 40
3.6.2 Program Synthesis for the Surveillance System Example 44
3.6.3 Remote GPU-accelerated Deep Learning Example 46
3.7 Document Generation 48
3.8 Related Works 49
Chapter 4 Model Transformation for Fault-tolerant Code Synthesis 56
4.1 Fault-tolerant Code Synthesis Techniques 56
4.2 Applying Fault Tolerance Techniques in HOPES 61
4.3 Experiments 62
4.3.1 Development Cost of Applying Fault Tolerance 62
4.3.2 Fault Tolerance Experiments 62
4.4 Random Fault Injection Experiments 65
4.5 Related Works 68
Chapter 5 Fault Injection Framework for Linux-based Embedded Systems 70
5.1 Background 70
5.1.1 Fault Injection Techniques 70
5.1.2 Kernel GNU Debugger 71
5.1.3 ARM Hardware Breakpoint 72
5.2 Fault Injection Framework 74
5.2.1 Overview 74
5.2.2 Architecture 75
5.2.3 Fault Injection Techniques 79
5.2.4 Implementation 83
5.3 Experiments 90
5.3.1 Experiment Setup 90
5.3.2 Performance Comparison of Two Fault Injection Methods 90
5.3.3 Bit-flip Fault Experiments 92
5.3.4 eMMC Controller Fault Experiments 94
Chapter 6 Conclusion 97
Bibliography 99
μ μ½ 108Docto
Relay: A New IR for Machine Learning Frameworks
Machine learning powers diverse services in industry including search,
translation, recommendation systems, and security. The scale and importance of
these models require that they be efficient, expressive, and portable across an
array of heterogeneous hardware devices. These constraints are often at odds;
in order to better accommodate them we propose a new high-level intermediate
representation (IR) called Relay. Relay is being designed as a
purely-functional, statically-typed language with the goal of balancing
efficient compilation, expressiveness, and portability. We discuss the goals of
Relay and highlight its important design constraints. Our prototype is part of
the open source NNVM compiler framework, which powers Amazon's deep learning
framework MxNet
Dataflow development of medium-grained parallel software
PhD ThesisIn the 1980s, multiple-processor computers (multiprocessors) based on conven-
tional processing elements emerged as a popular solution to the continuing demand
for ever-greater computing power. These machines offer a general-purpose parallel
processing platform on which the size of program units which can be efficiently
executed in parallel - the "grain size" - is smaller than that offered by distributed
computing environments, though greater than that of some more specialised
architectures. However, programming to exploit this medium-grained parallelism
remains difficult. Concurrent execution is inherently complex, yet there is a lack of
programming tools to support parallel programming activities such as program
design, implementation, debugging, performance tuning and so on.
In helping to manage complexity in sequential programming, visual tools have
often been used to great effect, which suggests one approach towards the goal of
making parallel programming less difficult.
This thesis examines the possibilities which the dataflow paradigm has to offer
as the basis for a set of visual parallel programming tools, and presents a dataflow
notation designed as a framework for medium-grained parallel programming. The
implementation of this notation as a programming language is discussed, and its
suitability for the medium-grained level is examinedScience and Engineering Research Council of Great Britain
EC ERASMUS schem
Advances in Architectures and Tools for FPGAs and their Impact on the Design of Complex Systems for Particle Physics
The continual improvement of semiconductor technology has provided rapid advancements in device frequency and density. Designers of electronics systems for high-energy physics (HEP) have benefited from these advancements, transitioning many designs from fixed-function ASICs to more flexible FPGA-based platforms. Todayβs FPGA devices provide a significantly higher amount of resources than those available during the initial Large Hadron Collider design phase. To take advantage of the capabilities of future FPGAs in the next generation of HEP experiments, designers must not only anticipate further improvements in FPGA hardware, but must also adopt design tools and methodologies that can scale along with that hardware. In this paper, we outline the major trends in FPGA hardware, describe the design challenges these trends will present to developers of HEP electronics, and discuss a range of techniques that can be adopted to overcome these challenges
- β¦