Analysis and evaluation of embedded graphics solutions for critical systems

En el camp dels sistemes crítics, que inclou l'automotriu, l'aviònica i els sistemes espacials, es necessita més capacitat de computació per aportar tant valor funcional com seguretat addicional. Per aconseguir-ho, la indústria està considerant noves arquitectures per futurs sistemes crítics. Una de les possibles opcions és l'ús de targetes gràfiques mòbils, que tenen un rendiment excel·lent per tasques computacionals complexes i un baix nivell de consum. Per desgràcia, les eines actuals de desenvolupament per programació de propòsit general de targetes gràfiques com CUDA o OpenCL no compleixen amb les regulacions dels estàndards de seguretat dels sistemes crítics segurs. Per altra banda, hi ha altres solucions per programar per gràfics, com ara OpenGL SC 2 i Brook Auto, que són fàcils de certificar. En aquest projecte, analitzem aquestes solucions per programar per targetes gràfiques i explorem els diferents aspectes del desenvolupament de programari de propòsit general amb elles. Us presentem la nostra experiència adaptant codi de dues aplicacions de dos sectors diferents de sistemes crítics, l'aviònica i els sistemes espacials, a diferents \textit{APIs} (OpenGL 2, OpenGL ES 2, OpenGL SC 2 i Brook Auto) i l'avaluació de les versions que nosaltres hem generat. En funcionalitat i rendiment, no s'ha observat cap diferència, tot i que sí que hem notat un gran salt comparatiu en la complexitat del desenvolupament i la productivitat entre eines orientades només a sistemes gràfics i Brook Auto.In the safety-critical systems domain, which includes automotive, avionics and space systems, more compute power is needed to provide additional functional value and safety. In order to achieve this, new hardware architectures are considered from industry for future critical systems. One of this approaches is the use of mobile GPUs, which have excellent performance capabilities for intensive computational tasks and low-power consumption. However, current programming models for general purpose programming of GPUs like CUDA and OpenCL do not comply with the safety standards of safety critical systems. On the other hand, there are alternative programming solutions based on graphics, namely OpenGL SC 2 and Brook Auto, which are certification-friendly. In this thesis, we perform an analysis of these safety-critical programming models for GPUs and we explore the different aspects of the development of general purpose software in them. We present our experience with porting two applications from two distinct safety-critical domains, aerospace and avionics, in several graphics-based APIs (OpenGL 2, OpenGL ES 2, OpenGL SC 2 and Brook Auto) and the evaluation of our produced versions. In terms of functionality and performance, no difference has been observed, whereas we noticed a big gap in the development complexity and productivity between pure graphics solutions and Brook Auto

Hydra: loosely coupling the graphics pipeline to facilitate digital preservation.

It can be argued that software can be seen as a form of art and digital heritage and yet it rarely enjoys the same efforts afforded to it compared to physical counterparts. There are many reasons for this, such as the increasing costs of maintenance or the reducing amount of expertise in the specific aging technology. Maintaining software and ensuring that it continues to work on current hardware and operating systems is known as digital preservation. There are many ways in which we can attempt to preserve digital software and one of the most effective ones is by using emulation to simulate the obsolete hardware. However, for games and other entertainment media, this technique is not always effective due to a requirement on specific hardware, such as an accelerated GPU in order to reach an acceptable performance for the user. It is often difficult to emulate a GPU and, as such, a different approach often needs to be taken, which reduces the flexibility and portability of the emulation software. Hydra is a new approach to accessing the native hardware from within an emulated environment which allows for a much simpler emulator to be developed and maintained and yet also offers the potential of accessing other types of hardware without needing to modify the emulation software itself. Hydra is designed to be platform agnostic in that not only is it possible to integrate with existing emulators but also be immediately usable from within guest operating systems, ranging from legacy platforms such as MS-DOS, through to modern platforms such as the PlayStation 4 (Orbis OS, a FreeBSD derivative), through to more exotic platforms such as Plan 9 from Bell Laboratories. It can do this because it does not rely on a complex emulator-specific virtual driver stack. This PhD thesis provides the research undertaken for Hydra, including the motivation behind it, the specific problems it was designed to solve and how it can be implemented in a platform agnostic manner. Hydra’s performance is analysed to ascertain the suitability of the output to cater for, specifically, a wide variety of platforms that it can run on in a satisfactory manner within less powerful or emulated environments. A performance analysis study is conducted to ensure that the technology provides an acceptable solution to accessing preserved titles. This study concluded with results showing that Hydra offers a greater performance than software rendering, especially within emulated environments. A bandwidth comparison between Hydra and VNC was undertaken to ascertain the use of the technology as a streaming medium. The results concluded that under specific conditions, Hydra performed better than VNC by streaming at a higher resolution and consuming less bandwidth. Hydra is also utilised in a number of engineering tasks relating to preservation of software. The experiences of using Hydra in this way are discussed, including any difficulties encountered. Lastly, a conclusion is made and any future work is identified

Basil Leaf Automation

Recent population and wage increases have forced farmers to grow more food without a proportionate increase in work force. Automation is a key factor in reducing cost and increasing efficiency. In this paper, we explore our automation solution that utilizes position manipulation and vision processing to identify, pick up, and drop a leaf into a can. Two stepper motors and a linear actuator drove the three-dimensional actuation. Leaf and can recognition were accomplished through edge detection and machine learning algorithms. Testing proved subsystem-level functionality and proof of concept of a delicate autonomous pick-and-place robot

Selection of a new hardware and software platform for railway interlocking

The interlocking system is one of the main actors for safe railway transportation. In most cases, the whole system is supplied by a single vendor. The recent regulations from the European Union direct for an “open” architecture to invite new game changers and reduce life-cycle costs. The objective of the thesis is to propose an alternative platform that could replace a legacy interlocking system. In the thesis, various commercial off-the-shelf hardware and software products are studied which could be assembled to compose an alternative interlocking platform. The platform must be open enough to adapt to any changes in the constituent elements and abide by the proposed baselines of new standardization initiatives, such as ERTMS, EULYNX, and RCA. In this thesis, a comparative study is performed between these products based on hardware capacity, architecture, communication protocols, programming tools, security, railway certifications, life-cycle issues, etc

Real-time scheduling for 3D rendering on automotive embedded systems

Im Automobilbereich erfreut sich der Einsatz von 3D-Grafik zunehmender Beliebtheit. Beispielsweise zeigte Mercedes-Benz im F125 Autoprototypen, wie analoge Zeiger der Kombiinstrumente durch digitale Displays ersetzt werden. Der Trend, 3D-Anwendungen zu nutzen, geht in zwei Richtungen: Zum einen hin zu kritischeren Anwendungen wie der Geschwindigkeitsanzeige, zum anderen hin zu Drittanbieteranwendungen, die beispielsweise über einen Appstore bezogen werden. Um Isolationsanforderungen zu erfüllen, werden traditionell neue Funktionen im Auto häufig mittels neuer Steuergeräte umgesetzt. Um jedoch Kosten, Energieverbrauch und Bauraum im Fahrzeug zu sparen, sollten alle 3D-Anwendungen eine einzige Hardwareplattform und somit auch eine einzige GPU als gemeinsame Ressource nutzen. Für zeitsensitive Anwendungen wie die Geschwindigkeitsanzeige ergibt sich hierbei die Herausforderung, Rendering in Echtzeit zu gewährleisten. Hierfür sind wirksame Konzepte für das Echtzeitscheduling der GPU erforderlich, welche Sicherheit und Isolation beim 3D-Rendering garantieren können. Da aktuelle GPUs nicht unterbrechbar sind, muss ein Deadline-basierter Scheduler die Ausführungszeit der GPU-Befehle im Voraus kennen. Bestehende Schedulingkonzepte unterstützen leider keine dynamischen Tasks, keine periodischen Echtzeitdeadlines, oder setzen unterbrechbare Ausführung voraus. In dieser Arbeit werden die für HMI-Rendering im Automobilbereich relevanten Anforderungen beschrieben. Basierend auf diesen Anforderungen wird das Konzept des virtualisierten automobilen Grafiksystems (VAGS) vorgestellt, welches einen Hypervisor nutzt um die Isolation zwischen verschiedenen VMs, insbesondere für die Headunit und die Kombiinstrumente, sicherzustellen. Des Weiteren wird ein neuartiges Framework vorgestellt, welches die Ausführungszeit von GPU-Befehlen misst und basierend auf OpenGL ES 2.0 vorhersagt. Hierbei werden für die relevanten GPU-Befehle wie Draw und SwapBuffers Vorhersagemodelle vorgestellt. Für Draw-Befehle werden zwei Heuristiken vorgeschlagen, welche die Anzahl der Fragmente abschätzen, zwei Konzepte, welche die Ausführungszeit der Grafikshader vorhersagen, sowie ein optionaler Echtzeit-Korrekturmechanismus. Die Anzahl der Fragmente wird entweder mittels einer Bounding-Box des gerenderten Modells, auf welche die Projektion des Vertexshaders angewendet wird, abgeschätzt, oder durch eine Teilmenge der gerenderten Dreiecke, welche genutzt wird um die Durchschnittsgröße eines Dreiecks zu ermitteln. Um die Laufzeit eines Shaders abzuschätzen, wird er entweder in einer Kalibrierungsumgebung in einem separaten OpenGL-Kontext ausgeführt, oder es wird ein offline trainiertes MARS-Modell verwendet. Die Implementierung und die Auswertungen des Frameworks zeigen dessen Machbarkeit und dass eine gute Vorhersagegenauigkeit erreicht werden kann. Beim Rendern einer Szene des bekannten Benchmarkprogramms Glmark2 wurden beispielsweise weniger 0,4 % der Messproben um mehr als 100 μs unterschätzt und weniger als 0,2 % der Messproben um mehr als 100 μs überschätzt. Unsere Implementierung verursacht bei langer Ausführung eine zusätzliche CPU-Rechenzeit von üblicherweise weniger als 25 %, bei manchen Szenarien ist diese sogar vernachlässigbar. Der Programmstart verlangsamt sich beim effizientesten Verfahren hierbei lediglich um etwa 30 ms. Auf lange Sicht liegt er typischerweise unter 25 % und ist für manche Szenarien sogar vernachlässigbar. Darüber hinaus wird ein echtzeitfähiges 3D-GPU-Schedulingframework vorgestellt, welches kritischen Anwendungen Garantien gibt und trotzdem die verbleibenden GPU-Ressourcen den weniger kritischen Anwendungen zur Verfügung stellt, wodurch eine hohe GPU-Auslastung erreicht wird. Da aktuelle GPUs nicht unterbrechbar sind, werden die vorgestellten Konzepte zur Vorhersage der Ausführungszeit verwendet um prioritätsbasiert Scheduling-Entscheidungen zu treffen. Die Implementierung basiert auf einem automobilkonformen eingebetteten System, auf welchem Linux ausgeführt wird. Die darauf ausgeführten Auswertungen zeigen die Machbarkeit und Wirksamkeit der vorgestellten Konzepte. Der GPU-Scheduler erfüllt die jeweiligen Echtzeitvorgaben für eine variable Anzahl von Anwendungen, welche unterschiedliche GPU-Befehlsfolgen erzeugen. Hierbei wird bei einem anspruchsvollen Szenario mit 17 Anwendungen eine hohe GPU-Auslastung von 99 % erzielt und 99,9 % der Deadlines der höchstprioren Anwendung erfüllt. Des Weiteren wird das Scheduling in Echtzeit mit weniger als 9 μs Latenz effizient ausgeführt

A Modular and Open-Source Framework for Virtual Reality Visualisation and Interaction in Bioimaging

Life science today involves computational analysis of a large amount and variety of data, such as volumetric data acquired by state-of-the-art microscopes, or mesh data from analysis of such data or simulations. The advent of new imaging technologies, such as lightsheet microscopy, has resulted in the users being confronted with an ever-growing amount of data, with even terabytes of imaging data created within a day. With the possibility of gentler and more high-performance imaging, the spatiotemporal complexity of the model systems or processes of interest is increasing as well. Visualisation is often the first step in making sense of this data, and a crucial part of building and debugging analysis pipelines. It is therefore important that visualisations can be quickly prototyped, as well as developed or embedded into full applications. In order to better judge spatiotemporal relationships, immersive hardware, such as Virtual or Augmented Reality (VR/AR) headsets and associated controllers are becoming invaluable tools. In this work we present scenery, a modular and extensible visualisation framework for the Java VM that can handle mesh and large volumetric data, containing multiple views, timepoints, and color channels. scenery is free and open-source software, works on all major platforms, and uses the Vulkan or OpenGL rendering APIs. We introduce scenery's main features, and discuss its use with VR/AR hardware and in distributed rendering. In addition to the visualisation framework, we present a series of case studies, where scenery can provide tangible benefit in developmental and systems biology: With Bionic Tracking, we demonstrate a new technique for tracking cells in 4D volumetric datasets via tracking eye gaze in a virtual reality headset, with the potential to speed up manual tracking tasks by an order of magnitude. We further introduce ideas to move towards virtual reality-based laser ablation and perform a user study in order to gain insight into performance, acceptance and issues when performing ablation tasks with virtual reality hardware in fast developing specimen. To tame the amount of data originating from state-of-the-art volumetric microscopes, we present ideas how to render the highly-efficient Adaptive Particle Representation, and finally, we present sciview, an ImageJ2/Fiji plugin making the features of scenery available to a wider audience.:Abstract Foreword and Acknowledgements Overview and Contributions Part 1 - Introduction 1 Fluorescence Microscopy 2 Introduction to Visual Processing 3 A Short Introduction to Cross Reality 4 Eye Tracking and Gaze-based Interaction Part 2 - VR and AR for System Biology 5 scenery — VR/AR for Systems Biology 6 Rendering 7 Input Handling and Integration of External Hardware 8 Distributed Rendering 9 Miscellaneous Subsystems 10 Future Development Directions Part III - Case Studies C A S E S T U D I E S 11 Bionic Tracking: Using Eye Tracking for Cell Tracking 12 Towards Interactive Virtual Reality Laser Ablation 13 Rendering the Adaptive Particle Representation 14 sciview — Integrating scenery into ImageJ2 & Fiji Part IV - Conclusion 15 Conclusions and Outlook Backmatter & Appendices A Questionnaire for VR Ablation User Study B Full Correlations in VR Ablation Questionnaire C Questionnaire for Bionic Tracking User Study List of Tables List of Figures Bibliography Selbstständigkeitserklärun

A wrapper generation tool for the creation of scriptable scientific applications

Journal ArticleIn recent years, there has been considerable interest in the use of scripting languages as a mechanism for controlling and developing scientific software. Scripting languages allow scientific applications to be encapsulated in an interpreted environment similar to that found in commercial scientific packages such as MATLAB, Mathematica, and IDL. This improves the usability of scientific software by providing a powerful meachanism for specifyling and controlling cimplex problems as well as giving users an interactive and exploratory problem solving environment. Scripting languages also provide a framework for building and integrating software components that allows tools be used in a more efficient manner. This streamlines the problem solving process and enable scientists to be more productive

Shader optimization and specialization

In the field of real-time graphics for computer games, performance has a significant effect on the player’s enjoyment and immersion. Graphics processing units (GPUs) are hardware accelerators that run small parallelized shader programs to speed up computationally expensive rendering calculations. This thesis examines optimizing shader programs and explores ways in which data patterns on both the CPU and GPU can be analyzed to automatically speed up rendering in games. Initially, the effect of traditional compiler optimizations on shader source-code was explored. Techniques such as loop unrolling or arithmetic reassociation provided speed-ups on several devices, but different GPU hardware responded differently to each set of optimizations. Analyzing execution traces from numerous popular PC games revealed that much of the data passed from CPU-based API calls to GPU-based shaders is either unused, or remains constant. A system was developed to capture this constant data and fold it into the shaders’ source-code. Re-running the game’s rendering code using these specialized shader variants resulted in performance improvements in several commercial games without impacting their visual quality

Doctor of Philosophy

dissertationStochastic methods, dense free-form mapping, atlas construction, and total variation are examples of advanced image processing techniques which are robust but computationally demanding. These algorithms often require a large amount of computational power as well as massive memory bandwidth. These requirements used to be ful lled only by supercomputers. The development of heterogeneous parallel subsystems and computation-specialized devices such as Graphic Processing Units (GPUs) has brought the requisite power to commodity hardware, opening up opportunities for scientists to experiment and evaluate the in uence of these techniques on their research and practical applications. However, harnessing the processing power from modern hardware is challenging. The di fferences between multicore parallel processing systems and conventional models are signi ficant, often requiring algorithms and data structures to be redesigned signi ficantly for efficiency. It also demands in-depth knowledge about modern hardware architectures to optimize these implementations, sometimes on a per-architecture basis. The goal of this dissertation is to introduce a solution for this problem based on a 3D image processing framework, using high performance APIs at the core level to utilize parallel processing power of the GPUs. The design of the framework facilitates an efficient application development process, which does not require scientists to have extensive knowledge about GPU systems, and encourages them to harness this power to solve their computationally challenging problems. To present the development of this framework, four main problems are described, and the solutions are discussed and evaluated: (1) essential components of a general 3D image processing library: data structures and algorithms, as well as how to implement these building blocks on the GPU architecture for optimal performance; (2) an implementation of unbiased atlas construction algorithms|an illustration of how to solve a highly complex and computationally expensive algorithm using this framework; (3) an extension of the framework to account for geometry descriptors to solve registration challenges with large scale shape changes and high intensity-contrast di fferences; and (4) an out-of-core streaming model, which enables developers to implement multi-image processing techniques on commodity hardware

PC-grade parallel processing and hardware acceleration for large-scale data analysis

Arguably, modern graphics processing units (GPU) are the first commodity, and desktop parallel processor. Although GPU programming was originated from the interactive rendering in graphical applications such as computer games, researchers in the field of general purpose computation on GPU (GPGPU) are showing that the power, ubiquity and low cost of GPUs makes them an ideal alternative platform for high-performance computing. This has resulted in the extensive exploration in using the GPU to accelerate general-purpose computations in many engineering and mathematical domains outside of graphics. However, limited to the development complexity caused by the graphics-oriented concepts and development tools for GPU-programming, GPGPU has mainly been discussed in the academic domain so far and has not yet fully fulfilled its promises in the real world. This thesis aims at exploiting GPGPU in the practical engineering domain and presented a novel contribution to GPGPU-driven linear time invariant (LTI) systems that are employed by the signal processing techniques in stylus-based or optical-based surface metrology and data processing. The core contributions that have been achieved in this project can be summarized as follow. Firstly, a thorough survey of the state-of-the-art of GPGPU applications and their development approaches has been carried out in this thesis. In addition, the category of parallel architecture pattern that the GPGPU belongs to has been specified, which formed the foundation of the GPGPU programming framework design in the thesis. Following this specification, a GPGPU programming framework is deduced as a general guideline to the various GPGPU programming models that are applied to a large diversity of algorithms in scientific computing and engineering applications. Considering the evolution of GPU’s hardware architecture, the proposed frameworks cover through the transition of graphics-originated concepts for GPGPU programming based on legacy GPUs and the abstraction of stream processing pattern represented by the compute unified device architecture (CUDA) in which GPU is considered as not only a graphics device but a streaming coprocessor of CPU. Secondly, the proposed GPGPU programming framework are applied to the practical engineering applications, namely, the surface metrological data processing and image processing, to generate the programming models that aim to carry out parallel computing for the corresponding algorithms. The acceleration performance of these models are evaluated in terms of the speed-up factor and the data accuracy, which enabled the generation of quantifiable benchmarks for evaluating consumer-grade parallel processors. It shows that the GPGPU applications outperform the CPU solutions by up to 20 times without significant loss of data accuracy and any noticeable increase in source code complexity, which further validates the effectiveness of the proposed GPGPU general programming framework. Thirdly, this thesis devised methods for carrying out result visualization directly on GPU by storing processed data in local GPU memory through making use of GPU’s rendering device features to achieve realtime interactions. The algorithms employed in this thesis included various filtering techniques, discrete wavelet transform, and the fast Fourier Transform which cover the common operations implemented in most LTI systems in spatial and frequency domains. Considering the employed GPUs’ hardware designs, especially the structure of the rendering pipelines, and the characteristics of the algorithms, the series of proposed GPGPU programming models have proven its feasibility, practicality, and robustness in real engineering applications. The developed GPGPU programming framework as well as the programming models are anticipated to be adaptable for future consumer-level computing devices and other computational demanding applications. In addition, it is envisaged that the devised principles and methods in the framework design are likely to have significant benefits outside the sphere of surface metrology.EThOS - Electronic Theses Online ServiceGBUnited Kingdo