Automated CNN pipeline generation for heterogeneous architectures

Heterogeneity is a vital feature in emerging processor chip designing. Asymmetric multicore-clusters such as high-performance cluster and power efficient cluster are common in modern edge devices. One example is Intel\u27s Alder Lake featuring Golden Cove high-performance cores and Gracemont power-efficient cores. Chiplet-based technology allows organization of multi cores in form of multi-chip-modules, thus housing large number of cores in a processor. Interposer based packaging has enabled embedding High Bandwidth Memory (HBM) on chip and reduced transmission latency and energy consumption of chiplet-chiplet interconnect.\ua0For Instance Intel\u27s XeHPC Ponte Vecchio package integrates multi-chip GPU organization along with HBM modules.Since new devices feature heterogeneity at the level of cores, memory and on-chip interconnect, it has become important to steer optimization at application level in order to leverage the new heterogeneous, high-performing and power-efficient features of underlying computing platforms. An important high-performance application paradigm is Convolution Neural Networks (CNN). CNNs are widely used in many practical applications. The pipelined parallel implementation of CNN is favored for inference on edge devices. In this Licentiate thesis we present a novel scheme for automatic scheduling of CNN pipelines on heterogeneous devices. A pipeline schedule is a configuration that provides information on depth of pipeline, grouping of CNN layers into pipeline stages and mapping of pipeline stages onto computing units. We utilize simple compile-time hints which consists of workload information of individual CNN layers and performance hints of computing units.The proposed approach provides near optimal solution for a throughput maximizing pipeline. We model the problem as a design space exploration technique. We developed a time-efficient design space navigation through heuristics extracted from the knowledge of CNN structure and underlying computing platform. The proposed search scheme converges faster and utilizes real-time performance measurements as fitness values. The results demonstrate that the proposed scheme converges faster and can scale when used with larger networks and computing platforms. Since the scheme utilizes online performance measurements, one of the challenges is to avoid expensive configurations during online tuning. The results demonstrate that on average, ~80\% of the tested configurations are sub-optimal solutions.Another challenge is to reduce convergence time. The experiments show that proposed approach is 35x faster than stochastic optimization algorithms. Since the design space is large and complex, We show that the proposed scheme explores only ~0.1% of the total design space in case of large CNNs (having 50+ layers) and results in near-optimal solution

X-MAP A Performance Prediction Tool for Porting Algorithms and Applications to Accelerators

Most modern high-performance computing systems comprise of one or more accelerators with varying architectures in addition to traditional multicore Central Processing Units (CPUs). Examples of these accelerators include Graphic Processing Units (GPU) and Intel’s Many Integrated Cores architecture called Xeon Phi (PHI). These architectures provide massive parallel computation capabilities, which provide substantial performance benefits over traditional CPUs for a variety of scientific applications. We know that all accelerators are not similar because each of them has their own unique architecture. This difference in the underlying architecture plays a crucial role in determining if a given accelerator will provide a significant speedup over its competition. In addition to the architecture itself, one more differentiating factor for these accelerators is the programming language used to program them. For example, Nvidia GPUs can be programmed using Compute Unified Device Architecture (CUDA) and OpenCL while Intel Xeon PHIs can be programmed using OpenMP and OpenCL. The choice of programming language also plays a critical role in the speedup obtained depending on how close the language is to the hardware in addition to the level of optimization. With that said, it is thus very difficult for an application developer to choose the ideal accelerator to achieve the best possible speedup. In light of this, we present an easy to use Graphical User Interface (GUI) Tool called X-MAP which is a performance prediction tool for porting algorithms and applications to architectures which encompasses a Machine Learning based inference model to predict the performance of an applica-tion on a number of well-known accelerators and at the same time predict the best architecture and programming language for the application. We do this by collecting hardware counters from a given application and predicting run time by providing this data as inputs to a Neural Network Regressor based inference model. We predict the architecture and associated programming language by pro viding the hardware counters as inputs to an inference model based on Random Forest Classification Model. Finally, with a mean absolute prediction error of 8.52 and features such as syntax high-lighting for multiple programming languages, a function-wise breakdown of the entire application to understand bottlenecks and the ability for end users to submit their own prediction models to further improve the system, makes X-MAP a unique tool that has a significant edge over existing performance prediction solutions

Models, Optimizations, and Tools for Large-Scale Phylogenetic Inference, Handling Sequence Uncertainty, and Taxonomic Validation

Das Konzept der Evolution ist in der modernen Biologie von zentraler Bedeutung. Deswegen liefert die Phylogenetik, die Lehre über die Verwandschaften und Abstam- mung von Organismen bzw. Spezies, entscheidende Hinweise zur Entschlüsselung einer Vielzahl biologischer Prozesse. Phylogenetische Stammbäume sind einerseits für die Grundlagenforschung wichtig, da sie in Studien über die Diversifizierung und Umweltanpassung einzelner Organismengruppen (z.B. Insekten oder Vögel) bis hin zu der großen Herausforderung, die Entstehung und Entwicklung aller Lebensfor- men in einem umfassenden evolutionären Baum darzustellen (der sog. Tree of Life) Anwendung finden. Andererseits werden phylogenetische Methoden auch in prax- isnahen Anwendungen eingesetzt, um beispielsweise die Verbreitungsdynamik von HIV-Infektionen oder, die Heterogenität der Krebszellen eines Tumors, zu verstehen. Den aktuellen Stand der Technik in der Stammbaumrekonstruktion stellen Meth- oden Maximum Likelihood (ML) und Bayes’sche Inferenz (BI) dar, welche auf der Analyse molekularer Sequenzendaten (DNA und Proteine) anhand probabilistis- cher Evolutionsmodelle basieren. Diese Methoden weisen eine hohe Laufzeitkom- plexität auf (N P -schwer), welche die Entwicklung effizienter Heuristiken unabding- bar macht. Hinzu kommt, dass die Berechnung der Zielfunktion (sog. Phylogenetic Likelihood Function, PLF) neben einem hohen Speicherverbrauch auch eine Vielzahl an Gleitkommaarithmetik-Operationen erfordert und somit extrem rechenaufwendig ist. Die neuesten Entwicklungen im Bereich der DNA-Sequenzierung (Next Gener- ation Sequencing, NGS) steigern kontinuierlich den Durchsatz und senken zugleich die Sequenzierungskosten um ein Vielfaches. Für die Phylogenetik hat dies zur Folge, dass die Dimensionen der zu analysierenden Datensätze alle 2–3 Jahre, um eine Grössenordnung zunhemen. War es bisher üblich, einige Dutzend bis Hun- derte Spezies anhand einzelner bzw. weniger Gene zu analysieren (Sequenzlänge: 1–10 Kilobasen), stellen derzeit Studien mit Tausenden Sequenzen oder Genen keine Seltenheit mehr dar. In den nächsten 1–2 Jahren ist zu erwarten, dass die Anal- ysen Tausender bis Zehntausender vollständiger Genome bzw. Transkriptome (Se- quenzlänge: 1–100 Megabasen und mehr) anstehen. Um diesen Aufgaben gewachsen zu sein, müssen die bestehenden Methoden weiterentwickelt und optimiert werden, um vor allem Höchstleistungsrechner sowie neue Hardware-Architekturen optimal nutzen zu können. Außerdem führt die sich beschleunigende Speicherung von Sequenzen in öffentli- chen Datenbanken wie NCBI GenBank (und ihren Derivaten) dazu, dass eine hohe Qualität der Sequenzannotierungen (z. B. Organismus- bzw. Speziesname, tax- onomische Klassifikation, Name eines Gens usw.) nicht zwangsläufig gewährleistet ist. Das hängt unter anderem auch damit zusammen, dass eine zeitnahe Korrektur durch entsprechende Experten nicht mehr möglich ist, solange ihnen keine adäquaten Software-Tools zur Verfügung stehen. In dieser Doktroarbeit leisten wir mehrere Beiträge zur Bewältigung der oben genannten Herausforderungen. Erstens haben wir ExaML, eine dedizierte Software zur ML-basierten Stamm- baumrekonstruktion für Höchstleistungsrechner, auf den Intel Xeon Phi Hardware- beschleuniger portiert. Der Xeon Phi bietet im Vergleich zu klassischen x86 CPUs eine höhere Rechenleistung, die allerdings nur anhand architekturspezifischer Op- timierungen vollständig genutzt werden kann. Aus diesem Grund haben wir zum einen die PLF-Berechnung für die 512-bit-Vektoreinheit des Xeon Phi umstrukturi- ert und optimiert. Zum anderen haben wir die in ExaML bereits vorhandene reine MPI-Parallelisierung durch eine hybride MPI/OpenMP-Lösung ersetzt. Diese hy- bride Lösung weist eine wesentlich bessere Skalierbarkeit für eine hohe Zahl von Kernen bzw. Threads innerhalb eines Rechenknotens auf (>100 HW-Threads für Xeon Phi). Des Weiteren haben wir eine neue Software zur ML-Baumrekonstruktion na- mens RAxML-NG entwickelt. Diese implementiert, bis auf kleinere Anpassungen, zwar denselben Suchalgorithmus wie das weit verbreitete Programm RAxML, bietet aber gegenüber RAxML mehrere Vorteile: (a) dank den sorgfältigen Optimierungen der PLF-Berechnung ist es gelungen, die Laufzeiten um den Faktor 2 bis 3 zu reduzieren (b) die Skalierbarkeit auf extrem großen Eingabedatensätzen wurde verbessert, in- dem ineffiziente topologische Operationen eliminiert bzw. optimiert wurden, (c) die bisher nur in ExaML verfügbaren, für große Datensätze relevanten Funktionen wie Checkpointing sowie ein dedizierter Datenverteilungsalgorithmus wurden nachimple- mentiert (d) dem Benutzer steht eine größere Auswahl an statistischen DNA-Evo- lutionsmodellen zur Verfügung, die zudem flexibler kombiniert und parametrisiert werden können (e) die Weiterentwicklung der Software wird aufgrund der modularen Architektur wesentlich erleichtert (die Funktionen zur PLF-Berechnung wurden in eine gesonderte Bibliothek ausgeglidert). Als nächstes haben wir untersucht, wie sich Sequenzierungsfehler auf die Genau- igkeit phylogenetischr Stammbaumrekonstruktionen auswirken. Wir modifizieren den RAxML bzw. RAxML-NG Code dahingehend, dass sowohl die explizite Angabe von Fehlerwahrscheinlichkeiten als auch die automatische Schätzung von Fehlerraten mittels der ML-Methode möglich ist. Unsere Simulationen zeigen: (a) Wenn die Fehler gleichverteilt sind, kann die Fehlerrate direkt aus den Sequenzdaten geschätzt werden. (b) Ab einer Fehlerrate von ca. 1% liefert die Baumrekonstruktion unter Berücksichtigung des Fehlermodells genauere Ergebnisse als die klassische Methode, welche die Eingabe als fehlerfrei annimmt. Ein weiterer Beitrag im Rahmen dieser Arbeit ist die Software-Pipeline SATIVA zur rechnergestützten Identifizierung und Korrektur fehlerhafter taxonomischer An- notierungen in großen Sequenzendatenbanken. Der Algorithmus funktioniert wie folgt: für jede Sequenz wird die Platzierung im Stammbaum mit dem höchst- möglichen Likelihood-Wert ermittelt und anschließend geprüft, ob diese mit der vorgegeben taxonomischen Klassifikation übereinstimmt. Ist dies nicht der Fall, wird also eine Sequenz beispielsweise innerhalb einer anderen Gattung platziert, wird die Sequenz als falsch annotiert gemeldet, und es wird eine entsprechende Umklassifizierung vorgeschlagen. Auf simulierten Datensätzen mit zufällig eingefüg- ten Fehlern, erreichte unsere Pipeline eine hohe Identifikationsquote (>90%) sowie Genauigkeit (>95%). Zur Evaluierung anhand empirischer Daten, haben wir vier öffentliche rRNA Datenbanken untersucht, welche zur Klassifizierung von Bakterien häufig als Referenz benutzt werden. Dabei haben wir je nach Datenbank 0.2% bis 2.5% aller Sequenzen als potenzielle Fehlannotierungen identifiziert

Kastra: Architecture and Culture in the Aegean Archipelago

Final version of Kastra: Architecture and Culture in the Aegean Archipelago, published Summer 2018. “Kastra: Architecture and Culture in the Aegean Archipelago,” is a sequel to “The Aegean Crucible: Tracing Vernacular Architecture in Post-Byzantine Centuries,” published in 2004. “The Aegean Crucible” focused on the vernacular architecture of the Aegean archipelago, while “Kastra” focuses on the collective fortification, a building type vital to survival in the region, during the thirteenth-to- eighteenth-century period. “Kastra” was also written on the conviction that what we identify today as the vernacular architecture of the Aegean islands emerged from the building of Kastra, the medieval collective fortifications of the Aegean archipelago. “Kastra” is a book about architecture and culture, written by an architect and addressed to the general public rather than to specialists. Observations and “notes” in the form of color slides taken during repeated visits to the region form the basic skeleton of the book, which is also enriched by the helicopter-based photographs of Nikos Daniilidis. Includes bibliographical references, index and gazeteer. Contents: Doges, Knights, Pashas and Pirates. The Aegean Archipelago. The Vernacular Response: Collective Fortifications. The Formal response: Detached Fortification Walls. The Hybrid Response: Sharing Lessons. Constantine (Dinos) E. Michaelides, FAIA, is emeritus dean and professor of the School of Architecture at Washington University in St. Louis. Born in Athens, he received an architecture diploma from the National Technical University in 1952 and earned an M.Arch. from Harvard University, Graduate School of Design in 1957.https://openscholarship.wustl.edu/books/1041/thumbnail.jp

Labor problems of the longshoremen in the United States

Thesis (Ph.D.)--Boston Universit

New horizons for female birdsong : evolution, culture and analysis tools : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Ecology at Massey University, Auckland, New Zealand

Published papers appear in Appendix 7.1. and 7.2 respectively under a CC BY 4.0 and CC BY licence: Webb, W. H., Brunton, D. H., Aguirre, J. D., Thomas, D. B., Valcu, M., & Dale, J. (2016). Female song occurs in songbirds with more elaborate female coloration and reduced sexual dichromatism. Frontiers in Ecology and Evolution, 4(22). https://doi.org/10.3389/fevo.2016.00022 Yukio Fukuzawa, Wesley Webb, Matthew Pawley, Michelle Roper, Stephen Marsland, Dianne Brunton, & Andrew Gilman. (2020). Koe: Web-based software to classify acoustic units and analyse sequence structure in animal vocalisations. Methods in Ecology and Evolution, 11(3). https://doi.org/10.1111/2041-210X.13336As a result of male-centric, northern-hemisphere-biased sexual selection theory, elaborate female traits in songbirds have been largely overlooked as unusual or non-functional by-products of male evolution. However, recent research has revealed that female song is present in most surveyed songbirds and was in fact the ancestral condition to the clade. Additionally, a high proportion of songbird species have colourful females, and both song and showy colours have demonstrated female-specific functions in a growing number of species. We have much to learn about the evolution and functions of elaborate female traits in general, and female song in particular. This thesis extends the horizons of female birdsong research in three ways: (1) by revealing the broad-scale evolutionary relationship of female song and plumage elaboration across the songbirds, (2) by developing new accessible tools for the measurement and analysis of song complexity, and (3) by showing—through a detailed field study on a large natural metapopulation—how vocal culture operates differentially in males and females. First, to understand the drivers of elaborate female traits, I tested the evolutionary relationship between female song presence and plumage colouration across the songbirds. I found strong support for a positive evolutionary correlation between traits, with female song more prevalent amongst species with elaborated female plumage. These results suggest that contrary to the idea of trade-off between showy traits, female plumage colouration and female song likely evolved together under similar selection pressures and that their respective functions are reinforcing. Second, I introduce new bioacoustics software, Koe, designed to meet the need for detailed classification and analysis of song complexity. The program enables visualisation, segmentation, rapid classification and analysis of song structure. I demonstrate Koe with a case study of New Zealand bellbird Anthornis melanura song, showcasing the capabilities for large-scale bioacoustics research and its application to female song. Third, I conducted one of the first detailed field-based analyses of female song culture, studying an archipelago metapopulation of New Zealand bellbirds. Comparing between male and female sectors of each population, I found equal syllable diversity, largely separate repertoires, and contrasting patterns of sharing between sites—revealing female dialects and pronounced sex differences in cultural evolution. By combining broad-scale evolutionary approaches, novel song analysis tools, and a detailed field study, this thesis demonstrates that female song can be as much an elaborate signal as male song. I describe how future work can build on these findings to expand understanding of elaborate female traits

A framework for evaluating presence in VR narrative games

This research develops a framework that unveils how players feel present in Virtual Reality (VR) narrative games. In 2017, there were over 2.2 billion active video gamers, and VR is a future game platform because VR provides immersive gaming experiences where players experience presence. Slater and & Wilbur (1997, p. 14) define presence as the “sense of being in the virtual environment.” Previous research demonstrated that narrative could improve presence in VR games but left open the discussion on the causal relationship. Therefore, understanding how game narrative contributes to presence could further improve VR gaming experiences. This research reviews Wei, Bizzocchi & Calvert’s (2010) and Ryan’s (2015) frameworks that evaluate narrative games and presence in VR narratives respectively. Wei et al. and Ryan's frameworks are combined to propose The Augmented Framework which is then used to evaluate the self-produced VR narrative game, Caillte. The evaluation yields insights which improve the usage of The Augmented Framework for analysing other VR narrative games

PiCo: A Domain-Specific Language for Data Analytics Pipelines

In the world of Big Data analytics, there is a series of tools aiming at simplifying programming applications to be executed on clusters. Although each tool claims to provide better programming, data and execution models—for which only informal (and often confusing) semantics is generally provided—all share a common under- lying model, namely, the Dataflow model. Using this model as a starting point, it is possible to categorize and analyze almost all aspects about Big Data analytics tools from a high level perspective. This analysis can be considered as a first step toward a formal model to be exploited in the design of a (new) framework for Big Data analytics. By putting clear separations between all levels of abstraction (i.e., from the runtime to the user API), it is easier for a programmer or software designer to avoid mixing low level with high level aspects, as we are often used to see in state-of-the-art Big Data analytics frameworks. From the user-level perspective, we think that a clearer and simple semantics is preferable, together with a strong separation of concerns. For this reason, we use the Dataflow model as a starting point to build a programming environment with a simplified programming model implemented as a Domain-Specific Language, that is on top of a stack of layers that build a prototypical framework for Big Data analytics. The contribution of this thesis is twofold: first, we show that the proposed model is (at least) as general as existing batch and streaming frameworks (e.g., Spark, Flink, Storm, Google Dataflow), thus making it easier to understand high-level data-processing applications written in such frameworks. As result of this analysis, we provide a layered model that can represent tools and applications following the Dataflow paradigm and we show how the analyzed tools fit in each level. Second, we propose a programming environment based on such layered model in the form of a Domain-Specific Language (DSL) for processing data collections, called PiCo (Pipeline Composition). The main entity of this programming model is the Pipeline, basically a DAG-composition of processing elements. This model is intended to give the user an unique interface for both stream and batch processing, hiding completely data management and focusing only on operations, which are represented by Pipeline stages. Our DSL will be built on top of the FastFlow library, exploiting both shared and distributed parallelism, and implemented in C++11/14 with the aim of porting C++ into the Big Data world

That all may Justice Share: Sydney Catholics in the interwar years 1919-1929

While the years immediately following Australia’s participation in World War I have received much academic attention, the churches in the post-war era have been largely overlooked. Responding to Michael McKernan’s claim that Australian churches became less relevant to Australian society after the war, this thesis examines the response of the Catholic Church in Sydney to life after the war. Utilising a dual methodology of religious history and social history, this thesis analyses the role of the Church at this time by examining its works and activities, drawing on primary sources in Archdiocesan archives and the archives of specific Catholic lay groups. It focuses in particular on two rarely studied organisations, the Catholic Returned Soldiers and Sailors Association and the Knights of the Southern Cross. In the final analysis, this thesis finds that the Church in post-World War I Australia played a significant role in the lives of its own members, that Catholic organisations responded to particular social challenges of the period, and that there was a move towards both a more nationalist definition of and an international perspective in Australian Catholicism, culminating in the 29th International Eucharistic Congress of 1928. In studying the Catholic Church in Sydney in the decade following World War I, this thesis contributes towards a more nuanced understanding of the history of Australia in the 1920s and, more specifically, to the history of the Catholic Church in Australia, while challenging the claim that Australian churches became less relevant after the war