Using Rewrite Strategies for Efficient Functional Automatic Differentiation

Automatic Differentiation (AD) has become a dominant technique in ML. AD frameworks have first been implemented for imperative languages using tapes. Meanwhile, functional implementations of AD have been developed, often based on dual numbers, which are close to the formal specification of differentiation and hence easier to prove correct. But these papers have focussed on correctness not efficiency. Recently, it was shown how an approach using dual numbers could be made efficient through the right optimizations. Optimizations are highly dependent on order, as one optimization can enable another. It can therefore be useful to have fine-grained control over the scheduling of optimizations. One method expresses compiler optimizations as rewrite rules, whose application can be combined and controlled using strategy languages. Previous work describes the use of term rewriting and strategies to generate high-performance code in a compiler for a functional language. In this work, we implement dual numbers AD in a functional array programming language using rewrite rules and strategy combinators for optimization. We aim to combine the elegance of differentiation using dual numbers with a succinct expression of the optimization schedule using a strategy language. We give preliminary evidence suggesting the viability of the approach on a micro-benchmark.Comment: to be published in FTfJP 202

Achieving High-Performance the Functional Way: A Functional Pearl on Expressing High-Performance Optimizations as Rewrite Strategies

Optimizing programs to run efficiently on modern parallel hardware is hard but crucial for many applications. The predominantly used imperative languages - like C or OpenCL - force the programmer to intertwine the code describing functionality and optimizations. This results in a portability nightmare that is particularly problematic given the accelerating trend towards specialized hardware devices to further increase efficiency. Many emerging DSLs used in performance demanding domains such as deep learning or high-performance image processing attempt to simplify or even fully automate the optimization process. Using a high-level - often functional - language, programmers focus on describing functionality in a declarative way. In some systems such as Halide or TVM, a separate schedule specifies how the program should be optimized. Unfortunately, these schedules are not written in well-defined programming languages. Instead, they are implemented as a set of ad-hoc predefined APIs that the compiler writers have exposed. In this functional pearl, we show how to employ functional programming techniques to solve this challenge with elegance. We present two functional languages that work together - each addressing a separate concern. RISE is a functional language for expressing computations using well known functional data-parallel patterns. ELEVATE is a functional language for describing optimization strategies. A high-level RISE program is transformed into a low-level form using optimization strategies written in ELEVATE . From the rewritten low-level program high-performance parallel code is automatically generated. In contrast to existing high-performance domain-specific systems with scheduling APIs, in our approach programmers are not restricted to a set of built-in operations and optimizations but freely define their own computational patterns in RISE and optimization strategies in ELEVATE in a composable and reusable way. We show how our holistic functional approach achieves competitive performance with the state-of-the-art imperative systems Halide and TVM

Signs and symbols represented in Germanic, particularly Scandinavian, iconography between the Migration Period and the end of the Viking Age.

This research focuses on the analysis of pictorial images from objects of Germanic/early Scandinavian cultures in order to discriminate elements which are only applied for decorative reasons from those which also express symbolic values. At the outset I introduced an interpretation of the terms 'signs' and 'symbols' in order to inform the reader to which extent these terms were applied. It appeared that techniques, such as the geographical and chronological classification, traditionally used by archaeologists and historians to analyse objects/artefacts, were not always sufficient enough to gain all information which images may offer. Sometimes it seems to be difficult to determine if certain images shown on objects from early mediaeval, central and northern European origin, were created as a space-filling decoration only or if they held additional, probably symbolic, information as well. I have investigated aspects from visual perception, as applied within the domains of psychology, visual art and design. The methods, as they are introduced in this thesis, can be used as a identification scheme, applied on objects of great diversity. I have applied them on guldgubber (little embossed gold foils), bracteates and Gotland picture-stones. Objects of great historical and geographic difference, as well as physical diversity, such as size and material were chosen purposely to create an overview of the symbolic expressions in Germanic/early Scandinavian artefacts and to test the extent of the identification method derived from the domain of visual perception. A detailed analysis of sixty four guldgubber, found in Lundeborg, near Gudme, on the island of Fyn (Denmark), is placed in an appendix. An analysis of these objects appeared to be particularly interesting because it is not known what the purpose of guldgubber were to the present day

Signs and symbols represented in Germanic, particularly Scandinavian, iconography between the Migration Period and the end of the Viking Age.

This research focuses on the analysis of pictorial images from objects of Germanic/early Scandinavian cultures in order to discriminate elements which are only applied for decorative reasons from those which also express symbolic values. At the outset I introduced an interpretation of the terms 'signs' and 'symbols' in order to inform the reader to which extent these terms were applied. It appeared that techniques, such as the geographical and chronological classification, traditionally used by archaeologists and historians to analyse objects/artefacts, were not always sufficient enough to gain all information which images may offer. Sometimes it seems to be difficult to determine if certain images shown on objects from early mediaeval, central and northern European origin, were created as a space-filling decoration only or if they held additional, probably symbolic, information as well. I have investigated aspects from visual perception, as applied within the domains of psychology, visual art and design. The methods, as they are introduced in this thesis, can be used as a identification scheme, applied on objects of great diversity. I have applied them on guldgubber (little embossed gold foils), bracteates and Gotland picture-stones. Objects of great historical and geographic difference, as well as physical diversity, such as size and material were chosen purposely to create an overview of the symbolic expressions in Germanic/early Scandinavian artefacts and to test the extent of the identification method derived from the domain of visual perception. A detailed analysis of sixty four guldgubber, found in Lundeborg, near Gudme, on the island of Fyn (Denmark), is placed in an appendix. An analysis of these objects appeared to be particularly interesting because it is not known what the purpose of guldgubber were to the present day

Celebration of Learning 2014: Full Program

Full program of the 2014 Celebration of Learning at Augustana College

National Identity in Northern and Eastern European Heavy Metal

In the last thirty years the genre of heavy metal has seen an increasing number of bands utilise aspects of their national or regional identity to inspire their work. This identity portrayal has been manifested in many forms — visual, textual and musical, and often with combinations of more than one. In the context of the work of Benedict Anderson, Eric Hobsbawm and Terence Ranger, and drawing comparison with the Romantic Nationalism movement of the 18th and 19th centuries throughout, this thesis examines the field and seeks to establish that writing on the subject has thus far failed in a number of key areas. The terms Viking metal, battle metal and Pagan metal form the core of current genre classification. I demonstrate how these are both inaccurate and insufficient, as they not only ignore the considerable breadth of work being produced, but also focus too heavily on bands from the Nordic region. As a result, the equally significant role that both the U.K. and Eastern Europe have played in the genre’s development has not been acknowledged. Subsequently, there is a need for transnational identities that are being both presented and received to be recognised or challenged where appropriate. Finally, the majority of writers to date have centered their attention on either textual content or matters of a (sometimes sensationalist) cultural or ethnographic nature. I seek to establish that the deficiency in academic musical analysis on the work being produced does the musicians a disservice. Tracing source material throughout, the thesis utilises a case study on the English band Oakenshield that presents a collage of all of the themes contained, and encapsulates the type of transnational identity being portrayed. The conflict between intention and reception in genre classification as discussed by writers such as Jeffrey Kallberg and Heather Dubrow is therefore demonstrated

A domain-extensible compiler with controllable automation of optimisations

In high performance domains like image processing, physics simulation or machine learning, program performance is critical. Programmers called performance engineers are responsible for the challenging task of optimising programs. Two major challenges prevent modern compilers targeting heterogeneous architectures from reliably automating optimisation. First, domain specific compilers such as Halide for image processing and TVM for machine learning are difficult to extend with the new optimisations required by new algorithms and hardware. Second, automatic optimisation is often unable to achieve the required performance, and performance engineers often fall back to painstaking manual optimisation. This thesis shows the potential of the Shine compiler to achieve domain-extensibility, controllable automation, and generate high performance code. Domain-extensibility facilitates adapting compilers to new algorithms and hardware. Controllable automation enables performance engineers to gradually take control of the optimisation process. The first research contribution is to add 3 code generation features to Shine, namely: synchronisation barrier insertion, kernel execution, and storage folding. Adding these features requires making novel design choices in terms of compiler extensibility and controllability. The rest of this thesis builds on these features to generate code with competitive runtime compared to established domain-specific compilers. The second research contribution is to demonstrate how extensibility and controllability are exploited to optimise a standard image processing pipeline for corner detection. Shine achieves 6 well-known image processing optimisations, 2 of them not being supported by Halide. Our results on 4 ARM multi-core CPUs show that the code generated by Shine for corner detection runs up to 1.4× faster than the Halide code. However, we observe that controlling rewriting is tedious, motivating the need for more automation. The final research contribution is to introduce sketch-guided equality saturation, a semiautomated technique that allows performance engineers to guide program rewriting by specifying rewrite goals as sketches: program patterns that leave details unspecified. We evaluate this approach by applying 7 realistic optimisations of matrix multiplication. Without guidance, the compiler fails to apply the 5 most complex optimisations even given an hour and 60GB of RAM. With the guidance of at most 3 sketch guides, each 10 times smaller than the complete program, the compiler applies the optimisations in seconds using less than 1GB

The Juggler of Notre Dame and the Medievalizing of Modernity, Volume 6: War and Peace, Sex and Violence

This ambitious and vivid study in six volumes explores the journey of a single, electrifying story, from its first incarnation in a medieval French poem through its prolific rebirth in the nineteenth and twentieth centuries. The Juggler of Notre Dame tells how an entertainer abandons the world to join a monastery, but is suspected of blasphemy after dancing his devotion before a statue of the Madonna in the crypt; he is saved when the statue, delighted by his skill, miraculously comes to life. Jan Ziolkowski tracks the poem from its medieval roots to its rediscovery in late nineteenth-century Paris, before its translation into English in Britain and the United States. The visual influence of the tale on Gothic revivalism and vice versa in America is carefully documented with lavish and inventive illustrations, and Ziolkowski concludes with an examination of the explosion of interest in The Juggler of Notre Dame in the twentieth century and its place in mass culture today. The Juggler of Notre Dame and the Medievalizing of Modernity is a rich case study for the reception of the Middle Ages in modernity. Spanning centuries and continents, the medieval period is understood through the lens of its (post)modern reception in Europe and America. Profound connections between the verbal and the visual are illustrated by a rich trove of images, including book illustrations, stained glass, postage stamps, architecture, and Christmas cards. Presented with great clarity and simplicity, Ziolkowski\u27s work is accessible to the general reader, while its many new discoveries will be valuable to academics in such fields and disciplines as medieval studies, medievalism, philology, literary history, art history, folklore, performance studies, and reception studies.https://ecommons.udayton.edu/ml_juggler/1011/thumbnail.jp

The Nordic Ingredient. European Nationalisms and Norwegian Music since 1905

publishedVersio