Automatic differentiation in machine learning: a survey

Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD), also called algorithmic differentiation or simply "autodiff", is a family of techniques similar to but more general than backpropagation for efficiently and accurately evaluating derivatives of numeric functions expressed as computer programs. AD is a small but established field with applications in areas including computational fluid dynamics, atmospheric sciences, and engineering design optimization. Until very recently, the fields of machine learning and AD have largely been unaware of each other and, in some cases, have independently discovered each other's results. Despite its relevance, general-purpose AD has been missing from the machine learning toolbox, a situation slowly changing with its ongoing adoption under the names "dynamic computational graphs" and "differentiable programming". We survey the intersection of AD and machine learning, cover applications where AD has direct relevance, and address the main implementation techniques. By precisely defining the main differentiation techniques and their interrelationships, we aim to bring clarity to the usage of the terms "autodiff", "automatic differentiation", and "symbolic differentiation" as these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure

SensorCloud: Towards the Interdisciplinary Development of a Trustworthy Platform for Globally Interconnected Sensors and Actuators

Although Cloud Computing promises to lower IT costs and increase users' productivity in everyday life, the unattractive aspect of this new technology is that the user no longer owns all the devices which process personal data. To lower scepticism, the project SensorCloud investigates techniques to understand and compensate these adoption barriers in a scenario consisting of cloud applications that utilize sensors and actuators placed in private places. This work provides an interdisciplinary overview of the social and technical core research challenges for the trustworthy integration of sensor and actuator devices with the Cloud Computing paradigm. Most importantly, these challenges include i) ease of development, ii) security and privacy, and iii) social dimensions of a cloud-based system which integrates into private life. When these challenges are tackled in the development of future cloud systems, the attractiveness of new use cases in a sensor-enabled world will considerably be increased for users who currently do not trust the Cloud.Comment: 14 pages, 3 figures, published as technical report of the Department of Computer Science of RWTH Aachen Universit

The 1958 Pekeris-Accad-WEIZAC Ground-Breaking Collaboration that Computed Ground States of Two-Electron Atoms (and its 2010 Redux)

In order to appreciate how well off we mathematicians and scientists are today, with extremely fast hardware and lots and lots of memory, as well as with powerful software, both for numeric and symbolic computation, it may be a good idea to go back to the early days of electronic computers and compare how things went then. We have chosen, as a case study, a problem that was considered a huge challenge at the time. Namely, we looked at C.L. Pekeris's seminal 1958 work on the ground state energies of two-electron atoms. We went through all the computations ab initio with today's software and hardware, with a special emphasis on the symbolic computations which in 1958 had to be made by hand, and which nowadays can be automated and generalized.Comment: 8 pages, 2 photos, final version as it appeared in the journa

Virtual reality in the service of user participation in architecture

The issue of user participation in the processes of building and urban design is enjoying renewed attention following its relative neglect over the last 20 years due, in large measure, to significant advances in emerging information technologies, particularly multimedia, virtual reality and internet technologies. This paper re-established the theoretical framework for participatory design evolved in the late sixties and early seventies as part of the movement towards a more explicit design methodology and attempts an explanation of why the concept failed to gain commitment from the architectural and urban design professionals. The paper then gives an account of two significant developments in the evolution of the application of information technologies with which the authors have been engaged. These are: i. a responsive and interactive interface to wholly immersive and realistic virtual reality representations of proposed buildings and urban neighbourhoods. ii. an intuitive and platform-independent VR modelling environment allowing collaborative evolution of the scheme from within the virtual world. The impact of these IT developments is demonstrated in the context of the design of a leisure facility for a community of users with physical impairment

PList-based Divide and Conquer Parallel Programming

This paper details an extension of a Java parallel programming framework – JPLF. The JPLF framework is a programming framework that helps programmers build parallel programs using existing building blocks. The framework is based on {em PowerLists} and PList Theories and it naturally supports multi-way Divide and Conquer. By using this framework, the programmer is exempted from dealing with all the complexities of writing parallel programs from scratch. This extension to the JPLF framework adds PLists support to the framework and so, it enlarges the applicability of the framework to a larger set of parallel solvable problems. Using this extension, we may apply more flexible data division strategies. In addition, the length of the input lists no longer has to be a power of two – as required by the PowerLists theory. In this paper we unveil new applications that emphasize the new class of computations that can be executed within the JPLF framework. We also give a detailed description of the data structures and functions involved in the PLists extension of the JPLF, and extended performance experiments are described and analyzed