Learning by stochastic serializations

Complex structures are typical in machine learning. Tailoring learning algorithms for every structure requires an effort that may be saved by defining a generic learning procedure adaptive to any complex structure. In this paper, we propose to map any complex structure onto a generic form, called serialization, over which we can apply any sequence-based density estimator. We then show how to transfer the learned density back onto the space of original structures. To expose the learning procedure to the structural particularities of the original structures, we take care that the serializations reflect accurately the structures' properties. Enumerating all serializations is infeasible. We propose an effective way to sample representative serializations from the complete set of serializations which preserves the statistics of the complete set. Our method is competitive or better than state of the art learning algorithms that have been specifically designed for given structures. In addition, since the serialization involves sampling from a combinatorial process it provides considerable protection from overfitting, which we clearly demonstrate on a number of experiments.Comment: Submission to NeurIPS 201

An efficient task-based all-reduce for machine learning applications

All-Reduce is a collective-combine operation frequently utilised in synchronous parameter updates in parallel machine learning algorithms. The performance of this operation - and subsequently of the algorithm itself - is heavily dependent on its implementation, configuration and on the supporting hardware on which it is run. Given the pivotal role of all-reduce, a failure in any of these regards will significantly impact the resulting scientific output. In this research we explore the performance of alternative all-reduce algorithms in data-flow graphs and compare these to the commonly used reduce-broadcast approach. We present an architecture and interface for all-reduce in task-based frameworks, and a parallelization scheme for object-serialization and computation. We present a concrete, novel application of a butterfly all-reduce algorithm on the Apache Spark framework on a high-performance compute cluster, and demonstrate the effectiveness of the new butterfly algorithm with a logarithmic speed-up with respect to the vector length compared with the original reduce-broadcast method - a 9x speed-up is observed for vector lengths in the order of 108. This improvement is comprised of both algorithmic changes (65%) and parallel-processing optimization (35%). The effectiveness of the new butterfly all-reduce is demonstrated using real-world neural network applications with the Spark framework. For the model-update operation we observe significant speed-ups using the new butterfly algorithm compared with the original reduce-broadcast, for both smaller (Cifar and Mnist) and larger (ImageNet) datasets

Implementation of labcreator and the integration of cyberlab

With the development of the World Wide Web, online courses are becoming more and more popular in modern science education. CyberLab aims to solve an important issue in distance science education -- laboratory experiments in online courses. It is a toolkit that handles creation, exportation, and execution of virtual experiments (within web browsers). It consists of LabCreator and LabExecutor. With LabCreator, instructors can create virtual experiments and export them into intermediate files. Students can download those files from online course websites and execute them in LabExecutor on their own computers. The paper reports on the completion of two important tasks in the development of CyberLab: (1) the implementation of LabCreator and (2) a system allowing exportation of the experiment to intermediate web accessible format and the loading of the experiment into LabExecutor. The feasibility of the design and structure of CyberLab is proved by integrating the LabCreator and LabExecutor for the first time. The advantage of CyberLab is shown through a demonstration of the deployment of a virtual experiment

Recovering historical climate records using artificial neural networks in GPU

This article presents a parallel implementation of Artificial Neural Networks over Graphic Processing Units, and its application for recovering his-torical climate records from the Digi-Clima project. Several strategies are intro-duced to handle large volumes of historical pluviometer records, and the paral-lel deployment is described. The experimental evaluation demonstrates that the proposed approach is useful for recovering the climate information, achieving classification rates up to 76% for a set of real images from the Digi-Clima pro-ject. The parallel algorithm allows reducing the execution times, with an accel-eration factor of up to 2.15×

Semantic Representation of Physics Research Data

Improvements in web technologies and artificial intelligence enable novel, more data-driven research practices for scientists. However, scientific knowledge generated from data-intensive research practices is disseminated with unstructured formats, thus hindering the scholarly communication in various respects. The traditional document-based representation of scholarly information hampers the reusability of research contributions. To address this concern, we developed the Physics Ontology (PhySci) to represent physics-related scholarly data in a machine-interpretable format. PhySci facilitates knowledge exploration, comparison, and organization of such data by representing it as knowledge graphs. It establishes a unique conceptualization to increase the visibility and accessibility to the digital content of physics publications. We present the iterative design principles by outlining a methodology for its development and applying three different evaluation approaches: data-driven and criteria-based evaluation, as well as ontology testing