Corpus Conversion Service: A Machine Learning Platform to Ingest Documents at Scale

Over the past few decades, the amount of scientific articles and technical literature has increased exponentially in size. Consequently, there is a great need for systems that can ingest these documents at scale and make the contained knowledge discoverable. Unfortunately, both the format of these documents (e.g. the PDF format or bitmap images) as well as the presentation of the data (e.g. complex tables) make the extraction of qualitative and quantitive data extremely challenging. In this paper, we present a modular, cloud-based platform to ingest documents at scale. This platform, called the Corpus Conversion Service (CCS), implements a pipeline which allows users to parse and annotate documents (i.e. collect ground-truth), train machine-learning classification algorithms and ultimately convert any type of PDF or bitmap-documents to a structured content representation format. We will show that each of the modules is scalable due to an asynchronous microservice architecture and can therefore handle massive amounts of documents. Furthermore, we will show that our capability to gather ground-truth is accelerated by machine-learning algorithms by at least one order of magnitude. This allows us to both gather large amounts of ground-truth in very little time and obtain very good precision/recall metrics in the range of 99\% with regard to content conversion to structured output. The CCS platform is currently deployed on IBM internal infrastructure and serving more than 250 active users for knowledge-engineering project engagements.Comment: Accepted paper at KDD 2018 conferenc

A fast and scalable low dimensional solver for charged particle dynamics in large particle accelerators

Particle accelerators are invaluable tools for research in the basic and applied sciences, in fields such as materials science, chemistry, the biosciences, particle physics, nuclear physics and medicine. The design, commissioning, and operation of accelerator facilities is a non-trivial task, due to the large number of control parameters and the complex interplay of several conflicting design goals. We propose to tackle this problem by means of multi-objective optimization algorithms which also facilitate massively parallel deployment. In order to compute solutions in a meaningful time frame, that can even admit online optimization, we require a fast and scalable software framework. In this paper, we focus on the key and most heavily used component of the optimization framework, the forward solver. We demonstrate that our parallel methods achieve a strong and weak scalability improvement of at least two orders of magnitude in today's actual particle beam configurations, reducing total time to solution by a substantial factor. Our target platform is the Blue Gene/P (Blue Gene/P is a trademark of the International Business Machines Corporation in the United States, other countries, or both) supercomputer. The space-charge model used in the forward solver relies significantly on collective communication. Thus, the dedicated TREE network of the platform serves as an ideal vehicle for our purposes. We demonstrate excellent strong and weak scalability of our software which allows us to perform thousands of forward solves in a matter of minutes, thus already allowing close to online optimization capabilit