Probabilistic Graphical Models on Multi-Core CPUs using Java 8

In this paper, we discuss software design issues related to the development of parallel computational intelligence algorithms on multi-core CPUs, using the new Java 8 functional programming features. In particular, we focus on probabilistic graphical models (PGMs) and present the parallelisation of a collection of algorithms that deal with inference and learning of PGMs from data. Namely, maximum likelihood estimation, importance sampling, and greedy search for solving combinatorial optimisation problems. Through these concrete examples, we tackle the problem of defining efficient data structures for PGMs and parallel processing of same-size batches of data sets using Java 8 features. We also provide straightforward techniques to code parallel algorithms that seamlessly exploit multi-core processors. The experimental analysis, carried out using our open source AMIDST (Analysis of MassIve Data STreams) Java toolbox, shows the merits of the proposed solutions.Comment: Pre-print version of the paper presented in the special issue on Computational Intelligence Software at IEEE Computational Intelligence Magazine journa

Solving optimisation problems in metal forming using Finite Element simulation and metamodelling techniques

During the last decades, Finite Element (FEM) simulations\ud of metal forming processes have become important\ud tools for designing feasible production processes. In more\ud recent years, several authors recognised the potential of\ud coupling FEM simulations to mathematical optimisation\ud algorithms to design optimal metal forming processes instead\ud of only feasible ones.\ud Within the current project, an optimisation strategy is being\ud developed, which is capable of optimising metal forming\ud processes in general using time consuming nonlinear\ud FEM simulations. The expression “optimisation strategy”\ud is used to emphasise that the focus is not solely on solving\ud optimisation problems by an optimisation algorithm, but\ud the way these optimisation problems in metal forming are\ud modelled is also investigated. This modelling comprises\ud the quantification of objective functions and constraints\ud and the selection of design variables.\ud This paper, however, is concerned with the choice for\ud and the implementation of an optimisation algorithm for\ud solving optimisation problems in metal forming. Several\ud groups of optimisation algorithms can be encountered in\ud metal forming literature: classical iterative, genetic and\ud approximate optimisation algorithms are already applied\ud in the field. We propose a metamodel based optimisation\ud algorithm belonging to the latter group, since approximate\ud algorithms are relatively efficient in case of time consuming\ud function evaluations such as the nonlinear FEM calculations\ud we are considering. Additionally, approximate optimisation\ud algorithms strive for a global optimum and do\ud not need sensitivities, which are quite difficult to obtain\ud for FEM simulations. A final advantage of approximate\ud optimisation algorithms is the process knowledge, which\ud can be gained by visualising metamodels.\ud In this paper, we propose a sequential approximate optimisation\ud algorithm, which incorporates both Response\ud Surface Methodology (RSM) and Design and Analysis\ud of Computer Experiments (DACE) metamodelling techniques.\ud RSM is based on fitting lower order polynomials\ud by least squares regression, whereas DACE uses Kriging\ud interpolation functions as metamodels. Most authors in\ud the field of metal forming use RSM, although this metamodelling\ud technique was originally developed for physical\ud experiments that are known to have a stochastic na-\ud ¤Faculty of Engineering Technology (Applied Mechanics group),\ud University of Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands,\ud email: [email protected]\ud ture due to measurement noise present. This measurement\ud noise is absent in case of deterministic computer experiments\ud such as FEM simulations. Hence, an interpolation\ud model fitted by DACE is thought to be more applicable in\ud combination with metal forming simulations. Nevertheless,\ud the proposed algorithm utilises both RSM and DACE\ud metamodelling techniques.\ud As a Design Of Experiments (DOE) strategy, a combination\ud of a maximin spacefilling Latin Hypercubes Design\ud and a full factorial design was implemented, which takes\ud into account explicit constraints. Additionally, the algorithm\ud incorporates cross validation as a metamodel validation\ud technique and uses a Sequential Quadratic Programming\ud algorithm for metamodel optimisation. To overcome\ud the problem of ending up in a local optimum, the\ud SQP algorithm is initialised from every DOE point, which\ud is very time efficient since evaluating the metamodels can\ud be done within a fraction of a second. The proposed algorithm\ud allows for sequential improvement of the metamodels\ud to obtain a more accurate optimum.\ud As an example case, the optimisation algorithm was applied\ud to obtain the optimised internal pressure and axial\ud feeding load paths to minimise wall thickness variations\ud in a simple hydroformed product. The results are satisfactory,\ud which shows the good applicability of metamodelling\ud techniques to optimise metal forming processes using\ud time consuming FEM simulations

A metamodel based optimisation algorithm for metal forming processes

Cost saving and product improvement have always been important goals in the metal\ud forming industry. To achieve these goals, metal forming processes need to be optimised. During\ud the last decades, simulation software based on the Finite Element Method (FEM) has significantly\ud contributed to designing feasible processes more easily. More recently, the possibility of\ud coupling FEM to mathematical optimisation algorithms is offering a very promising opportunity\ud to design optimal metal forming processes instead of only feasible ones. However, which\ud optimisation algorithm to use is still not clear.\ud In this paper, an optimisation algorithm based on metamodelling techniques is proposed\ud for optimising metal forming processes. The algorithm incorporates nonlinear FEM simulations\ud which can be very time consuming to execute. As an illustration of its capabilities, the\ud proposed algorithm is applied to optimise the internal pressure and axial feeding load paths\ud of a hydroforming process. The product formed by the optimised process outperforms products\ud produced by other, arbitrarily selected load paths. These results indicate the high potential of\ud the proposed algorithm for optimising metal forming processes using time consuming FEM\ud simulations

Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis

The widespread use of multi-sensor technology and the emergence of big datasets has highlighted the limitations of standard flat-view matrix models and the necessity to move towards more versatile data analysis tools. We show that higher-order tensors (i.e., multiway arrays) enable such a fundamental paradigm shift towards models that are essentially polynomial and whose uniqueness, unlike the matrix methods, is guaranteed under verymild and natural conditions. Benefiting fromthe power ofmultilinear algebra as theirmathematical backbone, data analysis techniques using tensor decompositions are shown to have great flexibility in the choice of constraints that match data properties, and to find more general latent components in the data than matrix-based methods. A comprehensive introduction to tensor decompositions is provided from a signal processing perspective, starting from the algebraic foundations, via basic Canonical Polyadic and Tucker models, through to advanced cause-effect and multi-view data analysis schemes. We show that tensor decompositions enable natural generalizations of some commonly used signal processing paradigms, such as canonical correlation and subspace techniques, signal separation, linear regression, feature extraction and classification. We also cover computational aspects, and point out how ideas from compressed sensing and scientific computing may be used for addressing the otherwise unmanageable storage and manipulation problems associated with big datasets. The concepts are supported by illustrative real world case studies illuminating the benefits of the tensor framework, as efficient and promising tools for modern signal processing, data analysis and machine learning applications; these benefits also extend to vector/matrix data through tensorization. Keywords: ICA, NMF, CPD, Tucker decomposition, HOSVD, tensor networks, Tensor Train

Fast Parallel Operations on Search Trees

Using (a,b)-trees as an example, we show how to perform a parallel split with logarithmic latency and parallel join, bulk updates, intersection, union (or merge), and (symmetric) set difference with logarithmic latency and with information theoretically optimal work. We present both asymptotically optimal solutions and simplified versions that perform well in practice - they are several times faster than previous implementations

Parallel Weighted Random Sampling

Data structures for efficient sampling from a set of weighted items are an important building block of many applications. However, few parallel solutions are known. We close many of these gaps both for shared-memory and distributed-memory machines. We give efficient, fast, and practicable algorithms for sampling single items, k items with/without replacement, permutations, subsets, and reservoirs. We also give improved sequential algorithms for alias table construction and for sampling with replacement. Experiments on shared-memory parallel machines with up to 158 threads show near linear speedups both for construction and queries

OpenACC Based GPU Parallelization of Plane Sweep Algorithm for Geometric Intersection

Line segment intersection is one of the elementary operations in computational geometry. Complex problems in Geographic Information Systems (GIS) like finding map overlays or spatial joins using polygonal data require solving segment intersections. Plane sweep paradigm is used for finding geometric intersection in an efficient manner. However, it is difficult to parallelize due to its in-order processing of spatial events. We present a new fine-grained parallel algorithm for geometric intersection and its CPU and GPU implementation using OpenMP and OpenACC. To the best of our knowledge, this is the first work demonstrating an effective parallelization of plane sweep on GPUs. We chose compiler directive based approach for implementation because of its simplicity to parallelize sequential code. Using Nvidia Tesla P100 GPU, our implementation achieves around 40X speedup for line segment intersection problem on 40K and 80K data sets compared to sequential CGAL library

High-Quality Shared-Memory Graph Partitioning

Partitioning graphs into blocks of roughly equal size such that few edges run between blocks is a frequently needed operation in processing graphs. Recently, size, variety, and structural complexity of these networks has grown dramatically. Unfortunately, previous approaches to parallel graph partitioning have problems in this context since they often show a negative trade-off between speed and quality. We present an approach to multi-level shared-memory parallel graph partitioning that guarantees balanced solutions, shows high speed-ups for a variety of large graphs and yields very good quality independently of the number of cores used. For example, on 31 cores, our algorithm partitions our largest test instance into 16 blocks cutting less than half the number of edges than our main competitor when both algorithms are given the same amount of time. Important ingredients include parallel label propagation for both coarsening and improvement, parallel initial partitioning, a simple yet effective approach to parallel localized local search, and fast locality preserving hash tables