On Improving Distributed Pregel-like Graph Processing Systems

The considerable interest in distributed systems that can execute algorithms to process large graphs has led to the creation of many graph processing systems. However, existing systems suffer from two major issues: (1) poor performance due to frequent global synchronization barriers and limited scalability; and (2) lack of support for graph algorithms that require serializability, the guarantee that parallel executions of an algorithm produce the same results as some serial execution of that algorithm. Many graph processing systems use the bulk synchronous parallel (BSP) model, which allows graph algorithms to be easily implemented and reasoned about. However, BSP suffers from poor performance due to stale messages and frequent global synchronization barriers. While asynchronous models have been proposed to alleviate these overheads, existing systems that implement such models have limited scalability or retain frequent global barriers and do not always support graph mutations or algorithms with multiple computation phases. We propose barrierless asynchronous parallel (BAP), a new computation model that overcomes the limitations of existing asynchronous models by reducing both message staleness and global synchronization while retaining support for graph mutations and algorithms with multiple computation phases. We present GiraphUC, which implements our BAP model in the open source distributed graph processing system Giraph, and evaluate it at scale to demonstrate that BAP provides efficient and transparent asynchronous execution of algorithms that are programmed synchronously. Secondly, very few systems provide serializability, despite the fact that many graph algorithms require it for accuracy, correctness, or termination. To address this deficiency, we provide a complete solution that can be implemented on top of existing graph processing systems to provide serializability. Our solution formalizes the notion of serializability and the conditions under which it can be provided for graph processing systems. We propose a partition-based synchronization technique that enforces these conditions efficiently to provide serializability. We implement this technique into Giraph and GiraphUC to demonstrate that it is configurable, transparent to algorithm developers, and more performant than existing techniques.4 month

Optimizing radial basis functions by D.C. programming and its use in direct search for global derivative-free optimization

In this paper we address the global optimization of functions subject to bound and linear constraints without using derivatives of the objective function. We investigate the use of derivative-free models based on radial basis functions (RBFs) in the search step of direct-search methods of directional type. We also study the application of algorithms based on difference of convex (d.c.) functions programming to solve the resulting subproblems which consist of the minimization of the RBF models subject to simple bounds on the variables. Extensive numerical results are reported with a test set of bound and linearly constrained problems

Communication-Efficient Federated Learning for LEO Satellite Networks Integrated with HAPs Using Hybrid NOMA-OFDM

Space AI has become increasingly important and sometimes even necessary for government, businesses, and society. An active research topic under this mission is integrating federated learning (FL) with satellite communications (SatCom) so that numerous low Earth orbit (LEO) satellites can collaboratively train a machine learning model. However, the special communication environment of SatCom leads to a very slow FL training process up to days and weeks. This paper proposes NomaFedHAP, a novel FL-SatCom approach tailored to LEO satellites, that (1) utilizes high-altitude platforms (HAPs) as distributed parameter servers (PS) to enhance satellite visibility, and (2) introduces non-orthogonal multiple access (NOMA) into LEO to enable fast and bandwidth-efficient model transmissions. In addition, NomaFedHAP includes (3) a new communication topology that exploits HAPs to bridge satellites among different orbits to mitigate the Doppler shift, and (4) a new FL model aggregation scheme that optimally balances models between different orbits and shells. Moreover, we (5) derive a closed-form expression of the outage probability for satellites in near and far shells, as well as for the entire system. Our extensive simulations have validated the mathematical analysis and demonstrated the superior performance of NomaFedHAP in achieving fast and efficient FL model convergence with high accuracy as compared to the state-of-the-art

Parallel strategies for Direct Multisearch

Publisher Copyright: © 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.Direct multisearch (DMS) is a derivative-free optimization class of algorithms, suited for computing approximations to the complete Pareto front of a given multiobjective optimization problem. In DMS class, constraints are addressed with an extreme barrier approach, only evaluating feasible points. It has a well-supported convergence analysis and simple implementations present a good numerical performance, both in academic test sets and in real applications. Recently, this numerical performance was improved with the definition of a search step based on the minimization of quadratic polynomial models, corresponding to the algorithm BoostDMS. In this work, we propose and numerically evaluate strategies to improve the performance of BoostDMS, mainly through parallelization applied to the search and to the poll steps. The final parallelized version not only considerably decreases the computational time required for solving a multiobjective optimization problem, but also increases the quality of the computed approximation to the Pareto front. Extensive numerical results will be reported in an academic test set and in a chemical engineering application.preprintpublishe

Characterization and optimization study of Poly(N-isopropylacrylamide) (P(NIPAM)) and β-cyclodextrin (βCD) system for Controlled Release Drug Delivery Systems with NMR and MRI techniques

This project is a direct continuation from my bachelor project that was based off Wisniewska,M (2017), where she investigated the changes in Poly(N-isopropylacrylamide) P(NIPAM) at the Volume Phase Transition (VPT) . The goal of the project was to expand the portfolio and characterization of P(NIPAM), particularly on how β-cyclodextrin (βCD) reacts as it goes through VPT both inside the hydrogel and outside the hydrogel. By using Magnetic Resonance Imaging (MRI), it allows us to monitor the changes of P(NIPAM) as it goes through the Lower Critical Solution Temperature (LCST), the Volume Phase Transition Temperature (VPTT), and 40 °C. Multi-Slice Multi-Echo (MSME) imaging was used to monitor the physical changes of the hydrogel with increasing temperature for comparison to spectral data using Magnetic Resonance Spectroscopy (MRS). Combining MRI with MRS allows for us to combine the macroscopic and microscopic picture of the βCD activity inside and outside the hydrogel. The changes of βCD in the polymer interior and exterior was monitored using STimulated Echo Acquisition Mode (STEAM), and the diffusion of βCD and water inside the hydrogel was recorded using MEshcher-GArwood Point RESolved Spectroscopy (MEGA-PRESS). The release of the βCD inside the hydrogel network was minimal until reaching VPTT, where the change in concentration sharply decreased due to the phase transition. As for the βCD change outside the hydrogel, the change was adherent to the shrinkage kinetics of P(NIPAM), showing delays in release due to restricted diffusion caused by the shrinkage of the P(NIPAM) after VPTT. For the self-diffusion data of the βCD, it was found that the βCD signals were loss using MEGA-PRESS after VPTT as a result of diffusion. The water self-diffusion values were also found to be decreasing consistently after a delay from 40°C.Master's Thesis in ChemistryKJEM399MAMN-KJE

Enabling the “Easy Button” for Broad, Parallel Optimization of Functions Evaluated by Simulation

Java Optimization by Simulation (JOBS) is presented: an open-source, object-oriented Java library designed to enable the study, research, and use of optimization for models evaluated by simulation. JOBS includes several novel design features that make it easy for a simulation modeler, without extensive expertise in optimization or parallel computation, to define an optimization model with deterministic and/or stochastic constraints, choose one or more metaheuristics to solve it and run, using massively parallel function evaluation to reduce wall-clock times. JOBS is supported by a new language independent, application programming interface (API) for remote simulation model evaluation and a serverless computing environment to provide massively parallel function evaluation, on demand. Dynamic loop scheduling methods are evaluated in the serverless environment with the opportunity for significant resource contention for master node computing power and network bandwidth. JOBS implements several population-based and single-solution improvement metaheuristics (solvers) for real, discrete, and mixed problems. The object-oriented design is extendible with classes that drastically reduce the amount of code required to implement a new solver and encourage re-use of solvers as building blocks for creating new multi-stage solvers or memetic algorithms

Modèles quadratiques et décomposition parallèle pour l’optimisation sans dérivées

RÉSUMÉ: L’optimisation sans dérivées (DFO) et l’optimisation de boites noires (BBO) sont deux disciplines qui traitent des problèmes dont la formulation analytique est inaccessible partiellement ou totalement et qui résultent souvent de simulations informatiques. Les algorithmes DFO et BBO s’appliquent typiquement à des problèmes de petite dimension. Parmi ces méthodes, l’algorithme de recherche directe par treillis adaptatifs (MADS) est une méthode itérative qui se base sur une discrétisation de l’espace et des directions de recherche pour sélectionner et évaluer des points de l’espace. Cette thèse explore deux extensions de MADS qui permettent d’améliorer les résultats de la méthode ainsi que de s’attaquer à des problèmes de plus grande taille. Dans la première extension, MADS utilise des modèles dans l’étape de recherche menant à la création d’une série de sous-problèmes quadratiques avec contraintes quadratiques. Deux méthodes d’optimisation non linéaire classiques sont décrites : la fonction de pénalité exacte en norme `1 et le Lagrangien augmenté. De plus, une nouvelle méthode nommée le Lagrangien augmenté en norme `1 combine les points forts des deux algorithmes précédents. Cette dernière méthode est bien adaptée pour les problèmes quadratiques avec contraintes quadratiques vu qu’elle se base sur un terme de pénalité en norme `1 qui permet de traiter un problème quadratique par morceaux au lieu d’un problème quartique si le Lagrangien augmenté standard est utilisé. La nouvelle méthode du Lagrangien augmenté en norme `1 est décrite pour les problèmes non linéaires avec des contraintes d’égalités. Une analyse conduite sur l’itération interne de l’algorithme prouve que la convergence vers un point stationnaire se fait avec une vitesse surperlinéaire en deux étapes. De plus, l’analyse de l’itération externe de la méthode établit que l’algorithme converge globalement et que le paramètre de pénalité est borné. Dans la seconde extension, l’algorithme de décomposition parallèle de l’espace de la recherche directe par treillis adaptatifs (PSD-MADS), qui est une méthode parallèle asynchrone pour les problèmes de boites noires de grande taille, utilise une stratégie de sélection aléatoire des variables pour la construction des sous-problèmes. Plusieurs stratégies sont proposées pour sélectionner les variables les plus influentes du problème et explorer l’espace des solutions de manière plus efficace. Ces stratégies se basent sur des outils statistiques pour évaluer l’influence des variables sur les différentes sorties et sur la méthode de classification k-mean pour grouper les variables avec plus ou moins le même niveau d’influence. De plus, une méthode hybride qui combine cette nouvelle stratégie avec l’approche aléatoire de sélection de variables est présentée. Les nouvelles méthodes améliorent les résultats de PSD-MADS et les tests numériques sont conduits sur des problèmes de taille allant jusqu’à 4000 variables.----------ABSTRACT: Derivative-free optimization (DFO) and blackbox optimization (BBO) are two fields studying problems for which the analytical formulation is partially or completely inaccessible, and which often result from computer simulations. Algorithms in DFO and BBO typically target problems with small dimension. One of these methods is the mesh adaptive direct search algorithm (MADS) which is an iterative method relying on a space discretization and search directions to select and assess new candidates. This thesis explores two extensions of the MADS algorithm that allow to improve its results and take on problems with a larger dimension. In the first extension, MADS uses models in the search step which generates a sequence of quadratic subproblems with quadratic constraints. Two classic nonlinear optimization methods are described: the `1-exact penalty function and the augmented Lagrangian. In addition, a new method, called the `1 augmented Lagrangian, combines the strengths of both previous methods. This new approach is well suited for quadratically constrained quadratic problems (QCQP) since the `1 penalty term allows the method to optimize a piecewise quadratic problem instead of a quartic one when using the standard augmented Lagrangien. The new `1 augmented Lagrangian is described for nonlinear problems with equality constraints. The analysis of the inner iteration of the algorithm proves a superlinear convergence to a stationary point. In addition,the analysis of the outer loop of the method establishes global convergence and shows that the penalty parameter is bounded away from zero. In the second extension, the parallel space decomposition of the mesh adaptive direct search algorithm (PSD-MADS), which is an asynchronous parallel method for large-sized blackbox problems, uses a random selection of variables to build subproblems. Several new strategies are introduced to select the most influential variables and explore the solution space more efficiently. These strategies are based on statistical tools to quantify the influence of variables on the different outputs and use the k-mean clustering method to group variables with the same level of influence together. In addition, a hybrid method combines this new strategy with the random variable selection of the original PSD-MADS. These new methods improve the results of PSD-MADS and are tested on problems with up to 4000 variables

Distributed multimedia systems

A distributed multimedia system (DMS) is an integrated communication, computing, and information system that enables the processing, management, delivery, and presentation of synchronized multimedia information with quality-of-service guarantees. Multimedia information may include discrete media data, such as text, data, and images, and continuous media data, such as video and audio. Such a system enhances human communications by exploiting both visual and aural senses and provides the ultimate flexibility in work and entertainment, allowing one to collaborate with remote participants, view movies on demand, access on-line digital libraries from the desktop, and so forth. In this paper, we present a technical survey of a DMS. We give an overview of distributed multimedia systems, examine the fundamental concept of digital media, identify the applications, and survey the important enabling technologies.published_or_final_versio