Methodology for modeling high performance distributed and parallel systems

Performance modeling of distributed and parallel systems is of considerable importance to the high performance computing community. To achieve high performance, proper task or process assignment and data or file allocation among processing sites is essential. This dissertation describes an elegant approach to model distributed and parallel systems, which combines the optimal static solutions for data allocation with dynamic policies for task assignment. A performance-efficient system model is developed using analytical tools and techniques. The system model is accomplished in three steps. First, the basic client-server model which allows only data transfer is evaluated. A prediction and evaluation method is developed to examine the system behavior and estimate performance measures. The method is based on known product form queueing networks. The next step extends the model so that each site of the system behaves as both client and server. A data-allocation strategy is designed at this stage which optimally assigns the data to the processing sites. The strategy is based on flow deviation technique in queueing models. The third stage considers process-migration policies. A novel on-line adaptive load-balancing algorithm is proposed which dynamically migrates processes and transfers data among different sites to minimize the job execution cost. The gradient-descent rule is used to optimize the cost function, which expresses the cost of process execution at different processing sites. The accuracy of the prediction method and the effectiveness of the analytical techniques is established by the simulations. The modeling procedure described here is general and applicable to any message-passing distributed and parallel system. The proposed techniques and tools can be easily utilized in other related areas such as networking and operating systems. This work contributes significantly towards the design of distributed and parallel systems where performance is critical

Parallel and Distributed Computing

The 14 chapters presented in this book cover a wide variety of representative works ranging from hardware design to application development. Particularly, the topics that are addressed are programmable and reconfigurable devices and systems, dependability of GPUs (General Purpose Units), network topologies, cache coherence protocols, resource allocation, scheduling algorithms, peertopeer networks, largescale network simulation, and parallel routines and algorithms. In this way, the articles included in this book constitute an excellent reference for engineers and researchers who have particular interests in each of these topics in parallel and distributed computing

Inventory routing problem with stochastic demand and lead time

In the supply chain, the integration of the different processes is critical to obtain high levels of coordination. Inventory control and its distribution are two of these processes whose coordination have been demonstrated by researchers as key in order to gain efficiency and effectiveness. They affect the synchronization of the supply chain management. With the intention to contribute to the integration of these processes and improve the problems of demand variability, we propose an integration of operations research area and the help of metaheuristics in a multi-objective approach. The expected results are to reduce the costs associated with inventory and its distribution, as well as to reduce the uncertainty in making decisions based on demand. This thesis presents methods for obtaining and analyzing near optimally solutions for dynamic and stochastic inventory-routing problems. The methods include retailers selection and clustering methods, algorithms and experiments on benchmark instances. We focus on problems with one and several suppliers that serve several dispersal geographically retailers. The thesis contains four parts. In Part I, we focus on the literature review. We first provide an overview of the literature on problems related to the coordination of the inventory and its distribution. Then we make a point in four elements: information management, inventory policies, stochastic demand and optimization methods. Also, we provide a scientometric analysis of the documentation collected in the last ten years. We provide a thorough review of papers working with dynamic and stochastic demand. The contributions of this part are i) the review of papers working with stochastic demand and stochastic lead times focusing on its stochastic and multi-depot aspects, ii) identify critical factors for the performance of many logistics activities and industries, iii) have shown that studying the behavior of the demand and the lead time are essential in order to achieve a useful representation of the system to take proper decisions and iv) provide the trends and patterns in the research in IRP problems. In Part II, we focus on the methodology of the research and of development. We first introduce the problem, state of the science, the gaps in the literature, variables under study, the instruments applied and assumptions. The development methodology is presented by a general model to address this type of research proposed in this thesis. Here, the general development process, decomposition of the problem and how the possible solutions are explained.. The importance of the this chapter is provided an effective way to face IRP problems. In Part III, the foundations in formulations for IRP problems are proposed. We begin with the formulation of the TSP problems with variants for one and many suppliers, likewise for VRP and IRP problems. The contributions of the model presented here aim identifying the variables and mathematical models frequently used to deal with these problems. In Part IV, we perform a single criteria objective and multi-criteria analysis of the solutions for one and many suppliers instances. Our methods yield significant improvements over a competing algorithm. Our contributions are i) propose three new customer selection methods for a dynamic and stochastic inventory-routing vii problem, ii) perform a multi-criteria analysis of the solutions, comparing distribution versus inventory management, iii) perform a single criteria objective experiment on benchmark instances from the literature.En la cadena de suministro, la integración de los diferentes procesos que la conforman, es fundamental para obtener altos niveles de coordinación. El control del inventario y su distribución son dos de estos procesos, cuya coordinación ha sido demostrada por los investigadores como clave para lograr mejoras en eficiencia y efectividad. Estos a su vez, afectan la sincronización y la administración de la cadena de suministro. Con el propósito de contribuir en la integración de éstos procesos y mejorar los problemas derivados de la variabilidad de la demanda, se propone usar los fundamentos del área de investigación de operaciones y la ayuda de metaheurísticas en un enfoque multi-obejtivo. Los resultados esperados son reducir los costos asociados a los procesos de inventario y distribución, así como también reducir la incertidumbre en la toma de decisiones a partir de la demanda. Ésta tesis presenta métodos para el análisis y obtención de soluciones cercanas a las óptimas para problemas de inventario y routeo, dinámico y estocástico. Los métodos incluyen selección de retailers y métodos de clustering, algoritmos y experimentos en instancias de prueba disponibles en la literatura. Se hace énfasis en instancias de un solo proveedor y varios proveedores que sirven varios retailers distribuidos geográficamente. La tesis está organizada en cuatro partes. En la Parte I, se revisa la literatura, para ello, primero se presentan los problemas relacionados con la coordinación del inventario y su distribución. Ésta revisión resalta cuatro elementos que han sido identificados como claves en la literatura como son: la administración de la información, políticas de inventario, demanda estocástica y métodos de optimización. Luego, se presenta un análisis cienciometrico de la literatura encontrada en los últimos 10 años. La revisión de la documentación se realiza de manera exhaustiva trabajando con demanda dinámica y estocástica. Las contribuciones de esta parte son: i) proporcionar una revisión pertinente y actualizada de artículos que emplean demanda estocástica, enfatizando en sus elementos dinámicos y estocásticos, así como también en aspectos que permitan abordar problemas con múltiples depósitos, ii) identificar factores críticos para el desempeño de actividades logísticas, iii) Demostrar que el estudio de la demanda es esencial para lograr una representación útil del sistema, la cual influye en la toma de decisiones y iv) proporcionar tendencias y patrones en la investigación de problemas de IRP. En la Parte II se aborda la metodología de la investigación y de desarrollo. Primero, se presenta el problema, el estado de la ciencia y los gaps encontrados en la literatura. Luego se identifican las variables de estudio, los instrumentos aplicados y los supuestos utilizados. La metodología de desarrollo es presentada por medio de un modelo general para abordar éste tipo de investigaciones que nosotros proponemos en ésta tesis. Esta metodología aborda aspectos como: el procedimiento general de desarrollo, la descomposición del problema y la forma en que se prueban las posibles soluciones. En la Parte III, se presentan los fundamentos en la formulación de IRP. Primero se formulan los problemas TSP con variantes para un solo depósito y también paramúltiples depósitos, igualmente se hace para VRP e IRP. La contribución de los modelos presentados son la identificación de las variables y los modelos matemáticos que frecuentemente son usados para tratar con éste tipo de problemas. En la Parte IV se presentan dos experimentos. El primero para el análisis de instancias con uno sólo depósito y en el segundo para analizar instancias con múltiples depósitos. Los métodos usados producen mejoras sobre resultados obtanidos con algoritmos similares. Las contribuciones de ésta parte son: i) proponer tres nuevos métodos para la selección de retailers para IRP dinámicos y estocásticos, ii) realizar análisis multi-criterio de las soluciones, comparando la distribución con la administración del inventario y iii) realizar análisis de un solo objetivo sobre instancias de pruebas proporcionada por la literatura existente

Towards Scalable Personalization

The ever-growing amount of online information calls for Personalization. Among the various personalization systems, recommenders have become increasingly popular in recent years. Recommenders typically use collaborative filtering to suggest the most relevant items to their users. The most prominent challenges underlying personalization are: scalability, privacy, and heterogeneity. Scalability is challenging given the growing rate of the Internet and its dynamics, both in terms of churn (i.e., users might leave/join at any time) and changes of user interests over time. Privacy is also a major concern as users might be reluctant to expose their profiles to unknown parties (e.g., other curious users), unless they have an incentive to significantly improve their navigation experience and sufficient guarantees about their privacy. Heterogeneity poses a major technical difficulty because, to be really meaningful, the profiles of users should be extracted from a number of their navigation activities (heterogeneity of source domains) and represented in a form that is general enough to be leveraged in the context of other applications (heterogeneity of target domains). In this dissertation, we address the above-mentioned challenges. For scalability, we introduce democratization and incrementality. Our democratization approach focuses on iteratively offloading the computationally expensive tasks to the user devices (via browsers or applications). This approach achieves scalability by employing the devices of the users as additional resources and hence the throughput of the approach (i.e., number of updates per unit time) scales with the number of users. Our incrementality approach deals with incremental similarity metrics employing either explicit (e.g., ratings) or implicit (e.g., consumption sequences for users) feedback. This approach achieves scalability by reducing the time complexity of each update, and thereby enabling higher throughput. We tackle the privacy concerns from two perspectives, i.e., anonymity from either other curious users (user-level privacy) or the service provider (system-level privacy). We strengthen the notion of differential privacy in the context of recommenders by introducing distance-based differential privacy (D2P) which prevents curious users from even guessing any category (e.g., genre) in which a user might be interested in. We also briefly introduce a recommender (X-REC) which employs uniform user sampling technique to achieve user-level privacy and an efficient homomorphic encryption scheme (X-HE) to achieve system-level privacy. We also present a heterogeneous recommender (X-MAP) which employs a novel similarity metric (X-SIM) based on paths across heterogeneous items (i.e., items from different domains). To achieve a general form for any user profile, we generate her AlterEgo profile in a target domain by employing an item-to-item mapping from a source domain (e.g., movies) to a target domain (e.g., books). Moreover, X-MAP also enables differentially private AlterEgos. While X-MAP employs user-item interactions (e.g., ratings), we also explore the possibility of heterogeneous recommendation by using content-based features of users (e.g., demography, time-varying preferences) or items (e.g., popularity, price)

Performance Evaluation - Annual Report Year 3

This report describes the work done and results obtained in third year of the CATNETS project. Experiments carried out with the different configurations of the prototype are reported and simulation results are evaluated with the CATNETS metrics framework. The applicability of the Catallactic approach as market model for service and resource allocation in application layer networks is assessed based on the results and experience gained both from the prototype development and simulations. --Grid Computing

Optimization of time-dependent routing problems considering dynamic paths and fuel consumption

Ces dernières années, le transport de marchandises est devenu un défi logistique à multiples facettes. L’immense volume de fret a considérablement augmenté le flux de marchandises dans tous les modes de transport. Malgré le rôle vital du transport de marchandises dans le développement économique, il a également des répercussions négatives sur l’environnement et la santé humaine. Dans les zones locales et régionales, une partie importante des livraisons de marchandises est transportée par camions, qui émettent une grande quantité de polluants. Le Transport routier de marchandises est un contributeur majeur aux émissions de gaz à effet de serre (GES) et à la consommation de carburant. Au Canada, les principaux réseaux routiers continuent de faire face à des problèmes de congestion. Pour réduire significativement l’impact des émissions de GES reliées au transport de marchandises sur l’environnement, de nouvelles stratégies de planification directement liées aux opérations de routage sont nécessaires aux niveaux opérationnel, environnemental et temporel. Dans les grandes zones urbaines, les camions doivent voyager à la vitesse imposée par la circulation. Les embouteillages ont des conséquences défavorables sur la vitesse, le temps de déplacement et les émissions de GES, notamment à certaines périodes de la journée. Cette variabilité de la vitesse dans le temps a un impact significatif sur le routage et la planification du transport. Dans une perspective plus large, notre recherche aborde les Problèmes de distribution temporels (Time-Dependent Distribution Problems – TDDP) en considérant des chemins dynamiques dans le temps et les émissions de GES. Considérant que la vitesse d’un véhicule varie en fonction de la congestion dans le temps, l’objectif est de minimiser la fonction de coût de transport total intégrant les coûts des conducteurs et des émissions de GES tout en respectant les contraintes de capacité et les restrictions de temps de service. En outre, les informations géographiques et de trafic peuvent être utilisées pour construire des multigraphes modélisant la flexibilité des chemins sur les grands réseaux routiers, en tant qu’extension du réseau classique des clients. Le réseau physique sous-jacent entre chaque paire de clients pour chaque expédition est explicitement considéré pour trouver des chemins de connexion. Les décisions de sélection de chemins complètent celles de routage, affectant le coût global, les émissions de GES, et le temps de parcours entre les nœuds. Alors que l’espace de recherche augmente, la résolution des Problèmes de distribution temporels prenant en compte les chemins dynamiques et les vitesses variables dans le temps offre une nouvelle possibilité d’améliorer l’efficacité des plans de transport... Mots clés : Routage dépendant du temps; chemins les plus rapides dépendant du temps; congestion; réseau routier; heuristique; émissions de gaz à effet de serre; modèles d’émission; apprentissage superviséIn recent years, freight transportation has evolved into a multi-faceted logistics challenge. The immense volume of freight has considerably increased the flow of commodities in all transport modes. Despite the vital role of freight transportation in the economic development, it also negatively impacts both the environment and human health. At the local and regional areas, a significant portion of goods delivery is transported by trucks, which emit a large amount of pollutants. Road freight transportation is a major contributor to greenhouse gas (GHG) emissions and to fuel consumption. To reduce the significant impact of freight transportation emissions on environment, new alternative planning and coordination strategies directly related to routing and scheduling operations are required at the operational, environmental and temporal dimensions. In large urban areas, trucks must travel at the speed imposed by traffic, and congestion events have major adverse consequences on speed level, travel time and GHG emissions particularly at certain periods of day. This variability in speed over time has a significant impact on routing and scheduling. From a broader perspective, our research addresses Time-Dependent Distribution Problems (TDDPs) considering dynamic paths and GHG emissions. Considering that vehicle speeds vary according to time-dependent congestion, the goal is to minimize the total travel cost function incorporating driver and GHG emissions costs while respecting capacity constraints and service time restrictions. Further, geographical and traffic information can be used to construct a multigraph modeling path flexibility on large road networks, as an extension to the classical customers network. The underlying physical sub-network between each pair of customers for each shipment is explicitly considered to find connecting road paths. Path selection decisions complement routing ones, impacting the overall cost, GHG emissions, the travel time between nodes, and thus the set of a feasible time-dependent least cost paths. While the search space increases, solving TDDPs considering dynamic paths and time-varying speeds may provide a new scope for enhancing the effectiveness of route plans. One way to reduce emissions is to consider congestion and being able to route traffic around it. Accounting for and avoiding congested paths is possible as the required traffic data is available and, at the same time, has a great potential for both energy and cost savings. Hence, we perform a large empirical analysis of historical traffic and shipping data. Therefore, we introduce the Time-dependent Quickest Path Problem with Emission Minimization, in which the objective function comprises GHG emissions, driver and congestion costs. Travel costs are impacted by traffic due to changing congestion levels depending on the time of the day, vehicle types and carried load. We also develop time-dependent lower and upper bounds, which are both accurate and fast to compute. Computational experiments are performed on real-life instances that incorporate the variation of traffic throughout the day. We then study the quality of obtained paths considering time-varying speeds over the one based only on fixed speeds... Keywords : Time-dependent routing; time-dependent quickest paths; traffic congestion; road network; heuristic; greenhouse gas emissions; emission models; supervised learning

Compressed and distributed least-squares regression: convergence rates with applications to Federated Learning

In this paper, we investigate the impact of compression on stochastic gradient algorithms for machine learning, a technique widely used in distributed and federated learning. We underline differences in terms of convergence rates between several unbiased compression operators, that all satisfy the same condition on their variance, thus going beyond the classical worst-case analysis. To do so, we focus on the case of least-squares regression (LSR) and analyze a general stochastic approximation algorithm for minimizing quadratic functions relying on a random field. We consider weak assumptions on the random field, tailored to the analysis (specifically, expected H\"older regularity), and on the noise covariance, enabling the analysis of various randomizing mechanisms, including compression. We then extend our results to the case of federated learning. More formally, we highlight the impact on the convergence of the covariance $\mathfrak{C}_{\mathrm{ania}}$ of the additive noise induced by the algorithm. We demonstrate despite the non-regularity of the stochastic field, that the limit variance term scales with $\mathrm{Tr}(\mathfrak{C}_{\mathrm{ania}} H^{-1})/K$ (where $H$ is the Hessian of the optimization problem and $K$ the number of iterations) generalizing the rate for the vanilla LSR case where it is $\sigma^2 \mathrm{Tr}(H H^{-1}) / K = \sigma^2 d / K$ (Bach and Moulines, 2013). Then, we analyze the dependency of $\mathfrak{C}_{\mathrm{ania}}$ on the compression strategy and ultimately its impact on convergence, first in the centralized case, then in two heterogeneous FL frameworks

Cost-Based Optimization of Integration Flows

Integration flows are increasingly used to specify and execute data-intensive integration tasks between heterogeneous systems and applications. There are many different application areas such as real-time ETL and data synchronization between operational systems. For the reasons of an increasing amount of data, highly distributed IT infrastructures, and high requirements for data consistency and up-to-dateness of query results, many instances of integration flows are executed over time. Due to this high load and blocking synchronous source systems, the performance of the central integration platform is crucial for an IT infrastructure. To tackle these high performance requirements, we introduce the concept of cost-based optimization of imperative integration flows that relies on incremental statistics maintenance and inter-instance plan re-optimization. As a foundation, we introduce the concept of periodical re-optimization including novel cost-based optimization techniques that are tailor-made for integration flows. Furthermore, we refine the periodical re-optimization to on-demand re-optimization in order to overcome the problems of many unnecessary re-optimization steps and adaptation delays, where we miss optimization opportunities. This approach ensures low optimization overhead and fast workload adaptation

Proceedings of the 8th Cologne-Twente Workshop on Graphs and Combinatorial Optimization

International audienceThe Cologne-Twente Workshop (CTW) on Graphs and Combinatorial Optimization started off as a series of workshops organized bi-annually by either Köln University or Twente University. As its importance grew over time, it re-centered its geographical focus by including northern Italy (CTW04 in Menaggio, on the lake Como and CTW08 in Gargnano, on the Garda lake). This year, CTW (in its eighth edition) will be staged in France for the first time: more precisely in the heart of Paris, at the Conservatoire National d’Arts et Métiers (CNAM), between 2nd and 4th June 2009, by a mixed organizing committee with members from LIX, Ecole Polytechnique and CEDRIC, CNAM