Multi-Provider Service Chain Embedding With Nestor

Network function (NF) virtualization decouples NFs from the underlying middlebox hardware and promotes their deployment on virtualized network infrastructures. This essentially paves the way for the migration of NFs into clouds (i.e., NF-as-a-Service), achieving a drastic reduction of middlebox investment and operational costs for enterprises. In this context, service chains (expressing middlebox policies in the enterprise network) should be mapped onto datacenter networks, ensuring correctness, resource efficiency, as well as compliance with the provider's policy. The network service embedding (NSE) problem is further exacerbated by two challenging aspects: 1) traffic scaling caused by certain NFs (e.g., caches and WAN optimizers) and 2) NF location dependencies. Traffic scaling requires resource reservations different from the ones specified in the service chain, whereas NF location dependencies, in conjunction with the limited geographic footprint of NF providers (NFPs), raise the need for NSE across multiple NFPs. In this paper, we present a holistic solution to the multi-provider NSE problem. We decompose NSE into: 1) NF-graph partitioning performed by a centralized coordinator and 2) NF-subgraph mapping onto datacenter networks. We present linear programming formulations to derive near-optimal solutions for both problems. We address the challenging aspect of traffic scaling by introducing a new service model that supports demand transformations. We also define topology abstractions for NF-graph partitioning. Furthermore, we discuss the steps required to embed service chains across multiple NFPs, using our NSE orchestrator (Nestor). We perform an evaluation study of multi-provider NSE with emphasis on NF-graph partitioning optimizations tailored to the client and NFPs. Our evaluation results further uncover significant savings in terms of service cost and resource consumption due to the demand transformations. © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works..EU/FP7/T-NOVA/619520DFG/Collaborative Research Center/1053 (MAKI)EU/FP7/T-NOVADFG/CRC/105

Multi-provider network service embedding

[no abstract

Development of a model simplification procedure for integrated urban water system models : conceptual catchment and sewer modelling

La modélisation intégrée du système d’assainissement urbain offre la flexibilité nécessaire pour développer des solutions qui bénéficient le plus au système global, en mettant l'accent sur la quantité et la qualité de l'eau, Les modèles intégrés offrent des avantages par rapport aux modèles traditionnels des sous-systèmes individuels en facilitant l’analyse efficace des interactions entre ces différents systèmes individuels (c.-à-d. les bassins versants, les égouts, les stations d’épuration et les eaux réceptrices) dans une seule plateforme de modélisation. La complexité réduite de ce type de modèle diminue le fardeau de calcul par rapport à leurs homologues détaillés, ce qui permet une plus large gamme d'évaluations telles que l'analyse de scénarios, l'optimisation par contrôle en temps réel et l’analyse d'incertitude par approche Monte Carlo. Le potentiel de créer ces types de modèles intégrés représentatifs a été démontré dans de multiples études, mais les méthodes existantes pour développer ces modèles ne sont pas bien établies ni bien documentées et nécessitent donc un grand effort pour chaque nouveau cas d’étude. De plus, l'absence d'une méthode standardisée pour représenter la partie du modèle qui simule la quantité d'eau limite l'application de ces modèles pour des études de qualité de l'eau. Bien que la recherche soit nécessaire pour développer et optimiser toutes les méthodologies impliquées dans le développement de modèles intégrés de systèmes d'eaux usées urbaines, ce projet se concentre sur les modèles conceptuels simplifiés des bassins versants et des égouts pour la quantité d'eau. L'objectif de cette étude était de développer une procédure structurée pour traduire des modèles hydrologiques et hydrauliques détaillés en modèles conceptuels simplifiés utilisés dans la modélisation du système intégré des eaux usées urbaines. L'objectif était d'améliorer la répétabilité, la flexibilité et l'efficacité de l'approche générale, indépendamment de la plateforme de modélisation choisie. Cette tâche a été réalisée en extrayant les principales étapes et considérations tout en construisant deux modèles conceptuels simplifiés d'une étude de cas au centre d'Ottawa, au Canada. La partie urbaine centrale (6 400 ha) d'un modèle détaillé PCSWMM de la Ville d'Ottawa, contenant une combinaison d'égouts séparés, partiellement séparés et combinés, a été utilisée comme modèle de référence dans cette étude de cas. La tâche principale consistait à déterminer comment traduire ce modèle détaillé en modèle conceptuel simplifié de manière structurée, systématique et répétable en utilisant WEST comme plateforme. La procédure développée suit une séquence similaire à celle des protocoles examinés dans la revue de la littérature, tout en tenant compte des spécificités liées à l'agrégation des bassins versants et des égouts. Les quatre phases principales sont la définition du projet, le développement du modèle, la calibration et la validation. Deux versions du modèle conceptuel ont été créées : le premier a d'abord été créé avec un certain niveau d'agrégation, tandis que le deuxième était plus agrégé que le premier modèle, avec environ la moitié du nombre de bloques et de réservoirs. Les deux modèles ont été calibrés et comparés au modèle détaillé. Les résultats des simulations ont montré que le volume total et la dynamique des débits calculés par les modèles conceptuels ont bien émulé ceux du modèle détaillé (< < 10% de différence), tout en fournissant une réduction significative du temps de calcul (10 à 80 fois). La réduction du temps de simulation pour le modèle le plus agrégé n'était pas équivalente au niveau d'agrégation augmentée, principalement parce qu’il y a une quantité de code qui est présente dans les deux codes et prend donc le même temps de calcul. Comme généralement anticipé, des différences plus grandes, mais acceptables, ont été observées en validation. Ces différences ont été attribuées à plusieurs facteurs, tels que le manque de calibration avec des données sur une période longue, les représentations simplifiées des structures spéciales, les différences entre les mécanismes utilisés dans les modèles détaillés et conceptuels pour représenter le durée de pluie, et la configuration du code de modèle. Dans l'ensemble, la validation a été une réussite étant donné que la calibration a été effectuée à l'aide d'événements de courte durée alors que la validation a utilisé une longue série de données. En général, la procédure conçue a permis de réduire le travail manuel associé à la construction d'un modèle et à bien structurer la façon de construire des modèles conceptuels. Des connaissances pour chacune des différentes phases de modélisation ont également été acquises tout au long du processus du développement des deux modèles. Dans la phase ‹‹ Définition du projet ››, les objectifs du modèle conceptuel ont guidé la méthode de développement et de calibration du modèle. Les bassins versants et les égouts ont été délimités simultanément dans la phase de ‹‹ Développement du modèle ››, tout en tenant compte des emplacements des structures hydrauliques clés, des pluviomètres et des structures de débordement. La phase de ‹‹ Calibration ›› a permis l'avancement le plus systématique étant donné qu'un bon ordre de calibration a été défini et un ensemble limité de paramètres a été ciblé pour chacune des étapes de calibration. La phase de ‹‹ Validation ›› s'est révélée essentielle pour repérer des lacunes dans les hypothèses de base et les valeurs calibrées, afin de déterminer si le modèle est prêt à être utilisé ou doit être modifié. Une procédure efficace et structurée qui traduit les représentations des bassins versants urbains et des égouts de modèles détaillés en modèles intégrés conceptuels a été développée et appliquée avec succès à une étude de cas. Comme démontré dans ce projet, l'application de la procédure structurée mènera au développement efficace de modèles intégrés représentatifs, ce qui augmentera leur utilisation potentielle pour tester des scénarios réalistes. Pour raffiner et améliorer la procédure formulée, il est recommandé de l'appliquer à d’autres études de cas.Modelling urban wastewater networks within integrated systems, focusing on both water quantity and quality, introduces flexibility to develop solutions with greatest benefit to the overall system. Integrated models provide benefits over traditional single sub-system models by facilitating efficient analysis of interactions between the individual components of urban water systems (i.e. catchments, sewers, treatment plants, and receiving waters) within a single modelling platform. The reduced complexity of this type of model decreases the computational burden compared to their detailed counterparts. This allows for a wider range of assessments such as scenario-testing, RTC optimization, and Monte Carlo uncertainty analyses. The potential to create these types of representative integrated models was proven in multiple studies, however, the current methods to develop these models are not well-established nor well documented, and therefore require significant work for each case study. Furthermore, the lack of a standardized method to represent the water quantity portion limits the wide-scale application of such models for water quality studies. Although research is required to further develop and optimize all methodologies involved with building Integrated Urban Wastewater System (IUWS) models, this project focuses on the simplified catchment and sewer conceptual models for water quantity. The objective of this study was to develop a structured procedure to translate detailed hydrologic and hydraulic models into the simplified conceptual models used in IUWS modelling. The aim was to improve repeatability, flexibility and efficiency of the general approach, regardless of chosen modelling platforms. This task was achieved by extracting the key steps and considerations while building two simplified conceptual models of a case study in central Ottawa, Canada. The central urban portion (6,400 ha) of a calibrated detailed PCSWMM model of the City of Ottawa, containing a mix of separated, partially-separated and combined sewer areas, was used as the reference model in this case study. The main task involved determining how to translate this detailed model into simplified conceptual models, using WEST as the platform, in a structured, systematic and repeatable way. The resultant developed procedure follows a similar sequence as the protocols reviewed in the literature review, while taking into consideration specifics related to aggregating catchments and sewers. The four main phases of this thesis are Project Definition, Model Development, Calibration and Validation. Two versions of the lumped model were created; the first was created with a certain level of aggregation, while the second was a further aggregation of the first model, resulting in about half the number of blocks and reservoirs. Both models were calibrated and compared to the detailed model as well as to each other. The simulation results showed that the volume and dynamics (ie. the shape of the hydrographs) of the conceptual models emulated those of the detailed model well (< < 10% differences), while providing a significant reduction in simulation-time speed-up (10 to 80 times faster than the detailed model). The simulation time reduction in the more aggregated model was not equivalent to the increased level of aggregation, mostly due to the fixed amount of basic calculation required in each model. As generally expected, larger but acceptable differences were found during the validation period compared to the calibration period. These differences were attributed to several factors, such as the lack of a long-time series calibration, oversimplified representations of special structures, the different mechanisms in the detailed and conceptual models used to represent wet weather flow, and the configuration of the model code. Overall, the validation was successful given the fact that the calibration was performed using events whereas the validation used an extended time series of 45 days. In general, the devised procedure helped reduce the manual labour associated with building a model and structured the approach to build the conceptual models. General findings from the various identified phases were also documented throughout the model building process. In the Project Definition phase, the conceptual model’s objectives guided the method of model development and calibration. The catchments and sewers were delineated concurrently in the Model Development phase, while taking into consideration the locations of the key hydraulic structures, raingauges and overflows. The Calibration phase allowed for the most systematic advancement of the model build, given that a good calibration order was defined and a limited set of parameters was targeted in each successive run. The Validation phase proved critical in pinpointing deficiencies in the initial assumptions and calibrated values, thus determining whether the model is ready for use or needs to be modified through one of the preceding phases. An efficient and structured procedure that translates catchment and sewer representations from detailed to conceptual models in IUWS was developed and successfully applied to a case study. As demonstrated in this project, applying the proposed structured procedure will lead to the efficient development of representative IUWS models, thus increasing their potential use to test real-life scenarios. To challenge and improve the formulated procedure, applying it to multiple case studies is recommended

Graph Neural Networks for Pressure Estimation in Water Distribution Systems

Pressure and flow estimation in Water Distribution Networks (WDN) allows water management companies to optimize their control operations. For many years, mathematical simulation tools have been the most common approach to reconstructing an estimate of the WDN hydraulics. However, pure physics-based simulations involve several challenges, e.g. partially observable data, high uncertainty, and extensive manual configuration. Thus, data-driven approaches have gained traction to overcome such limitations. In this work, we combine physics-based modeling and Graph Neural Networks (GNN), a data-driven approach, to address the pressure estimation problem. First, we propose a new data generation method using a mathematical simulation but not considering temporal patterns and including some control parameters that remain untouched in previous works; this contributes to a more diverse training data. Second, our training strategy relies on random sensor placement making our GNN-based estimation model robust to unexpected sensor location changes. Third, a realistic evaluation protocol considers real temporal patterns and additionally injects the uncertainties intrinsic to real-world scenarios. Finally, a multi-graph pre-training strategy allows the model to be reused for pressure estimation in unseen target WDNs. Our GNN-based model estimates the pressure of a large-scale WDN in The Netherlands with a MAE of 1.94mH$_2$O and a MAPE of 7%, surpassing the performance of previous studies. Likewise, it outperformed previous approaches on other WDN benchmarks, showing a reduction of absolute error up to approximately 52% in the best cases.Comment: submitted to Water Resources Research. Huy Truong and Andr\'es Tello contributed equally to this wor

Topology Control Multi-Objective Optimisation in Wireless Sensor Networks: Connectivity-Based Range Assignment and Node Deployment

The distinguishing characteristic that sets topology control apart from other methods, whose motivation is to achieve effects of energy minimisation and an increased network capacity, is its network-wide perspective. In other words, local choices made at the node-level always have the goal in mind of achieving a certain global, network-wide property, while not excluding the possibility for consideration of more localised factors. As such, our approach is marked by being a centralised computation of the available location-based data and its reduction to a set of non-homogeneous transmitting range assignments, which elicit a certain network-wide property constituted as a whole, namely, strong connectedness and/or biconnectedness. As a means to effect, we propose a variety of GA which by design is multi-morphic, where dependent upon model parameters that can be dynamically set by the user, the algorithm, acting accordingly upon either single or multiple objective functions in response. In either case, leveraging the unique faculty of GAs for finding multiple optimal solutions in a single pass. Wherefore it is up to the designer to select the singular solution which best meets requirements. By means of simulation, we endeavour to establish its relative performance against an optimisation typifying a standard topology control technique in the literature in terms of the proportion of time the network exhibited the property of strong connectedness. As to which, an analysis of the results indicates that such is highly sensitive to factors of: the effective maximum transmitting range, node density, and mobility scenario under observation. We derive an estimate of the optimal constitution thereof taking into account the specific conditions within the domain of application in that of a WSN, thereby concluding that only GA optimising for the biconnected components in a network achieves the stated objective of a sustained connected status throughout the duration.fi=Opinnäytetyö kokotekstinä PDF-muodossa.|en=Thesis fulltext in PDF format.|sv=Lärdomsprov tillgängligt som fulltext i PDF-format

A reduced complexity model with graph partitioning for rapid hydraulic assessment of sewer networks

Existing, high-fidelity models for sewer network modelling are accurate but too slow and inflexible for modern applications such as optimisation or scenario analysis. Reduced complexity surrogate modelling has been applied in response to this, however, current approaches are expensive to set up and still require high-fidelity simulations to derive parameters. In this study, we compare and develop graph partitioning algorithms to automatically group sections of sewer networks into semi-distributed compartments. These compartments can then be simulated using sewer network information only in the integrated modelling framework, CityWat-SemiDistributed (CWSD), which has been developed for application to sewer network modelling in this study. We find that combining graph partitioning with CWSD can produce accurate simulations 100-1,000x faster than existing high-fidelity modelling. Because we anticipate that many CWSD users will not have high-fidelity models available, we demonstrate that the approach provides reasonable simulations even under significant parametric uncertainty through a sensitivity analysis. We compare multiple graph partitioning techniques enabling users to specify the spatial aggregation of the partitioned network, also enabling them to preserve key locations for simulation. We test the impact of temporal resolution, finding that accurate simulations can be produced with timesteps up to one hour. Our experiments show a log-log relationship between temporal/spatial resolution and simulation time, enabling users to pre-specify the efficiency and accuracy needed for their applications. We expect that the efficiency and flexibility of our approach may facilitate novel applications of sewer network models ranging from continuous simulations for long-term planning to spatially optimising the placement of network sensors

Implementation and evaluation of the Pitman model in seasonal hydrological forecasting mode using the Kraai River catchment in Eastern Cape South Africa as a case study

Seasonal hydrologic extremes such as drought and floods have devastating impacts on human and natural systems (e.g. 2015-2017 Western Cape drought). Sentence has been reworded to: Therefore, the need for a reliable seasonal hydrologic forecast is significant and becoming even more urgent under future climate, as the assimilation of seasonal forecast information in decision making. Hence, SHF becomes part of the short and long-term climate change adaptation strategies in a range of contexts such as energy supply, water supply and management, rural-urban, agriculture, infrastructure and disaster preparedness and relief. This work deals with implementation and evaluation of the Pitman/Water Resources Simulation Model 2012 model (WR2012) in seasonal hydrological forecasting mode. The aim of the study is to improve the understanding of seasonal hydrological forecasting by evaluating the performance of a hydrological model (Pitman Model) in the seasonal forecast mode in Kraai River tertiary catchment (D13) as a case study and the objectives are: To determine steps to be undertaken to implement integration of Pitman in WR2012 configuration with climate forecast to generate seasonal hydrological forecast and to evaluate the performance of the model forced by climate model data in the simulation and forecast mode. Pitman model in the WR2012 version works with a specific rainfall dataset spanning the period of 1920-2009. Operationalizing the seasonal hydrological forecast with Pitman model requires, therefore, updating of the WR2012 rainfall so that it extends to-date. To achieve that, two datasets were evaluated: Climate Hazards Group InfraRed Precipitation with Station Data (CHIRPS), which is a satellite-based gridded rainfall dataset, and rain gauge-based dataset from South African Weather Service (SAWS). The analyses revealed that CHIRPS rainfall data had better correlation and lower bias with respect to the WR2012 data when compared with SAWS rainfall data for the overlap period 1981-2009. The CHIRPS data showed no significant difference from the WR2012 in all the three rainfall zones of the Kraai River catchment. Therefore, CHIRPS data were used to extend the WR2012 data and were used as input to set up Pitman model/WR2012 in the seasonal hydrological forecasting mode. The Pitman/WR2012 model was forced with 10 ensemble seasonal climate forecast from Climate Forecast Systems v.2 which is downscaled using the Principal Components Regression (PCR) approach. The generated seasonal hydrological forecast focused on the summer season, in particular on the Dec-Jan-Feb (DJF) period, which is the rainy season in the catchment. The hydrological forecast showed skills more especially in Dec and Feb (assessed through ROC and RPSS forecast verification methods) with Jan having a poor skill. Importantly, the skill of streamflow forecast was better than that of rainfall forecast, which likely results from the influence of initial conditions of the hydrological model. In conclusion Pitman/WR2012 model can perform realistically when implemented in seasonal hydrological forecasts mode, and it is important that in that model, the model is run with near real time rainfall data in order to achieve good initial conditions. However, the results in terms of forecast skill are specific to the studied catchment and analysed forecast, and skill of forecast in any other catchment has to be investigated separately