Dynamic data placement and discovery in wide-area networks

The workloads of online services and applications such as social networks, sensor data platforms and web search engines have become increasingly global and dynamic, setting new challenges to providing users with low latency access to data. To achieve this, these services typically leverage a multi-site wide-area networked infrastructure. Data access latency in such an infrastructure depends on the network paths between users and data, which is determined by the data placement and discovery strategies. Current strategies are static, which offer low latencies upon deployment but worse performance under a dynamic workload. We propose dynamic data placement and discovery strategies for wide-area networked infrastructures, which adapt to the data access workload. We achieve this with data activity correlation (DAC), an application-agnostic approach for determining the correlations between data items based on access pattern similarities. By dynamically clustering data according to DAC, network traffic in clusters is kept local. We utilise DAC as a key component in reducing access latencies for two application scenarios, emphasising different aspects of the problem: The first scenario assumes the fixed placement of data at sites, and thus focusses on data discovery. This is the case for a global sensor discovery platform, which aims to provide low latency discovery of sensor metadata. We present a self-organising hierarchical infrastructure consisting of multiple DAC clusters, maintained with an online and distributed split-and-merge algorithm. This reduces the number of sites visited, and thus latency, during discovery for a variety of workloads. The second scenario focusses on data placement. This is the case for global online services that leverage a multi-data centre deployment to provide users with low latency access to data. We present a geo-dynamic partitioning middleware, which maintains DAC clusters with an online elastic partition algorithm. It supports the geo-aware placement of partitions across data centres according to the workload. This provides globally distributed users with low latency access to data for static and dynamic workloads.Open Acces

Contributions aux systèmes répartis en environnements ubiquitaires : adaptation, sensibilité au contexte et tolérance aux fautes

D'années en années, nous observons l'arrivée sur le marche d'ordinateurs personnels de plus en plus petits pour des utilisateurs de plus en plus nombreux, ainsi des assistants personnels numériques et des objets dits connectés, en passant par les téléphones mobiles. Tous ces dispositifs tendent à être interchangeables du point de vue des ressources en mémoire, en calcul et en connectivité : par exemple, les téléphones mobiles sont devenus des équipements informatiques de moins en moins spécialisés ou de plus en plus universels et font dorénavant office en la matière de portails d'accès aux capteurs présents dans l'environnement immédiat de l'utilisateur. L'enjeu abordé dans nos travaux est la construction de systèmes répartis incluant ces nouveaux dispositifs matériels. L'objectif de mes recherches est la conception des paradigmes d'intermédiation génériques sous-jacents aux applications réparties de plus en plus ubiquitaires. Plus particulièrement, la problématique générale de mes travaux est la définition du rôle des intergiciels dans l'intégration des dispositifs mobiles et des objets connectés dans les architectures logicielles réparties. Ces architectures logicielles reposaient très majoritairement sur des infrastructures logicielles fixes au début des travaux présentés dans ce manuscrit. Dans ce manuscrit, je décris mes travaux sur trois sujets : 1) l'adaptation des applications réparties pour la continuité de service pendant les déconnexions, 2) la gestion des informations du contexte d'exécution des applications réparties pour leur sensibilité au contexte, et 3) les mécanismes de détection des entraves dans les environnements fortement dynamiques tels que ceux construits avec des réseaux mobiles spontanés. Sur le premier sujet, nous fournissons une couche intergicielle générique pour la gestion des aspects répartis de la gestion des déconnexions en utilisant une stratégie d'adaptation collaborative dans les architectures à base d'objets et de composants. Sur le deuxième sujet, nous étudions les paradigmes architecturaux pour la construction d'un service de gestion de contexte générique, afin d'adresser la diversité des traitements (fusion et agrégation, corrélation, détection de situation par apprentissage, etc.), puis nous adressons le problème de la distribution des informations de contexte aux différentes échelles de l'Internet des objets. Enfin, sur le troisième sujet, nous commençons par la détection des modes de fonctionnement pour l'adaptation aux déconnexions afin de faire la différence, lorsque cela est possible, entre une déconnexion et une défaillance, et ensuite nous spécifions et construisons un service de gestion de groupe partitionnable. Ce service est assez fort pour interdire la construction de partitions ne correspondant pas à la réalité de l'environnement à un instant donné et est assez faible pour être mis en oeuvre algorithmiquemen

Scalable Emulation of Heterogeneous Systems

The breakdown of Dennard's transistor scaling has driven computing systems toward application-specific accelerators, which can provide orders-of-magnitude improvements in performance and energy efficiency over general-purpose processors. To enable the radical departures from conventional approaches that heterogeneous systems entail, research infrastructure must be able to model processors, memory and accelerators, as well as system-level changes---such as operating system or instruction set architecture (ISA) innovations---that might be needed to realize the accelerators' potential. Unfortunately, existing simulation tools that can support such system-level research are limited by the lack of fast, scalable machine emulators to drive execution. To fill this need, in this dissertation we first present a novel machine emulator design based on dynamic binary translation that makes the following improvements over the state of the art: it scales on multicore hosts while remaining memory efficient, correctly handles cross-ISA differences in atomic instruction semantics, leverages the host floating point (FP) unit to speed up FP emulation without sacrificing correctness, and can be efficiently instrumented to---among other possible uses---drive the execution of a full-system, cross-ISA simulator with support for accelerators. We then demonstrate the utility of machine emulation for studying heterogeneous systems by leveraging it to make two additional contributions. First, we quantify the trade-offs in different coupling models for on-chip accelerators. Second, we present a technique to reuse the private memories of on-chip accelerators when they are otherwise inactive to expand the system's last-level cache, thereby reducing the opportunity cost of the accelerators' integration

Detecting Non-Line of Sight to Prevent Accidents in Vehicular Ad hoc Networks

There are still many challenges in the field of VANETs that encouraged researchers to conduct further investigation in this field to meet these challenges. The issue pertaining to routing protocols such as delivering the warning messages to the vehicles facing Non-Line of Sight (NLOS) situations without causing the storm problem and channel contention, is regarded as a serious dilemma which is required to be tackled in VANET, especially in congested environments. This requires the designing of an efficient mechanism of routing protocol that can broadcast the warning messages from the emergency vehicles to the vehicles under NLOS, reducing the overhead and increasing the packet delivery ratio with a reduced time delay and channel utilisation. The main aim of this work is to develop the novel routing protocol for a high-density environment in VANET through utilisation of its high mobility features, aid of the sensors such as Global Positioning System (GPS) and Navigation System (NS). In this work, the cooperative approach has been used to develop the routing protocol called the Co-operative Volunteer Protocol (CVP), which uses volunteer vehicles to disseminate the warning message from the source to the target vehicle under NLOS issue; this also increases the packet delivery ratio, detection of NLOS and resolution of NLOS by delivering the warning message successfully to the vehicle under NLOS, thereby causing a direct impact on the reduction of collisions between vehicles in normal mode and emergency mode on the road near intersections or on highways. The cooperative approach adopted for warning message dissemination reduced the rebroadcast rate of messages, thereby decreasing significantly the storm issue and the channel contention. A novel architecture has been developed by utilising the concept of a Context-Aware System (CAS), which clarifies the OBU components and their interaction with each other in order to collect data and take the decisions based on the sensed circumstances. The proposed architecture has been divided into three main phases: sensing, processing and acting. The results obtained from the validation of the proposed CVP protocol using the simulator EstiNet under specific conditions and parameters showed that performance of the proposed protocol is better than that of the GRANT protocol with regard to several metrics such as packet delivery ratio, neighbourhood awareness, channel utilisation, overhead and latency. It is also successfully shown that the proposed CVP could detect the NLOS situation and solves it effectively and efficiently for both the intersection scenario in urban areas and the highway scenario

Analyse und Vorhersage der Aktualisierungen von Web-Feeds

Feeds werden unter anderem eingesetzt, um Nutzer in einem einheitlichen Format und in aggregierter Form über Aktualisierungen oder neue Beiträge auf Webseiten zu informieren. Da bei Feeds in der Regel keine Benachrichtigungsfunktionalitäten angeboten werden, müssen Interessenten Feeds regelmäßig auf Aktualisierungen überprüfen. Die Betrachtung entsprechender Techniken bildet den Kern der Arbeit. Die in den verwandten Domänen Web Crawling und Web Caching eingesetzten Algorithmen zur Vorhersage der Zeitpunkte von Aktualisierungen werden aufgearbeitet und an die spezifischen Anforderungen der Domäne Feeds angepasst. Anschließend wird ein selbst entwickelter Algorithmus vorgestellt, der bereits ohne den Einsatz spezieller Konfigurationsparameter und ohne Trainingsphase im Durchschnitt bessere Vorhersagen trifft, als die übrigen betrachteten Algorithmen. Auf Basis der Analyse verschiedener Metriken zur Beurteilung der Qualität von Vorhersagen erfolgt die Definition eines zusammenfassenden Gütemaßes, welches den Vergleich von Algorithmen anhand eines einzigen Wertes ermöglicht. Darüber hinaus werden abfragespezifische Attribute der Feed-Formate untersucht und es wird empirisch gezeigt, dass die auf der partiellen Historie der Feeds basierende Vorhersage von Änderungen bereits bessere Ergebnisse erzielt, als die Einbeziehung der von den Diensteanbietern bereitgestellten Werte in die Berechnung ermöglicht. Die empirischen Evaluationen erfolgen anhand eines breitgefächerten, realen Feed-Datensatzes, welcher der wissenschaftlichen Gemeinschaft frei zur Verfügung gestellt wird, um den Vergleich mit neuen Algorithmen zu erleichtern

Indexed dependence metadata and its applications in software performance optimisation

To achieve continued performance improvements, modern microprocessor design is tending to concentrate an increasing proportion of hardware on computation units with less automatic management of data movement and extraction of parallelism. As a result, architectures increasingly include multiple computation cores and complicated, software-managed memory hierarchies. Compilers have difficulty characterizing the behaviour of a kernel in a general enough manner to enable automatic generation of efficient code in any but the most straightforward of cases. We propose the concept of indexed dependence metadata to improve application development and mapping onto such architectures. The metadata represent both the iteration space of a kernel and the mapping of that iteration space from a given index to the set of data elements that iteration might use: thus the dependence metadata is indexed by the kernel’s iteration space. This explicit mapping allows the compiler or runtime to optimise the program more efficiently, and improves the program structure for the developer. We argue that this form of explicit interface specification reduces the need for premature, architecture-specific optimisation. It improves program portability, supports intercomponent optimisation and enables generation of efficient data movement code. We offer the following contributions: an introduction to the concept of indexed dependence metadata as a generalisation of stream programming, a demonstration of its advantages in a component programming system, the decoupled access/execute model for C++ programs, and how indexed dependence metadata might be used to improve the programming model for GPU-based designs. Our experimental results with prototype implementations show that indexed dependence metadata supports automatic synthesis of double-buffered data movement for the Cell processor and enables aggressive loop fusion optimisations in image processing, linear algebra and multigrid application case studies