BIGhybrid - A Toolkit for Simulating MapReduce on Hybrid Infrastructures

Cloud computing has increasingly been used as a platform for running large business and data processing applications. Although clouds have become highly popular, when it comes to data processing, the cost of usage is not negligible. Conversely, Desktop Grids, have been used by a plethora of projects, taking advantage of the high number of resources provided for free by volunteers. Merging cloud computing and desktop grids into hybrid infrastructure can provide a feasible low-cost solution for big data analysis. Although frameworks like MapReduce have been conceived to exploit commodity hardware, their use on hybrid infrastructure poses some challenges due to large resource heterogeneity and high churn rate. This study introduces BIGhybrid a toolkit to simulate MapReduce on hybrid environments. The main goal is to provide a framework for developers and system designers to address the issues of hybrid MapReduce. In this paper, we describe the framework which simulates the assembly of two existing middleware: BitDew- MapReduce for Desktop Grids and Hadoop-BlobSeer for Cloud Computing. Experimental results included in this work demonstrate the feasibility of our approach

HIV transmission dynamics: Infectivity, sexual partnership patterns, and the role of early infection

Although remarkable progress has been made in the diagnosis, treatment, and prevention of HIV, a cure is unavailable and many of the most promising prevention interventions have failed. At this critical juncture in the epidemic, there is a necessity for improved understanding of the fundamental drivers of the epidemic, as well as an urgent need for innovative interventions against HIV. This dissertation focuses on two of these fundamental drivers - the heterosexual infectivity of HIV-1 and the details of sexual partnership patterns - as well as the power of interventions initiated during the highly infectious period of early HIV infection (EHI). We conducted a systematic review and meta-analysis of the heterosexual infectivity of HIV-1, defined as the per-contact probability of HIV-1 transmission in a single heterosexual contact between an infected and a susceptible individual. Infectivity estimates were extremely heterogeneous, ranging from zero transmissions after more than 100 penile-vaginal contacts in some sero-discordant couples to one transmission for every 3.1 episodes of heterosexual anal intercourse. Several co-factors were associated with increased infectivity. Infectivity differences (95% confidence intervals), expressed as number of transmissions per 1000 contacts, were 8 (0-16) comparing uncircumcised to circumcised male susceptibles, 6 (3-9) comparing susceptible individuals with and without GUD, 2 (1-3) comparing late-stage to mid-stage index cases, and 3 (0-5) comparing early-stage to mid-stage index cases. We also analyzed recent sexual partnership patterns in a sexually transmitted infections (STI) clinic in Lilongwe, Malawi. We found that multiple sexual partnerships were uncommon (14%), and partnerships were long on average (mean=858 days). Among those reporting multiple recent partners, patterns ranged from long-term concurrency (mean overlap=246 days) to narrowly spaced consecutive partnerships (mean gap=21 days), presenting a substantial risk for efficient HIV transmission. Finally, we conducted a mathematical modeling study to determine the contribution of EHI to epidemic spread in Lilongwe, Malawi. Our analyses suggest that 38.4% (95% CI: 18.6%-57.5%) of ongoing HIV transmissions in Lilongwe can be attributed to EHI index cases, and that interventions targeting the entire duration of infection will be needed to have a significant, lasting effect on the epidemic

Event-driven industrial robot control architecture for the Adept V+ platform

Modern industrial robotic systems are highly interconnected. They operate in a distributed environment and communicate with sensors, computer vision systems, mechatronic devices, and computational components. On the fundamental level, communication and coordination between all parties in such distributed system are characterized by discrete event behavior. The latter is largely attributed to the specifics of communication over the network, which, in terms, facilitates asynchronous programming and explicit event handling. In addition, on the conceptual level, events are an important building block for realizing reactivity and coordination. Eventdriven architecture has manifested its effectiveness for building loosely-coupled systems based on publish-subscribe middleware, either general-purpose or robotic-oriented. Despite all the advances in middleware, industrial robots remain difficult to program in context of distributed systems, to a large extent due to the limitation of the native robot platforms. This paper proposes an architecture for flexible event-based control of industrial robots based on the Adept V+ platform. The architecture is based on the robot controller providing a TCP/IP server and a collection of robot skills, and a high-level control module deployed to a dedicated computing device. The control module possesses bidirectional communication with the robot controller and publish/subscribe messaging with external systems. It is programmed in asynchronous style using pyadept, a Python library based on Python coroutines, AsyncIO event loop and ZeroMQ middleware. The proposed solution facilitates integration of Adept robots into distributed environments and building more flexible robotic solutions with eventbased logic

Modelling and analysis of Markov reward automata (extended version)

Costs and rewards are important ingredients for cyberphysical systems, modelling critical aspects like energy consumption, task completion, repair costs, and memory usage. This paper introduces Markov reward automata, an extension of Markov automata that allows the modelling of systems incorporating rewards (or costs) in addition to nondeterminism, discrete probabilistic choice and continuous stochastic timing. Rewards come in two flavours: action rewards, acquired instantaneously when taking a transition; and state rewards, acquired while residing in a state. We present algorithms to optimise three reward functions: the expected accumulative reward until a goal is reached; the expected accumulative reward until a certain time bound; and the long-run average reward. We have implemented these algorithms in the SCOOP/IMCA tool chain and show their feasibility via several case studies

Finite horizon analysis of Markov automata

Markov automata constitute an expressive continuous-time compositional modelling formalism, featuring stochastic timing and nondeterministic as well as probabilistic branching, all supported in one model. They span as special cases, the models of discrete and continuous-time Markov chains, as well as interactive Markov chains and probabilistic automata. Moreover, they might be equipped with reward and resource structures in order to be used for analysing quantitative aspects of systems, like performance metrics, energy consumption, repair and maintenance costs. Due to their expressive nature, they serve as semantic backbones of engineering frameworks, control applications and safety critical systems. The Architecture Analysis and Design Language (AADL), Dynamic Fault Trees (DFT) and Generalised Stochastic Petri Nets (GSPN) are just some examples. Their expressiveness thus far prevents them from efficient analysis by stochastic solvers and probabilistic model checkers. A major problem context of this thesis lies in their analysis under some budget constraints, i.e. when only a finite budget of resources can be spent by the model. We study mathematical foundations of Markov automata since these are essential for the analysis addressed in this thesis. This includes, in particular, understanding their measurability and establishing their probability measure. Furthermore, we address the analysis of Markov automata in the presence of both reward acquisition and resource consumption within a finite budget of resources. More specifically, we put the problem of computing the optimal expected resource-bounded reward in our focus. In our general setting, we support transient, instantaneous and final reward collection as well as transient resource consumption. Our general formulation of the problem encompasses in particular the optimal time-bound reward and reachability as well as resource-bounded reachability. We develop a sound theory together with a stable approximation scheme with a strict error bound to solve the problem in an efficient way. We report on an implementation of our approach in a supporting tool and also demonstrate its effectiveness and usability over an extensive collection of industrial and academic case studies.Markov-Automaten bilden einen mächtigen Formalismus zur kompositionellen Modellierung mit kontinuierlicher stochastischer Zeit und nichtdeterministischer sowie probabilistischer Verzweigung, welche alle in einem Modell unterstützt werden. Sie enthalten als Spezialfälle die Modelle diskreter und kontinuierlicher Markov-Ketten sowie interaktive Markov-Ketten und probabilistischer Automaten. Darüber hinaus können sie mit Belohnungs- und Ressourcenstrukturen ausgestattet werden, um quantitative Aspekte von Systemen wie Leistungsfähigkeit, Energieverbrauch, Reparatur- und Wartungskosten zu analysieren. Sie dienen aufgrund ihrer Ausdruckskraft als semantisches Rückgrat von Engineering Frameworks, Steuerungsanwendungen und sicherheitskritischen Systemen. Die Architekturanalyse und Designsprache (AADL), Dynamic Fault Trees (DFT) und Generalized Stochastic Petri Nets (GSPN) sind nur einige Beispiele dafür. Ihre Aussagekraft verhindert jedoch bisher eine effiziente Analyse durch stochastische Löser und probabilistische Modellprüfer. Ein wichtiger Problemzusammenhang dieser Arbeit liegt in ihrer Analyse unter Budgetbeschränkungen, das heisst wenn nur ein begrenztes Budget an Ressourcen vom Modell aufgewendet werden kann. Wir studieren mathematische Grundlagen von Markov-Automaten, da diese für die in dieser Arbeit angesprochene Analyse von wesentlicher Bedeutung sind. Dazu gehört insbesondere das Verständnis ihrer Messbarkeit und die Festlegung ihrer Wahrscheinlichkeitsmaßes. Darüber hinaus befassen wir uns mit der Analyse von Markov-Automaten in Bezug auf Belohnungserwerb sowie Ressourcenverbrauch innerhalb eines begrenzten Ressourcenbudgets. Genauer gesagt stellen wir das Problem der Berechnung der optimalen erwarteten Ressourcen-begrenzte Belohnung in unserem Fokus. Dieser Fokus umfasst transiente, sofortige und endgültige Belohnungssammlung sowie transienten Ressourcenverbrauch. Unsere allgemeine Formulierung des Problems beinhalet insbesondere die optimale zeitgebundene Belohnung und Erreichbarkeit sowie ressourcenbeschränkte Erreichbarkeit. Wir entwickeln die grundlegende Theorie dazu. Zur effizienten Lösung des Problems entwerfen wir ein stabilen Approximationsschema mit einer strikten Fehlerschranke. Wir berichten über eine Umsetzung unseres Ansatzes in einem Software-Werkzeug und zeigen seine Wirksamkeit und Verwendbarkeit anhand einer umfangreichen Sammlung von industriellen und akademischen Fallstudien

Data infrastructures and spatial models for biodiversity assessment and analysis: applications to vertebrate communities.

In conservation biology the computation of biodiversity maps, based on statistical models is a central concern. These maps, produced with objective and repeatable methods are an essential tool for conservation and monitoring programs as well as for landuse planning. Since the computation of biodiversity maps requires complex and time consuming procedures for data processing and analysis, it is necessary to design methods for homogeneous, scalable and repeatable data management and analysis. Moreover, the huge volume of data used in ecological modelling requires suitable software architectures to store, analyze, retrieve and distribute information in order to support research and management actions in due time. First of all we developed an analysis system (SOS - Species Open Spreader) providing statistical and mathematical models to predict species distribution in relation to a set of predictive environmental and geographical variables The system is composed of a module for data input/output toward and from the GIS and of a package of scripts for the application of different modelling techniques. At present, three statistical techniques are integrated in SOS: Logistic Regression Analysis (LRA), Environmental Niche Factor Analysis (ENFA) and flexible Discriminant Analysis with method BRUTO. Furthermore, two empirical spatial methods of analysis are available within SOS: Habitat Suitability Index (HSI) and Spatial Overlay. The system is designed to work with the GIS (Geographical Information System) soft-ware GRASS and the statistical environment R, coupled together through the SPGRASS6 library. Three different outputs are expected: text and graphical outputs with statistical results and suitability maps. Second, we tested the use of spatial Database Management Systems (Spatial DBMS) to handle wildlife and socio-economic data and we developed a web database application to provide facilities for database access. The information system was built for the Meru district (Tanzania) in the context of an Italian cooperation project of land use planning in Maasai rural areas. We tested two di_erent solutions: SpatiaLite and PostgreSQL-PostGIS; they both offer advanced technical facilities and spatial extensions to analyze spatial data. SpatiaLite is a new solution and offers the main advantages to consist of a unique file and to present a user-friendly interface, which make it the best solution for many applications. in spite of this we used PostgreSQL-PostGIS since it represents a well-established information system supported by libraries for web applications development. We applied SOS to three case studies at different spatial scale: Brescia plain (small scale), Mount Meru region - Tanzania (medium scale) and Lombardy region (big scale) in order to produce maps of species potential distribution and biodiversity maps for planning and management. We applied logistic regression analyses to compute models and ROC analysis for classification performance evaluation. The automation of processes through SOS gave us the possibility to build models for a large number of vertebrate species. The analysis produced very reliable results at middle and big scale while regression methods did not converge at small scale. This is probably due to habitat homogeneity and to the use of environmental variables with an insufficient level of detail. The potential distribution and biodiversity maps produced also had in all cases an applicative use in fact we used mammal species models computed for Mt. Meru region to produce a map of biodiversity within the area: this map represents an informative base for land use planning at village level within a cooperation project for Maasai economic development and environmental redemption. Amphibians and reptiles models, computed for Lombardy, represent a good informative base for planning management actions in the region

Big Data and Large-scale Data Analytics: Efficiency of Sustainable Scalability and Security of Centralized Clouds and Edge Deployment Architectures

One of the significant shifts of the next-generation computing technologies will certainly be in the development of Big Data (BD) deployment architectures. Apache Hadoop, the BD landmark, evolved as a widely deployed BD operating system. Its new features include federation structure and many associated frameworks, which provide Hadoop 3.x with the maturity to serve different markets. This dissertation addresses two leading issues involved in exploiting BD and large-scale data analytics realm using the Hadoop platform. Namely, (i)Scalability that directly affects the system performance and overall throughput using portable Docker containers. (ii) Security that spread the adoption of data protection practices among practitioners using access controls. An Enhanced Mapreduce Environment (EME), OPportunistic and Elastic Resource Allocation (OPERA) scheduler, BD Federation Access Broker (BDFAB), and a Secure Intelligent Transportation System (SITS) of multi-tiers architecture for data streaming to the cloud computing are the main contribution of this thesis study