Self-management for large-scale distributed systems

Autonomic computing aims at making computing systems self-managing by using autonomic managers in order to reduce obstacles caused by management complexity. This thesis presents results of research on self-management for large-scale distributed systems. This research was motivated by the increasing complexity of computing systems and their management. In the first part, we present our platform, called Niche, for programming self-managing component-based distributed applications. In our work on Niche, we have faced and addressed the following four challenges in achieving self-management in a dynamic environment characterized by volatile resources and high churn: resource discovery, robust and efficient sensing and actuation, management bottleneck, and scale. We present results of our research on addressing the above challenges. Niche implements the autonomic computing architecture, proposed by IBM, in a fully decentralized way. Niche supports a network-transparent view of the system architecture simplifying the design of distributed self-management. Niche provides a concise and expressive API for self-management. The implementation of the platform relies on the scalability and robustness of structured overlay networks. We proceed by presenting a methodology for designing the management part of a distributed self-managing application. We define design steps that include partitioning of management functions and orchestration of multiple autonomic managers. In the second part, we discuss robustness of management and data consistency, which are necessary in a distributed system. Dealing with the effect of churn on management increases the complexity of the management logic and thus makes its development time consuming and error prone. We propose the abstraction of Robust Management Elements, which are able to heal themselves under continuous churn. Our approach is based on replicating a management element using finite state machine replication with a reconfigurable replica set. Our algorithm automates the reconfiguration (migration) of the replica set in order to tolerate continuous churn. For data consistency, we propose a majority-based distributed key-value store supporting multiple consistency levels that is based on a peer-to-peer network. The store enables the tradeoff between high availability and data consistency. Using majority allows avoiding potential drawbacks of a master-based consistency control, namely, a single-point of failure and a potential performance bottleneck. In the third part, we investigate self-management for Cloud-based storage systems with the focus on elasticity control using elements of control theory and machine learning. We have conducted research on a number of different designs of an elasticity controller, including a State-Space feedback controller and a controller that combines feedback and feedforward control. We describe our experience in designing an elasticity controller for a Cloud-based key-value store using state-space model that enables to trade-off performance for cost. We describe the steps in designing an elasticity controller. We continue by presenting the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores that combines feedforward and feedback control

Envirosuite: An Environmentally-Immersive Programming Framework for Wireless Sensor Networks

Networked, embedded sensors allow for an instrumentation of the physical world at unprecedented granularities and from unimagined perspectives. The advent of a ubiquitous sensing era is evident. Yet, sensor network techniques are still far from entering mainstream adoption due to multiple unresolved research challenges, especially due to the high development cost of sensor network applications. Therefore, in this dissertation, we propose to design, implement, and evaluate an environmentally-immersive programming framework, called EnviroSuite, to reduce sensor network software development cost. The goal of our research is to create reusable sensor network development support for the community and reduce the adoption barriers for a broader category of users, ultimately leading to a transition of sensor networks from a research concept to a general-purpose technology available for use for a wide variety of research, government, industry, and everyday purposes. Current sensor network programming practice remains very cumbersome and inefficient for several reasons. First, most existing programming abstractions for sensor networks are either too low-level (thus too tedious and error-prone) or too high-level (unable to support the diversity of sensor network applications). Second, there is no clear separation between application-level programming and system-level programming. A significant concern is the lack of a general middleware library to isolate application developers from low-level details. Finally, testing sensor network systems is particularly challenging. Sensor systems interact heavily with a (non-repeatable) physical environment, making lab experiments not representative and on-site experiments very costly. This dissertation is targeted for a comprehensive solution that addresses all the above-mentioned problems. The EnviroSuite framework consists of (i) a new programming paradigm that exports environment-based abstractions, (ii) critical middleware services that support the abstractions and separate application programmers from tedious, low-level details, and (iii) testing tools geared for in-situ experimenting, debugging, and troubleshooting. First, we introduce a new programming paradigm, called environmentally-immersive programming (EIP), to capture the common characteristics of sensor network applications, the rich, distributed interactions with the physical environment. EIP refers to an object-based programming model in which individual objects represent physical elements in the external environment. It allows the programmer to think directly in terms of physical objects or events of interest. We provide language primitives for programmers to easily implement their environmental tracking and monitoring applications in EIP. A preprocessor translates such EIP code transparently into a library of support middleware services, central to which are object management algorithms, responsible for maintaining a unique mapping between physical and logical objects. The major outcome of sensor networks is observations of the instrumented environment, in other words, sensory data. Implementing an application mainly involves encoding how to generate, store, and collect such data. EIP object abstractions provide simple means for programmers to define how observations of the environment should be made via distributed coordination among multiple nodes, thus simplifying data generation. Yet, the next steps, namely, data storage and collection, remain complicated and fastidious. To isolate programmers from such concerns, we also include in the support library a set of data management services, comprising both network protocols and storage systems to allow data to be collected either in real-time or in a delay-tolerant manner. The final phase in sensor network software development life-cycle is testing, typically performed in-field, where the effects of environmental realities can be studied. However, physical events from the dynamic environment are normally asynchronous and non-repeatable. This lack of repeatability makes the last phase particularly difficult and costly. Hence, it is essential to have the capability to capture and replay sensing events, providing a basis not only for software testing, but also for realistic protocol comparison and parameter tuning. To achieve that, EnviroSuite also provides testing and debugging facilities that enable controllable and repeatable in-field experiments. Finally, to demonstrate the benefits of our framework, we build multiple representative applications upon EnviroSuite, drawn from both tracking systems such as military surveillance, and monitoring systems such as environmental acoustic monitoring. We install these applications into off-the-shelf hardware platforms and physically deploy the hardware into realistic environments. Empirical results collected from such deployments demonstrate the efficacy of EnviroSuite

Sistema sensor autónomo inalámbrico para el control de riego e iluminación de invernaderos

Greenhouse automation through the Arduino platform emerges as a highly effective and environmentally friendly solution for cultivating plants under optimal conditions. This project is focused on the development of an advanced control system that monitors and regulates a variety of environmental variables, with the primary goal of optimizing plant growth while simultaneously reducing the consumption of essential resources. At the core of this solution lies the Arduino microcontroller, responsible for collecting precise data from a strategically distributed network of sensors throughout the greenhouse. This data is processed and utilized to make automated decisions, such as activating irrigation systems or adjusting lighting. Additionally, an intuitive software has been designed to assist the user in monitoring control parameters from their computer, greatly simplifying comprehensive greenhouse management. This approach to greenhouse automation not only drives efficiency in agricultural production by providing an optimal environment for plant development but also significantly contributes to the conservation of valuable resources. Simultaneously, it offers greenhouse operators more accessible and precise management. In summary, the implementation of Arduino as an automation system marks a step towards the future of agriculture, characterized by sustainability and efficiency.Objectius de Desenvolupament Sostenible::12 - Producció i Consum Responsable

Middleware for Wireless Sensor Networks: An Outlook

In modern distributed computing, applications are rarely built directly atop operating system facilities, e.g., sockets. Higher-level middleware abstractions and systems are often employed to simplify the programmer’s chore or to achieve interoperability. In contrast, real-world wireless sensor network (WSN) applications are almost always developed by relying directly on the operating system. Why is this the case? Does it make sense to include a middleware layer in the design of WSNs? And, if so, is it the same kind of software system as in traditional distributed computing? What are the fundamental concepts, reasonable assumptions, and key criteria guiding its design? What are the main open research challenges, and the potential pitfalls? Most importantly, is it worth pursuing research in this field? This paper provides a (biased) answer to these and other research questions, preceded by a brief account on the state of the art in the field

Security for Grid Services

Grid computing is concerned with the sharing and coordinated use of diverse resources in distributed "virtual organizations." The dynamic and multi-institutional nature of these environments introduces challenging security issues that demand new technical approaches. In particular, one must deal with diverse local mechanisms, support dynamic creation of services, and enable dynamic creation of trust domains. We describe how these issues are addressed in two generations of the Globus Toolkit. First, we review the Globus Toolkit version 2 (GT2) approach; then, we describe new approaches developed to support the Globus Toolkit version 3 (GT3) implementation of the Open Grid Services Architecture, an initiative that is recasting Grid concepts within a service oriented framework based on Web services. GT3's security implementation uses Web services security mechanisms for credential exchange and other purposes, and introduces a tight least-privilege model that avoids the need for any privileged network service.Comment: 10 pages; 4 figure

Practical applications of multi-agent systems in electric power systems

The transformation of energy networks from passive to active systems requires the embedding of intelligence within the network. One suitable approach to integrating distributed intelligent systems is multi-agent systems technology, where components of functionality run as autonomous agents capable of interaction through messaging. This provides loose coupling between components that can benefit the complex systems envisioned for the smart grid. This paper reviews the key milestones of demonstrated agent systems in the power industry and considers which aspects of agent design must still be addressed for widespread application of agent technology to occur