Distributed resource allocation for contributory systems

La tesis presenta una aproximació a un conjunt de tècniques per permetre l'extensió de les capacitats computacionals, sota demanada, en comunitats formades per usuaris d'Internet que volen agregar els seus recursos per aconseguir una finalitat comuna. Aquest tipus de comunitats, emergeixen com a noves formes d'organització gràcies al increment de les capacitats computacionals i l'ampla de banda. La computació voluntària com la que es dóna en sistemes com SETI@home, Grids col·laboratius com OurGrid o LaCOLLA, Ad-hoc i Grids d'igual a igual com P-Grid o X-Grid d'Apple, Grids Oberts com els adreçats per SORMA o Grid4All i moltes d'altres formes de computació Grid basades en agrupació de recursos en forma d'Organitzacions Virtuals són l'objectiu d'aquest treball. Aquests sistemes es caracteritzen pel propòsit dels seus participants, és a dir, dur a terme un objectiu comú fent ús de l'agregació dels seus propis recursos. Els sistemes esmentats, contràriament als sistemes Grid d'alt rendiment computacional, són oberts a nous participants fet que els converteix en escenaris impredictibles, dinàmics i on els recursos poden connectar-se i desconnectar-se de forma espontània. Mentre que l'aspecte crític dels Grids d'altes prestacions és el rendiment computacional, l'estabilitat i la disponibilitat són els aspectes més importants en els sistemes adreçats en aquest treball. La tesis homogeneïtza els conceptes dels paradigmes anteriors sota el nom de Sistemes Contributius, nom que és usat al llarg de la dissertació per referir-nos a sistemes en els quals els seus usuaris fan contribució dels seus recursos per tal que aquest siguin usats de forma col·lectiva i així aconseguir un objectiu comú. L'expansió de recursos en els Sistemes Contributius és una funcionalitat requerida per tal de augmentar les limitades capacitats computacionals dels grups col·laboratius formats de forma ad-hoc. Sobretot en moments puntuals quan els recursos necessaris són majors que els disponibles en el grup. Quatre aspectes són conduïts al llarg de la dissertació. Primer, les propietats i principals aplicacions dels Sistemes Contributius són identificades, així com es motiva la necessitat d'infraestructures que permetin l'expansió de recursos computacionals sota demanda. Aquestes idees van en la direcció de l'Utility Computing, emergents línies de negoci de les principals companyies de las TIC. D'aquesta manera, la tesis proposa la provisió de recursos computacionals sota demanda a aquelles organitzacions que en necessitin, mitjançant l'agregació de recursos dels extrems d'Internet, ja siguin usuaris finals de la xarxa, altres organitzacions virtuals o proveïdors de recursos. En aquest treball, l'assignació de recursos es gestionada per models de mercat ja que proveeixen de mecanismes eficients i simples per gestionar l'assignació de recursos. Aquesta proposta aporta noves oportunitats als usuaris finals d'Internet per tal d'establir el seu negoci a la xarxa mitjançant la venda dels seus recursos no usats. A més a més aquest treball óona l'oportunitat a comunitats petites a créixer i a portar capacitats de super-computació als usuaris finals d'Internet. En segon lloc, la tesis descriu semànticament els recursos computacionals per tal de construir una base comuna de coneixement sobre els recursos d'Internet. La descripció semàntica dels recursos permet un enteniment comú de la naturalesa dels recursos, permetent així l'agrupació i agregació de diferents tipus de tecnologies mentre es mantenen la mateixa semàntica. Una base semàntica comuna permet que aplicacions i sistemes de gestió de recursos siguin independents de la naturalesa real dels recursos. En aquest treball considerem com a aspecte fonamental aïllar la gestió dels recursos de la seva naturalesa específica. La descripció semàntica a més permet el desenvolupament de especificacions genèriques que ens permeten definir els requeriments dels usuaris en sistemes d'assignació de recursos basats en mercats computacionals. Tercer, són presentades arquitectures que permeten l'expansió de recursos computacionals sota demanda en Sistemes Contributius. Aquestes arquitectures han estat especialment dissenyades per proveir de recursos computacionals mitjançant mercats a escenaris caracteritzats pel dinamisme, evolució i heterogeneïtat dels seus recursos. L'arquitectura aporta les principals funcionalitats orientades a l'assignació de recursos mitjançant subhastes i permet a més a més l'execució d'aquests mercats sota demanda. Finalment, es presenta un mecanisme de mercat adaptat a l'assignació de recursos computacionals. Aquesta contribució es motiva pel fet que no existeix fins avui cap mecanisme que permeti l'assignació eficient de recursos computacionals en que la seva única diferència és la unitat de temps en la que s'ofereixen. La tesis construeix un camí per assolir l'expansió de recursos computacionals de forma flexible i descentralitzada en comunitats on els recursos són compartits pels seus participants. Aquest camí es construeix mitjançant l'anàlisi dels escenaris d'aplicació, l'estudi i definició de models semàntics que permeten la descripció dels recursos computacionals, proposant també arquitectures flexibles i configurables que permeten aconseguir l'expansió dels recursos computacionals sota demanda i proposant mecanismes de mercat adaptats a tal escenari.La tesis presenta una aproximación a un conjunto de técnicas para permitir la extensión de las capacidades computacionales, bajo pedido, en comunidades formadas por usuarios de Internet que quieren agregar sus recursos para conseguir una finalidad común. Este tipo de comunidades, emergen como nuevas formas de organización gracias al incremento de las capacidades computacionales y lel ancho de banda. La computación voluntaria como la que se da en sistemas como Seti@home, Grids colaborativos como OurGrid o LaCOLLA, Ad-hoc y Grids de igual a igual como P-Grid o X-Grid de Apple, Grids Abiertos como los dirigidos por SORMA o Grid4All y muchas otras formas de computación Grid basadas en agrupación de recursos en forma de Organizaciones Virtuales son el objetivo de este trabajo. Estos sistemas se caracterizan por el propósito de sus participantes, es decir, llevar a cabo un objetivo común haciendo uso de la agregación de sus propios recursos. Los sistemas mencionados, contrariamente a los sistemas Grid de alto rendimiento computacional, son abiertos a nuevos participantes hecho que los convierte en escenarios impredecibles, dinámicos y donde los recursos pueden conectarse y desconectarse de forma espontánea. Mientras que el aspecto crítico de los Grids de altas prestaciones es el rendimiento computacional, la estabilidad y la disponibilidad son los aspectos más importantes en los sistemas dirigidos en este trabajo. La tesis homogeneiza los conceptos de los paradigmas anteriores bajo el nombre de Sistemas Contributivos, nombre que es usado a lo largo de la disertación para referirnos a sistemas en los cuales sus usuarios hacen contribución de sus recursos para que este sean usados de forma colectiva y así conseguir un objetivo común. La expansión de recursos en los Sistemas Contributivos es una funcionalidad requerida para aumentar las limitadas capacidades computacionales de los grupos colaborativos formados de forma ad-hoc. Sobre todo en momentos puntuales cuando los recursos necesarios son mayores que los disponibles en el grupo. Cuatro aspectos son conducidos a lo largo de la disertación. Primero, las propiedades y principales aplicaciones de los Sistemas Contributivos son identificadas, así como se motiva la necesidad de infraestructuras que permitan la expansión de recursos computacionales bajo demanda. Estas ideas van en la dirección de la Utility Computing, emergentes líneas de negocio de las principales compañías de las TIC. De este modo, la tesis propone la provisión de recursos computacionales bajo demanda a aquellas organizaciones que necesiten, mediante la agregación de recursos de los extremos de Internet, ya sean usuarios finales de la red, otras organizaciones virtuales o proveedores de recursos. En este trabajo, la asignación de recursos es gestionada por modelos de mercado puesto que proveen de mecanismos eficientes y simples para gestionar la asignación de recursos. Esta propuesta aporta nuevas oportunidades a los usuarios finales de Internet para establecer su negocio en la red mediante la venta de sus recursos no usados. Además este trabajo da la oportunidad a comunidades pequeñas a crecer y a traer capacidades de super-computación a los usuarios finales de Internet. En segundo lugar, la tesis describe semánticamente los recursos computacionales para construir una base común de conocimiento sobre los recursos de Internet. La descripción semántica de los recursos permite un entendimiento común de la naturaleza de los recursos, permitiendo así la agrupación y agregación de diferentes tipos de tecnologías mientras se mantienen la misma semántica. Una base semántica común permite que aplicaciones y sistemas de gestión de recursos sean independientes de la naturaleza real de los recursos. En este trabajo consideramos como aspecto fundamental aislar la gestión de los recursos de su naturaleza específica. La descripción semántica además permite el desarrollo de especificaciones genéricas que nos permiten definir los requerimientos de los usuarios en sistemas de asignación de recursos basados en mercados computacionales. Tercero, son presentadas arquitecturas que permiten la expansión de recursos computacionales bajo demanda en Sistemas Contributivos. Estas arquitecturas han sido especialmente diseñadas para proveer de recursos computacionales mediante mercados a escenarios caracterizados por el dinamismo, evolución y heterogeneidad de sus recursos. La arquitectura aporta las principales funcionalidades orientadas a la asignación de recursos mediante subastas y permite además la ejecución de estos mercados bajo demanda. Finalmente, se presenta un mecanismo de mercado adaptado a la asignación de recursos computacionales. Esta contribución se motiva por el hecho que no existe hasta hoy ningún mecanismo que permita la asignación eficiente de recursos computacionales en que su única diferencia es la unidad de tiempo en la que se ofrecen. La tesis construye un camino para lograr la expansión de recursos computacionales de forma flexible y descentralizada en comunidades donde los recursos son compartidos por sus participantes. Este camino se construye mediante el análisis de los escenarios de aplicación, el estudio y definición de modelos semánticos que permiten la descripción de los recursos computacionales, proponiendo también arquitecturas flexibles y configurables que permiten conseguir la expansión de los recursos computacionales bajo demanda y proponiendo mecanismos de mercado adaptados a tal escenario.The thesis presents an approach to on-demand capacity expansion in communities of Internet users that aggregate their resources to achieve a common objective. Such communities are emerging as forms of organisation taking advantage of an increasing broadband access and computational capacity. Volunteer computing such as SETI@home, Collab- orative Grids such as OurGrid and LaCOLLA, Ad-hoc and Peer-to-Peer Grids, such as P-Grid and the XGrid project from Apple, Open Grids such as those addressed by SORMA and Grid4All and many other approaches of Grid Computing based on Virtual Organisations are the focus of our work. These systems are characterised by the purpose of their participants, i.e. to achieve a common objective taking advantage of the aggregation of other resources. The cited systems, in contrast to high performance computing Grids, are open to new participants, which makes their behaviour unpredictable and dynamic, and resources are usually connected and disconnected spontaneously. While the critical aspect of high performance Grids is computational performance, stability and availability are the main issues for the systems addressed in this work. The thesis homogenises the concepts of those paradigms under the term Contributory System, which is used throughout the thesis to refer to the systems where users provide their resources to be used collectively to achieve a common objective. Resource expansion in Contributory Systems is required so as to increase the limited capacities of ad-hoc collaborative groups under unexpected load surges, temporary resource requirements or other policies defined by the objectives of the Virtual Organisation that they constitute. Four aspects are addressed by the dissertation. Firstly, it identifies the main properties and applications of Contributory Systems and motivates the need for infrastructures to enable on-demand resource expansion. This goes in the direction of Utility Computing trends which are main business lines for IT companies. Thus the thesis proposes the on-demand provision of idle resources from the extremes of the Internet, other Virtual Organisations or Resource Providers to those organisations that have resource needs. In this work, resource allocation is handled by market models which provide efficient while simple mechanisms to mediate the allocation of resources. This proposal enables new emerging opportunities to Internet users to make their business on the Internet by selling their idle resources. Besides, this brings the opportunity to small communities to grow and to bring super-computing capacities to Internet end-users. Secondly, the thesis describes semantically Computational Resources so as to build a common knowledge about the Internets resources. The semantic description enables a common understanding of the nature of resources, permitting the pooling and aggregation of distinct types of technologies while maintaining the same semantics. This makes applications and resource management frameworks independent of the real nature of the resources which we claim as a fundamental aspect to keep resource management independent of the dynamics and evolution of technology in computational environments, such as in Contributory Systems. A semantic description permits the development of generic specifications to provide bid and offer descriptions in computational markets. Thirdly, the architecture for on-demand resource expansion in Contributory Systems is presented. It has been designed to provide the main functionalities to on-demand provision of resources through markets to scenarios characterized by dynamism, evolution and heterogeneity. The architecture provides the main market oriented functionalities and enables dynamic and on-demand execution of market mechanisms. Finally, a specific Grid-oriented market mechanism is presented. The approach is motivated due to the unsuitability of current auctions to efficiently allocate time-differentiated resources (usually provided by many different resource providers) such as most of the re- sources in a Contributory System. The thesis builds a roadmap to achieve flexible and decentralized resource expansion in communities where resources are shared by their participants by analysing the main scenarios where it can be applied, providing the semantics and specification to enable the description of the users' requirements, proposing a flexible and configurable architecture to deal with on-demand resource expansion in Virtual Organisations and proposing an specific mechanism adapted to trade computational resources

Lean sensing: exploiting contextual information for most energy-efficient sensing

Cyber-physical technologies enable event-driven applications, which monitor in real-time the occurrence of certain inherently stochastic incidents. Those technologies are being widely deployed in cities around the world and one of their critical aspects is energy consumption, as they are mostly battery powered. The most representative examples of such applications today is smart parking. Since parking sensors are devoted to detect parking events in almost-real time, strategies like data aggregation are not well suited to optimize energy consumption. Furthermore, data compression is pointless, as events are essentially binary entities. Therefore, this paper introduces the concept of Lean Sensing, which enables the relaxation of sensing accuracy at the benefit of improved operational costs. To this end, this paper departs from the concept of instantaneous randomness and it explores the correlation structure that emerges from it in complex systems. Then, it examines the use of this system-wide aggregated contextual information to optimize power consumption, thus going in the opposite way; from the system-level representation to individual device power consumption. The discussed techniques include customizing the data acquisition to temporal correlations (i.e, to adapt sensor behavior to the expected activity) and inferring the system-state from incomplete information based on spatial correlations. These techniques are applied to real-world smart-parking application deployments, aiming to evaluate the impact that a number of system-level optimization strategies have on devices power consumption

Lean sensing: exploiting contextual information for most energy-efficient sensing

Cyber-physical technologies enable event-driven applications, which monitor in real-time the occurrence of certain inherently stochastic incidents. Those technologies are being widely deployed in cities around the world and one of their critical aspects is energy consumption, as they are mostly battery powered. The most representative examples of such applications today is smart parking. Since parking sensors are devoted to detect parking events in almost-real time, strategies like data aggregation are not well suited to optimize energy consumption. Furthermore, data compression is pointless, as events are essentially binary entities. Therefore, this paper introduces the concept of Lean Sensing, which enables the relaxation of sensing accuracy at the benefit of improved operational costs. To this end, this paper departs from the concept of instantaneous randomness and it explores the correlation structure that emerges from it in complex systems. Then, it examines the use of this system-wide aggregated contextual information to optimize power consumption, thus going in the opposite way; from the system-level representation to individual device power consumption. The discussed techniques include customizing the data acquisition to temporal correlations (i.e, to adapt sensor behavior to the expected activity) and inferring the system-state from incomplete information based on spatial correlations. These techniques are applied to real-world smart-parking application deployments, aiming to evaluate the impact that a number of system-level optimization strategies have on devices power consumption

Accurate clock discipline for long-term synchronization intervals

Efficient spectrum usage and optimized energy consumption directly depend on the radio duty cycle of the communicating devices. Tight synchronization of communicating nodes enables optimal orchestration of the access to the medium as nodes turn their radios on precisely when needed. Achieving synchronization, however, requires a reference time source to synchronize with and a periodic discipline mechanism to cope with the inherent clock drift. Packet-based synchronization or external time sources, such as GPS, are usually required to achieve that goal. When low-power operation is combined with bandwidth limitations and scale, as in the case of low power wide area networks, the overhead of such approaches is not affordable and advanced clock discipline mechanisms are required. In this paper, we propose a novel adaptive mechanism to discipline clocks in order to guarantee a 1ppm drift with minimal communication overhead

Optimal rate allocation in cluster-tree WSNs

In this paper, we propose a solution to the problem of guaranteed time slot allocation in cluster-tree WSNs. Our design uses the so-called Network Utility Maximization (NUM) approach as far as we aim to provide a fair distribution of the available resources. From the point of view of implementation, we extend here the authors' proposed Coupled Decompositions Method (CDM) in order to compute the NUM problem inside the cluster tree topology and we prove the optimality of this new extended version of the method. As a result, we obtain a distributed solution that reduces the total amount of signalling information in the network up to a factor of 500 with respect to the classical techniques, that is, primal and dual decomposition. This is possible because the CDM finds the optimal solution with a small number of iterations. Furthermore, when we compare our solution to the standard-proposed First Come First Serve (FCFS) policy, we realize that FCFS becomes pretty unfair as the traffic load in the network increases and thus, a fair allocation of resources can be considered whenever the price to pay in terms of signaling and computational complexity is controlled

Accurate clock discipline for long-term synchronization intervals

Efficient spectrum usage and optimized energy consumption directly depend on the radio duty cycle of the communicating devices. Tight synchronization of communicating nodes enables optimal orchestration of the access to the medium as nodes turn their radios on precisely when needed. Achieving synchronization, however, requires a reference time source to synchronize with and a periodic discipline mechanism to cope with the inherent clock drift. Packet-based synchronization or external time sources, such as GPS, are usually required to achieve that goal. When low-power operation is combined with bandwidth limitations and scale, as in the case of low power wide area networks, the overhead of such approaches is not affordable and advanced clock discipline mechanisms are required. In this paper, we propose a novel adaptive mechanism to discipline clocks in order to guarantee a 1ppm drift with minimal communication overhead

Evaluation of IEEE802.15.4g for environmental observations

IEEE802.15.4g is a low-power wireless standard initially designed for Smart Utility Networks, i.e., for connecting smart meters. IEEE802.15.4g operates at sub-GHz frequencies to offer 2.3× longer communication range compared to its 2.4 GHz counterpart. Although the standard offers 3 PHYs (Frequncy Shift Keying, Orthogonal Frequency Division Multiplexing and Offset-Quadrature Phase Shift Keying) with numerous configurations, 2-FSK at 50 kbps is the mandatory and most prevalent radio setting used. This article looks at whether IEEE802.15.4g can be used to provide connectivity for outdoor deployments. We conduct range measurements using the totality of the standard (all modulations with all further parametrization) in the 863.870 MHz band, within four scenarios which we believe cover most low-power wireless outdoor applications: line of sight, smart agriculture, urban canyon, and smart metering. We show that there are radio settings that outperform the 2-FSK at 50 kbps base setting in terms of range, throughput and reliability. Results show that highly reliable communications with data rates up to 800 kbps can be achieved in urban environments at 540 m between nodes, and the longest useful radio link is obtained at 779 m. We discuss how IEEE802.15.4g can be used for outdoor operation, and reduce the number of repeater nodes that need to be placed compared to a 2.4 GHz solution

Evaluation of IEEE802.15.4g for environmental observations

IEEE802.15.4g is a low-power wireless standard initially designed for Smart Utility Networks, i.e., for connecting smart meters. IEEE802.15.4g operates at sub-GHz frequencies to offer 2.3× longer communication range compared to its 2.4 GHz counterpart. Although the standard offers 3 PHYs (Frequncy Shift Keying, Orthogonal Frequency Division Multiplexing and Offset-Quadrature Phase Shift Keying) with numerous configurations, 2-FSK at 50 kbps is the mandatory and most prevalent radio setting used. This article looks at whether IEEE802.15.4g can be used to provide connectivity for outdoor deployments. We conduct range measurements using the totality of the standard (all modulations with all further parametrization) in the 863.870 MHz band, within four scenarios which we believe cover most low-power wireless outdoor applications: line of sight, smart agriculture, urban canyon, and smart metering. We show that there are radio settings that outperform the 2-FSK at 50 kbps base setting in terms of range, throughput and reliability. Results show that highly reliable communications with data rates up to 800 kbps can be achieved in urban environments at 540 m between nodes, and the longest useful radio link is obtained at 779 m. We discuss how IEEE802.15.4g can be used for outdoor operation, and reduce the number of repeater nodes that need to be placed compared to a 2.4 GHz solution

jxSensor: a sensor network integration layer for JXTA

Nowadays, Wireless Sensor Networks (WSN) are already a very important data source to obtain data about the environment. Thus, they are key to the creation of Cyber-Physical Systems (CPS). Given the popularity of P2P middlewares as a means to efficiently process information and distribute services, being able to integrate them to WSN¿s is an interesting proposal. JXTA is a widely used P2P middleware that allows peers to easily exchange information, heavily relying on its main architectural highlight, the capability to organize peers with common interests into peer groups. However, right now, approaches to integrate WSNs to a JXTA network seldom take advantage of peer groups. For this reason, in this paper we present jxSensor, an integration layer for sensor motes which facilitates the deployment of CPS¿s under this architecture. This integration has been done taking into account JXTA¿s idiosyncrasies and proposing novel ideas, such as the Virtual Peer, a group of sensors that acts as a single entity within the peer group context

Distributed PID-based scheduling for 6TiSCH networks

Industrial low-power networks are becoming the nexus of operational technologies and the Internet thanks to the standardization of networking layer interfaces. One of the main promoters of this shift is the IETF 6TiSCH WG, which addresses network management and IP integration of time synchronized channel hopping (TSCH) networks as those developed by the IEEE802.15.4 TG. The 6TiSCH WG is defining the operational interface and mechanism by which the network schedule can be distributed amongst the devices in the network. This operational sublayer, called 6top, supports distributed scheduling and enables implementers to define the scheduling policy, only standardizing the distribution mechanism. This letter proposes a novel distributed scheduling policy based on the well-known industrial control paradigm referred as proportional, integral, and derivative (PID) control. The proposed technique is completely decentralized, enabling each node to determine the number of cells to schedule to one another, according to its traffic demand. The mechanism is reactive to sudden or bursty traffic patterns, while staying conservative in over-provisioning cells