ROAR: increasing the flexibility and performance of distributed search

Search engines are a fundamental building block of the web. Be they general purpose web search engines, product search engines for online catalogues or people search in online networks, search engines provide easy access to a huge amount of information. To cope with large amounts of information, search engines use many distributed servers to perform their functionality. For instance, to search the web quickly, search engines partition the web index over many machines, and consult every partition when answering a query. To increase throughput, replicas are added for each of these machines. The key parameter of these search algorithms is the trade-off between replication and partitioning: increasing the partitioning level typically improves query completion time since more servers handle the query. However, partitioning too much also has drawbacks: startup costs for each sub-query are not negligible, and will decrease total throughput. Finding the right operating point and adapting to it can significantly improve performance and reduce costs. In this thesis we propose that the tradeoff between partitioning and replication should be easily configurable. To this end we introduce Rendezvous On a Ring (ROAR), a novel distributed algorithm that enables on-the-fly re-configuration of the partitioning level. ROAR can add and remove servers without stopping the system, cope with server failures, and provide good load-balancing even with a heterogeneous server pool. We experimentally show that it is possible to dynamically adjust the partitioning level to cope with different loads while meeting target query delays, and in doing so the system can reduce its power consumption significantly. To test ROAR we introduce Privacy Preserving Search: a particular search application that allows users to store encrypted data online while being able to easily search that data. Our contributions include novel protocols that allow PPS for numeric values, as well as a proof of concept implementation of PPS running on top of ROAR and allowing users to match as many as 5 million files in well under 1s

Monitoring and analysis system for performance troubleshooting in data centers

It was not long ago. On Christmas Eve 2012, a war of troubleshooting began in Amazon data centers. It started at 12:24 PM, with an mistaken deletion of the state data of Amazon Elastic Load Balancing Service (ELB for short), which was not realized at that time. The mistake first led to a local issue that a small number of ELB service APIs were affected. In about six minutes, it evolved into a critical one that EC2 customers were significantly affected. One example was that Netflix, which was using hundreds of Amazon ELB services, was experiencing an extensive streaming service outage when many customers could not watch TV shows or movies on Christmas Eve. It took Amazon engineers 5 hours 42 minutes to find the root cause, the mistaken deletion, and another 15 hours and 32 minutes to fully recover the ELB service. The war ended at 8:15 AM the next day and brought the performance troubleshooting in data centers to world’s attention. As shown in this Amazon ELB case.Troubleshooting runtime performance issues is crucial in time-sensitive multi-tier cloud services because of their stringent end-to-end timing requirements, but it is also notoriously difficult and time consuming. To address the troubleshooting challenge, this dissertation proposes VScope, a flexible monitoring and analysis system for online troubleshooting in data centers. VScope provides primitive operations which data center operators can use to troubleshoot various performance issues. Each operation is essentially a series of monitoring and analysis functions executed on an overlay network. We design a novel software architecture for VScope so that the overlay networks can be generated, executed and terminated automatically, on-demand. From the troubleshooting side, we design novel anomaly detection algorithms and implement them in VScope. By running anomaly detection algorithms in VScope, data center operators are notified when performance anomalies happen. We also design a graph-based guidance approach, called VFocus, which tracks the interactions among hardware and software components in data centers. VFocus provides primitive operations by which operators can analyze the interactions to find out which components are relevant to the performance issue. VScope’s capabilities and performance are evaluated on a testbed with over 1000 virtual machines (VMs). Experimental results show that the VScope runtime negligibly perturbs system and application performance, and requires mere seconds to deploy monitoring and analytics functions on over 1000 nodes. This demonstrates VScope’s ability to support fast operation and online queries against a comprehensive set of application to system/platform level metrics, and a variety of representative analytics functions. When supporting algorithms with high computation complexity, VScope serves as a ‘thin layer’ that occupies no more than 5% of their total latency. Further, by using VFocus, VScope can locate problematic VMs that cannot be found via solely application-level monitoring, and in one of the use cases explored in the dissertation, it operates with levels of perturbation of over 400% less than what is seen for brute-force and most sampling-based approaches. We also validate VFocus with real-world data center traces. The experimental results show that VFocus has troubleshooting accuracy of 83% on average.Ph.D

Yhteisöllinen energiatehokkuus mobiililaitteilla

We have created a mobile energy measurement application and gathered energy measurement data from over 725,000 devices, running over 300,000 applications, in heterogeneous environments, and constructed models of what is normal in each context for each application. We have used this data to find energy abnormalities in the wild, and provide users of our application advice on how to deal with them. These abnormalities cannot be discovered in laboratory conditions due to the rich interaction of the smartphone and its operating environment. Employing a collaborative mobile energy awareness application with thousands of users allows us to gather a large amount of data in a short time. Such a large and diverse dataset has helped us answer many research questions. Our work is the first collaborative approach in the area of mobile energy debugging. Information received from each device running our application improves the advice given to other users running the same applications. The author has developed a context data gathering hub for smartphones, discovered the need for a common API that unifies network connectivity, energy awareness, and user experience, and investigated the impact of mobile collaborative energy awareness applications, to find previously unknown energy bugs on smartphones, and to improve users' knowledge of smartphone energy behavior.Viime vuosien aikana älypuhelinten laitteistot ovat kehittyneet entistä tehokkaammiksi, mutta akkuteknologia ei ole kehittynyt yhtä nopeasti. Tämä on synnyttänyt tarpeen tehostaa sekä laitteiston että ohjelmiston energiatehokkuutta. Älypuhelimen energiatehokkuuden optimointi on haastavaa, koska toimintaympäristö on moninainen ja käsittää paitsi laitteiston ja sen asetukset, niin myös sovellukset, jotka käyttävät laitteiston toimintoja. Tässä väitöstyössä on keskitytty mobiililaitteiden energiaongelmien ja poikkeamien löytämiseen ja niiden korjaamiseen. Väitöskirja käsittelee yhteisöllisen metodin käyttöä energiankulutukseen liittyvien epätehokkuuksien löytämisessä ja korjaamisessa mobiililaitteilla. Tätä metodia on ensimmäistä kertaa sovellettu mobiililaitteille väitöstyöhön liittyvässä Carat-projektissa. Projektissa on luotu energianmittaussovellus mobiililaitteille ja kerätty energiamittauksia yli 725 000 laitteelta ja 300 000 sovelluksesta monipuolisissa ympäristöissä. Näiden pohjalta on tehty malleja sovellusten normaalista energiankulutuksesta eri konteksteissa. Tietojen ja mallien avulla on löydetty energiapoikkeavuuksia tavallisessa käytössä olevilta laitteilta ja annettu sovelluksen käyttäjille neuvoja poikkeavuuksien korjaamiseen. Väitöstyön aikana kerätty suurikokoinen ja monipuolinen aineisto on auttanut vastaamaan moniin kysymyksiin koskien älypuhelinten energiankulutusta arkikäytössä. Kaikkia poikkeavuuksia ei voida löytää laboratorio-olosuhteissa, sillä mobiililaitteen ympäristö vaikuttaa vahvasti sen toimintaan. Esitetty menetelmä on ensimmäinen, joka soveltaa yhteisöllistä lähestymistapaa mobiililaitteiden energiaongelmien löytämiseen. Kirjoittaja on kehittänyt kontekstitietojen keräysratkaisun älypuhelimille. Hän on huomannut tarpeen järjestelmälle, joka yhdistää mobiililaitteen tilanteen, käytön, energiatehokkuuden ja käyttäjäkokemuksen. Työssä on kehitetty uusi menetelmä energiapoikkeamien analyysiin yhteisöllisesti kerättyjen mittausten perusteella sekä tutkittu energiatehokkuussovellusten vaikutusta eri mobiililaitteilla. Näiden avulla on löydetty ennen tuntemattomia energiaongelmia älypuhelimista ja parannettu käyttäjien ymmärrystä älypuhelinten energiakäyttäytymisestä

Commons-oriented information syntheses : a model for user-driven design and creation activities

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2008.Includes bibliographical references (p. 173-195).The phenomenon of user-driven creation activities has recently emerged and is quickly expanding, especially on the Web. A growing number of people participate in online activities, where they generate content by themselves, freely share their creations, and combine one another's creations in order to synthesize new material. Similar activities also occur in the area of product development, as people design products for themselves and share their designs for others to reuse or build upon. The phenomenon shows that under some special circumstances, typically passive users can become active creators. Also, under such circumstances, creation activities are not just isolated do-it-yourself activities of an individual; instead, people build on one another's creations and further share their own. Recognizing the positive potential of user-driven design, this work endeavored to understand the underlying drivers of open source creation and essential environmental elements. The most important element is the commons, or shared resources, of the communities where the activities take place. A model of commons-oriented information syntheses was formulated. The model provides a unifying description of user-driven creation activities and, more importantly, serves as a general prescription for how to construct a circumstance to recreate the phenomenon for desired applications. Key aspects of the model include: that, in this particular form of information synthesis, the processes of information creating, participating in a community, and sharing of information take place integrally; that the three processes revolve around the commons; and that people consider the prospective benefits and costs of all three processes when they decide on whether or not to engage in a synthesis activity. This understanding can be employed to build circumstances under which the phenomenon can be recreated.(cont.) The ability to recreate the phenomenon of user-driven creation activities can be beneficial in many areas, including design and knowledge transfer. In the design area, the understanding can be used to build an environment that induces and fosters open-source design. With such environment, people can design things for themselves by reusing, remixing, and building on designs shared by others. They can also freely make available their own designs, which can continue to evolve through a series of building-on processes by others. In the knowledge transfer area, the understanding can be a key to constructing an environment that not only supports transfer of knowledge, but also enables people to further generate knowledge by building on what they receive, particularly when the transferred knowledge is in meta-forms such simulation models. Possible applications include: engineering education (where students can connect models of fundamental topics in various ways to create simulations of complex systems and learn from them), sustainable development (where citizens can integrate models of potential environmental remedies to figure out which solution mix will be the most effective in their situations), and academic communities (where researchers can share and allow their colleagues to reuse or build on simulation models from which the results they publish in journal papers are derived). A prototypical online environment was designed and implemented, employing the essential elements outlined in the model. Hosting a commons of environmental and energy-related simulation models, the environment functions as an open-source design environment for alternative energy systems and a public platform for generative transfers of environmental knowledge. Anyone can freely access the commons, build on them to synthesize new simulation models, and further share their synthesized models as new commons.by Sittha Sukkasi.Ph.D

Efficient Passive Clustering and Gateways selection MANETs

Passive clustering does not employ control packets to collect topological information in ad hoc networks. In our proposal, we avoid making frequent changes in cluster architecture due to repeated election and re-election of cluster heads and gateways. Our primary objective has been to make Passive Clustering more practical by employing optimal number of gateways and reduce the number of rebroadcast packets

Energieeffizienz in Büroumgebungen

The increasing cost of energy and the worldwide desire to reduce CO2 emissions has raised concern about the energy efficiency of information and communication technology. Whilst research has focused on data centres recently, this thesis identifies office computing environments as significant consumers of energy. Office computing environments offer great potential for energy savings: On one hand, such environments consist of a large number of hosts. On the other hand, these hosts often remain turned on 24~hours per day while being underutilised or even idle. This thesis analyzes the energy consumption within office computing environments and suggests an energy-efficient virtualized office environment. The office environment is virtualized to achieve flexible virtualized office resources that enable an energy-based resource management. This resource management stops idle services and idle hosts from consuming resources within the office and consolidates utilised office services on office hosts. This increases the utilisation of some hosts while other hosts are turned off to save energy. The suggested architecture is based on a decentralized approach that can be applied to all kinds of office computing environments, even if no centralized data centre infrastructure is available. The thesis develops the architecture of the virtualized office environment together with an energy consumption model that is able to estimate the energy consumption of hosts and network within office environments. The model enables the energy-related comparison of ordinary and virtualized office environments, considering the energy-efficient management of services. Furthermore, this thesis evaluates energy efficiency and overhead of the suggested approach. First, it theoretically proves the energy efficiency of the virtualized office environment with respect to the energy consumption model. Second, it uses Markov processes to evaluate the impact of user behaviour on the suggested architecture. Finally, the thesis develops a discrete-event simulation that enables the simulation and evaluation of office computing environments with respect to varying virtualization approaches, resource management parameters, user behaviour, and office equipment. The evaluation shows that the virtualized office environment saves more than half of the energy consumption within office computing environments, depending on user behaviour and office equipment.Die steigenden Kosten von Energie und die weltweiten Bemühungen CO2-Emmissionen zu reduzieren, führt aktuell zu einer intensiven Untersuchung der Energieeffizienz von Informations- und Kommunikationstechnologien. Während ein großer Teil der aktuellen Forschung sich auf Rechenzentren fokussiert, betrachtet diese Arbeit Büroumgebungen mit ihren Rechnern und dem verbindenden Netzwerk. Eine energieeffiziente Architektur wird vorgeschlagen, die auf die Virtualisierung und Konsolidierung von Diensten setzt, ohne auf zentralisierte Rechenzentrumshardware oder Thin Clients angewiesen zu sein

An Energy-Efficient and Reliable Data Transmission Scheme for Transmitter-based Energy Harvesting Networks

Energy harvesting technology has been studied to overcome a limited power resource problem for a sensor network. This paper proposes a new data transmission period control and reliable data transmission algorithm for energy harvesting based sensor networks. Although previous studies proposed a communication protocol for energy harvesting based sensor networks, it still needs additional discussion. Proposed algorithm control a data transmission period and the number of data transmission dynamically based on environment information. Through this, energy consumption is reduced and transmission reliability is improved. The simulation result shows that the proposed algorithm is more efficient when compared with previous energy harvesting based communication standard, Enocean in terms of transmission success rate and residual energy.This research was supported by Basic Science Research Program through the National Research Foundation by Korea (NRF) funded by the Ministry of Education, Science and Technology(2012R1A1A3012227)

Intelligent Sensor Networks

In the last decade, wireless or wired sensor networks have attracted much attention. However, most designs target general sensor network issues including protocol stack (routing, MAC, etc.) and security issues. This book focuses on the close integration of sensing, networking, and smart signal processing via machine learning. Based on their world-class research, the authors present the fundamentals of intelligent sensor networks. They cover sensing and sampling, distributed signal processing, and intelligent signal learning. In addition, they present cutting-edge research results from leading experts

Proceedings of the 7th International Conference EEDAL 2013 Energy Efficiency in Domestic Appliances and Lighting

This book contains the papers presented at the seventh international conference on Energy Efficiency in Domestic Appliances and Lighting. EEDAL'2013 was organised in Coimbra, Portugal in September 2013. This major international conference, which was previously been staged in Florence 1997, Naples 2000, Turin 2003, London 2006, B2e0r0l9in, Copenhagen 2011 has been very successful in attracting an international community of stakeholders dealing with residential appliances, equipment, metering liagnhdti ng (including manufacturers, retailers, consumers, governments, international organisations aangde ncies, academia and experts) to discuss the progress achieved in technologies, behavioural aspects and poliacineds , the strategies that need to be implemented to further progress this important work. Potential readers who may benefit from this book include researchers, engineers, policymakers, and all those who can influence the design, selection, application, and operation of electrical appliances and lighting.JRC.F.7-Renewables and Energy Efficienc