Non-Monotonic Snapshot Isolation: scalable and strong consistency for geo-replicated transactional systems

International audienceModern cloud systems are geo-replicated to improve application latency and availability. Transactional consistency is essential for application developers; however, the corresponding concurrency control and commitment protocols are costly in a geo-replicated setting. To minimize this cost, we identify the following essential scalability properties: (i) only replicas updated by a transaction T make steps to execute T; (ii) a read-only transaction never waits for concurrent transactions and always commits; (iii) a transaction may read object versions committed after it started; and (iv) two transactions synchronize with each other only if their writes conflict. We present Non-Monotonic Snapshot Isolation (NMSI), the first strong consistency criterion to allow implementations with all four properties. We also present a practical implementation of NMSI called Jessy, which we compare experimentally against a number of well-known criteria. Our measurements show that the latency and throughput of NMSI are comparable to the weakest criterion, read-committed, and between two to fourteen times faster than well-known strong consistencies

Towards Operator-less Data Centers Through Data-Driven, Predictive, Proactive Autonomics

Continued reliance on human operators for managing data centers is a major impediment for them from ever reaching extreme dimensions. Large computer systems in general, and data centers in particular, will ultimately be managed using predictive computational and executable models obtained through data-science tools, and at that point, the intervention of humans will be limited to setting high-level goals and policies rather than performing low-level operations. Data-driven autonomics, where management and control are based on holistic predictive models that are built and updated using live data, opens one possible path towards limiting the role of operators in data centers. In this paper, we present a data-science study of a public Google dataset collected in a 12K-node cluster with the goal of building and evaluating predictive models for node failures. Our results support the practicality of a data-driven approach by showing the effectiveness of predictive models based on data found in typical data center logs. We use BigQuery, the big data SQL platform from the Google Cloud suite, to process massive amounts of data and generate a rich feature set characterizing node state over time. We describe how an ensemble classifier can be built out of many Random Forest classifiers each trained on these features, to predict if nodes will fail in a future 24-hour window. Our evaluation reveals that if we limit false positive rates to 5%, we can achieve true positive rates between 27% and 88% with precision varying between 50% and 72%.This level of performance allows us to recover large fraction of jobs' executions (by redirecting them to other nodes when a failure of the present node is predicted) that would otherwise have been wasted due to failures. [...

Safety Kernel for cooperative sensor-based systems

Tese de mestrado em Segurança Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2013Os sistemas críticos, usados em indústrias como a aeroespacial, aeronáutica ou automóvel, requerem novas soluções tecnológicas para responder à constante procura por novas funcionalidades que respondam aos novos desafios do futuro, tornando-se cada vez mais complexos. Estes sistemas necessitam, contudo, de respeitar elevados e rígidos requisitos, não só em termos de segurança na operação e fiabilidade, mas também em termos de requisitos de tamanho, peso e consumo energético. Arquiteturas tradicionais usadas no desenho deste tipo de sistemas críticos baseiam a segurança na operação possibilidade de provar, em tempo de desenvolvimento, que o sistema garante a previsibilidade necessária. Contudo, o aparecimento de novas tecnologias acarreta um aumento na complexidade das aplicações usadas, o que torna o objetivo de provar a sua fiabilidade uma tarefa árdua ou mesmo impossível, limitando as funcionalidades passíveis de serem integradas nestes sistemas. Por exemplo, o aparecimento de comunicações sem fios abriu um novo mundo de oportunidades: a mesma poderia permitir um conjunto de veículos comunicar e cooperar mutuamente para atingir um objetivo comum. Contudo, a incerteza que caracteriza este tipo de comunicações tem travado o desenvolvimento de aplicações passiveis de ser usados por sistemas críticos. Nesta tese, propomos uma arquitetura híbrida, constituída por componentes simples e previsíveis que coexistem com componentes complexos e imprevisíveis sem que isso, sem que essa coexistência ponha em causa as garantias de segurança na operação. A possibilidade de incluir novas aplicações, que façam uso de novas tecnologias, abre portas à introdução de novas funcionalidades em sistemas críticos, permitindo melhorar a performance e serviço prestado pelos sistemas atualmente existentes. A nossa arquitetura assenta num componente chamado Núcleo de Segurança (Safety Kernel), que tem como tarefa a monitorização dos requisitos de segurança e a gestão da configuração do sistema, assegurando-se que este se adapta às limitações observadas e que podem por em causa a segurança do sistema, evitando assim possíveis acidentes. Este documento descreve a arquitetura deste componente bem como a integração e interação do mesmo na arquitetura do sistema, apresentando a implementação de um protótipo do mesmo na arquitetura AIR - uma arquitetura baseada no conceito de compartimentação no espaço e tempo (CET) desenvolvida para sistemas aeroespaciais.Future safety-critical systems, used in, for example, the aerospacial, aeronautic and automotive industries, call for innovative computing architectures, with increased complexity. These systems must still cope with strict requirements, not only in terms of safety and reliability, but also in terms of size, weight and power consumption (SWaP). Traditional approaches used in the design of such critical systems, rely on proving and guaranteeing, at design time, the safety and predictability of their applications. However, with the emergence of new technological solutions and the increase of the complexity of applications, it gets harder or even infeasible to prove their safety by design, limiting the scope and possible features to include in such systems. For instance, the use of wireless communications opens a new world of possibilities: it may be used to develop smart vehicles that cooperate with each other to achieve some common goal. However, due to its uncertainty, the development of such applications for safety-critical systems turns out to be a challenging task. In this thesis, we propose a hybrid architecture, in which simple and predictable components coexist with complex and unpredictable ones, without compromising safety, despite the unavoidable uncertainty. The inclusion of complex components into safetycritical systems allows the emergence of new applications that provide new features or that improve the existing ones. Furthermore, we want to deal with the uncertainty that characterizes wireless communications and provide mechanisms which allow systems to cooperate with each other in a safe way. We rely on a component called Safety Kernel, in charge of monitoring and managing the runtime configuration of the system, forcing it to adapt to faults and runtime constraints in order to avoid hazardous situations. We describe the architecture and role of such Safety Kernel, and how they interact with other components in the system architecture, including the functional components of the control system. Finally we present a prototype implementation of such Safety Kernel over AIR, an architecture based on the concept of Time- and Space Partitioning (TSP) developed for aerospace systems

Area-wide Integrated Pest Management

Over 98% of sprayed insecticides and 95% of herbicides reach a destination other than their target species, including non-target species, air, water and soil. The extensive reliance on insecticide use reduces biodiversity, contributes to pollinator decline, destroys habitat, and threatens endangered species. This book offers a more effective application of the Integrated Pest Management (IPM) approach, on an area-wide (AW) or population-wide (AW-IPM) basis, which aims at the management of the total population of a pest, involving a coordinated effort over often larger areas. For major livestock pests, vectors of human diseases and pests of high-value crops with low pest tolerance, there are compelling economic reasons for participating in AW-IPM. This new textbook attempts to address various fundamental components of AW-IPM, e.g. the importance of relevant problem-solving research, the need for planning and essential baseline data collection, the significance of integrating adequate tools for appropriate control strategies, and the value of pilot trials, etc. With chapters authored by 184 experts from more than 31 countries, the book includes many technical advances in the areas of genetics, molecular biology, microbiology, resistance management, and social sciences that facilitate the planning and implementing of area-wide strategies. The book is essential reading for the academic and applied research community as well as national and regional government plant and human/animal health authorities with responsibility for protecting plant and human/animal health

Area-wide Integrated Pest Management

Extensive reliance on insecticides reduces biodiversity, contributes to pollinator decline, destroys habitat and threatens endangered species. This book offers a more effective application of the Integrated Pest Management (IPM) approach, on an area-wide (AW) or population-wide (AW-IPM) basis. It addresses the importance of problem-solving research, planning and baseline data collection, integrating tools for appropriate control strategies, and pilot trials. The 48 chapters authored by 184 experts cover advances in genetics, molecular biology, biological control, resistance management, modelling, automated surveillance and unmanned aerial release systems

3D printing-as-a-service for collaborative engineering

3D printing or Additive Manufacturing (AM) are utilised as umbrella terms to denote a variety of technologies to manufacture or create a physical object based on a digital model. Commonly, these technologies create the objects by adding, fusing or melting a raw material in a layer-wise fashion. Apart from the 3D printer itself, no specialised tools are required to create almost any shape or form imaginable and designable. The possibilities of these technologies of these technologies are plentiful and cover the ability to manufacture every object, rapidly, locally and cost-efficiently without wasted resources and material. Objects can be created to specific forms to perform as perfectly fitting functions without consideration of the assembly process. To further the advance the availability and applicability of 3D printing, this thesis identifies the problems that currently exist and attempts to solve them. During the 3D printing process, data (i. e., files) must be converted from their original representation, e. g., CAD file, to the machine instructions for a specific 3D printer. During this process, information is lost, and other information is added. Traceability is lacking in 3D printing. The actual 3D printing can require a long period of time to complete, during which errors can occur. In 3D printing, these errors are often non-recoverable or reversible, which results in wasted material and time. In addition to the lack of closed-loop control systems for 3D printers, careful planning and preparation are required to avoid these costly misprints. 3D printers are usually located remotely from users, due to health and safety considerations, special placement requirements or out of comfort. Remotely placed equipment is impractical to monitor in person; however, such monitoring is essential. Especially considering the proneness of 3D printing to errors and the implications of this as described previously. Utilisation of 3D printers is an issue, especially with expensive 3D printers. As there are a number of differing 3D printing technologies available, having the required 3D printer, might be problematic. 3D printers are equipped with a variety of interfaces, depending on the make and model. These differing interfaces, both hard- and software, hinder the integration of different 3D printers into consistent systems. There exists no proper and complete ontology or resource description schema or mechanism that covers all the different 3D printing technologies. Such a resource description mechanism is essential for the automated scheduling in services or systems. In 3D printing services the selection and matching of appropriate and suitable 3D printers is essential, as not all 3D printing technologies are able to perform on all materials or are able to create certain object features, such as thin walls or hollow forms. The need for companies to sell digital models for AM will increase in scenarios where replacement or customised parts are 3D printed by consumers at home or in local manufacturing centres. Furthermore, requirements to safeguard these digital models will increase to avoid a repetition of the problems from the music industry, e. g., Napster. Replication and ‘theft’ of these models are uncontrollable in the current situation. In a service oriented deployment, or in scenarios where the utilisation is high, estimations of the 3D printing time are required to be available. Common 3D printing time estimations are inaccurate, which hinder the application of scheduling. The complete and comprehensive understanding of the complexity of an object is discordant, especially in the domain of AM. This understanding is required to both support the design of objects for AM and match appropriate manufacturing resources to certain objects. Quality in AM and FDM have been incompletely researched. The quality in general is increased with maturity of the technology; however, research on the quality achievable with consumer-grade 3D printers is lacking. Furthermore, cost-sensitive measurement methods for quality assessment are expandable. This thesis presents the structured design and implementation of a 3D printing service with associated contributions that provide solutions to particular problems present in the AM domain. The 3D printing service is the overarching component of this thesis and provides the platform for the other contributions with the intention to establish an online, cloud-based 3D printing service for use in end-user and professional settings with a focus on collaboration and cooperation