Continuously Providing Approximate Results under Limited Resources: Load Shedding and Spilling in XML Streams

Because of the high volume and unpredictable arrival rates, stream processing systems may not always be able to keep up with the input data streams, resulting in buffer overflow and uncontrolled loss of data. To continuously supply online results, two alternate solutions to tackle this problem of unpredictable failures of such overloaded systems can be identified. One technique, called load shedding, drops some fractions of data from the input stream to reduce the memory and CPU requirements of the workload. However, dropping some portions of the input data means that the accuracy of the output is reduced since some data is lost. To produce eventually complete results, the second technique, called data spilling, pushes some fractions of data to persistent storage temporarily when the processing speed cannot keep up with the arrival rate. The processing of the disk resident data is then postponed until a later time when system resources become available. This dissertation explores these load reduction technologies in the context of XML stream systems. Load shedding in the specific context of XML streams poses several unique opportunities and challenges. Since XML data is hierarchical, subelements, extracted from different positions of the XML tree structure, may vary in their importance. Further, dropping different subelements may vary in their savings of storage and computation. Hence, unlike prior work in the literature that drops data completely or not at all, in this dissertation we introduce the notion of structure-oriented load shedding, meaning selectively some XML subelements are shed from the possibly complex XML objects in the XML stream. First we develop a preference model that enables users to specify the relative importance of preserving different subelements within the XML result structure. This transforms shedding into the problem of rewriting the user query into shed queries that return approximate answers with their utility as measured by the user preference model. Our optimizer finds the appropriate shed queries to maximize the output utility driven by our structure-based preference model under the limitation of available computation resources. The experimental results demonstrate that our proposed XML-specific shedding solution consistently achieves higher utility results compared to the existing relational shedding techniques. Second, we introduces structure-based spilling, a spilling technique customized for XML streams by considering the spilling of partial substructures of possibly complex XML elements. Several new challenges caused by structure-based spilling are addressed. When a path is spilled, multiple other paths may be affected. We categorize varying types of spilling side effects on the query caused by spilling. How to execute the reduced query to produce the correct runtime output is also studied. Three optimization strategies are developed to select the reduced query that maximizes the output quality. We also examine the clean-up stage to guarantee that an entire result set is eventually generated by producing supplementary results to complement the partial results output earlier. The experimental study demonstrates that our proposed solutions consistently achieve higher quality results compared to the state-of-the-art techniques. Third, we design an integrated framework that combines both shedding and spilling policies into one comprehensive methodology. Decisions on the choice of whether to shed or spill data may be affected by the application needs and data arrival patterns. For some input data, it may be worth to flush it to disk if a delayed output of its result will be important, while other data would best directly dropped from the system given that a delayed delivery of these results would no longer be meaningful to the application. Therefore we need sophisticated technologies capable of deploying both shedding and spilling techniques within one integrated strategy with the ability to deliver the most appropriate decision customers need for each specific circumstance. We propose a novel flexible framework for structure-based shed and spill approaches, applicable in any XML stream system. We propose a solution space that represents all the shed and spill candidates. An age-based quality model is proposed for evaluating the output quality for different reduced query and supplementary query pairs. We also propose a family of four optimization strategies, OptF, OptSmart, HiX and Fex. OptF and OptSmart are both guaranteed to identify an optimal solution of reduced and supplementary query pair, with OptSmart exhibiting significantly less overhead than OptF. HiX and Fex use heuristic-based approaches that are much more efficient than OptF and OptSmart

The Impact of Contextual Political Factors on Personnel, Rulemaking, and Partisanship

The context in which an institution operates structures the way political actors respond to it. Broadly, this dissertation explores these contextual variables. The first chapter provides an overview of the arguments I will make in the dissertation and the results I find. The second chapter considers political context as it relates to excepted political appointees. I argue that presidents utilize Schedule C appointees more frequently in ideologically proximate agencies and when ideological conflict in the Senate is high. I show some evidence for these arguments using an original OPM dataset on Schedule C appointees from 1998 through 2013. The third chapter shifts to a discussion of public participation on federal rulemaking activity. Looking at an original dataset of all regulations on regulations.gov from 2004-2016, I show preliminary evidence that several contextual political factors impact the number of comments agencies receive on their rules. Agency expertise requirements and rules with legal deadlines are negatively correlated with comments while congressional attention on the producing agency and more significant rules tend to receive greater public attention. In the fourth chapter, John Patty and I consider the presidential appointment decision. We present a theory of bureaucratic staffing which allows staffers to affect agency policy priorities. To our knowledge, it is the first paper to consider the impact of structural characteristics like agency productivity and policy breadth on the appointment decision. The fifth chapter, coauthored with Jon Rogowski, examines the relationship between partisanship and voting behavior in the antebellum Congress. We show that the effect of partisanship varied over time in conjunction with institutional changes. Partisanship was stronger in the House than the Senate and the relationship exhibited three distinct periods. The sixth chapter provides an overview of an original dataset of all final rules produced by agencies from 2000 through 2014. These data are unique in that previous studies utilize the Unified Agenda, an incomplete data source. I overview the purpose of these data, describe the variables, and show a number of interesting descriptive statistics

Model based test suite minimization using metaheuristics

Software testing is one of the most widely used methods for quality assurance and fault detection purposes. However, it is one of the most expensive, tedious and time consuming activities in software development life cycle. Code-based and specification-based testing has been going on for almost four decades. Model-based testing (MBT) is a relatively new approach to software testing where the software models as opposed to other artifacts (i.e. source code) are used as primary source of test cases. Models are simplified representation of a software system and are cheaper to execute than the original or deployed system. The main objective of the research presented in this thesis is the development of a framework for improving the efficiency and effectiveness of test suites generated from UML models. It focuses on three activities: transformation of Activity Diagram (AD) model into Colored Petri Net (CPN) model, generation and evaluation of AD based test suite and optimization of AD based test suite. Unified Modeling Language (UML) is a de facto standard for software system analysis and design. UML models can be categorized into structural and behavioral models. AD is a behavioral type of UML model and since major revision in UML version 2.x it has a new Petri Nets like semantics. It has wide application scope including embedded, workflow and web-service systems. For this reason this thesis concentrates on AD models. Informal semantics of UML generally and AD specially is a major challenge in the development of UML based verification and validation tools. One solution to this challenge is transforming a UML model into an executable formal model. In the thesis, a three step transformation methodology is proposed for resolving ambiguities in an AD model and then transforming it into a CPN representation which is a well known formal language with extensive tool support. Test case generation is one of the most critical and labor intensive activities in testing processes. The flow oriented semantic of AD suits modeling both sequential and concurrent systems. The thesis presented a novel technique to generate test cases from AD using a stochastic algorithm. In order to determine if the generated test suite is adequate, two test suite adequacy analysis techniques based on structural coverage and mutation have been proposed. In terms of structural coverage, two separate coverage criteria are also proposed to evaluate the adequacy of the test suite from both perspectives, sequential and concurrent. Mutation analysis is a fault-based technique to determine if the test suite is adequate for detecting particular types of faults. Four categories of mutation operators are defined to seed specific faults into the mutant model. Another focus of thesis is to improve the test suite efficiency without compromising its effectiveness. One way of achieving this is identifying and removing the redundant test cases. It has been shown that the test suite minimization by removing redundant test cases is a combinatorial optimization problem. An evolutionary computation based test suite minimization technique is developed to address the test suite minimization problem and its performance is empirically compared with other well known heuristic algorithms. Additionally, statistical analysis is performed to characterize the fitness landscape of test suite minimization problems. The proposed test suite minimization solution is extended to include multi-objective minimization. As the redundancy is contextual, different criteria and their combination can significantly change the solution test suite. Therefore, the last part of the thesis describes an investigation into multi-objective test suite minimization and optimization algorithms. The proposed framework is demonstrated and evaluated using prototype tools and case study models. Empirical results have shown that the techniques developed within the framework are effective in model based test suite generation and optimizatio

The Digital Classicist 2013

This edited volume collects together peer-reviewed papers that initially emanated from presentations at Digital Classicist seminars and conference panels. This wide-ranging volume showcases exemplary applications of digital scholarship to the ancient world and critically examines the many challenges and opportunities afforded by such research. The chapters included here demonstrate innovative approaches that drive forward the research interests of both humanists and technologists while showing that rigorous scholarship is as central to digital research as it is to mainstream classical studies. As with the earlier Digital Classicist publications, our aim is not to give a broad overview of the field of digital classics; rather, we present here a snapshot of some of the varied research of our members in order to engage with and contribute to the development of scholarship both in the fields of classical antiquity and Digital Humanities more broadly

The Digital Classicist 2013

This edited volume collects together peer-reviewed papers that initially emanated from presentations at Digital Classicist seminars and conference panels. This wide-ranging volume showcases exemplary applications of digital scholarship to the ancient world and critically examines the many challenges and opportunities afforded by such research. The chapters included here demonstrate innovative approaches that drive forward the research interests of both humanists and technologists while showing that rigorous scholarship is as central to digital research as it is to mainstream classical studies. As with the earlier Digital Classicist publications, our aim is not to give a broad overview of the field of digital classics; rather, we present here a snapshot of some of the varied research of our members in order to engage with and contribute to the development of scholarship both in the fields of classical antiquity and Digital Humanities more broadly

Budget pressures on Australian governments 2014

Unprecedented infrastructure spending by states and territories is largely responsible for a $106 billion decline in their finances since 2006, argues this report. Overview Australian government budgets are under pressure. Without tough decisions, they risk posting deficits of around 4½ per cent of GDP within 10 years. The problems have got worse since our first Budget Pressures report. We would be better off if we faced up to the tough problems sooner rather than later. We now need to find savings and tax increases of$70 billion a year. Over the economic cycle of boom and bust, balanced budgets are much better than the alternative. Persistent government deficits incur interest payments, and limit future borrowings, reducing flexibility in a crisis. They are also unfair: they require future taxpayers to pay for today’s spending. Despite relatively favourable economic conditions, Australian governments will post a collective deficit of between 2-3 per cent of GDP this year, and will remain in deficit by 1 per cent of GDP in 2017. Long-term spending has increased. The biggest driver was the sustained increase in health spending. Over the past decade health expenditure rose by over $40 billion in real terms. The ageing population was not the prime cause. Rather, people of any age saw doctors more often, had more tests and operations and took more prescription drugs. Similarly, Age Pension costs grew much faster than GDP, not because of population ageing, but with policy decisions to increase benefits and widen eligibility. New analysis in this edition of Budget Pressures shows that budget sustainability is also threatened by infrastructure spending. After a threefold increase in capital spending over the last 10 years, states are paying 3 per cent more of their revenues in interest and depreciation. Capital recycling and public private partnerships may improve credit ratings, but ultimately future recurrent budgets must still pay for the cost of past infrastructure. Continued trends in health and Age Pension costs are likely to drag future budgets backwards by 2 per cent of GDP by 2023. Future budgets will also be strained by promises of substantial new spending on the National Disability Insurance Scheme, schools, and defence, costing an extra 1 per cent of GDP. In addition, prices of Australia’s minerals are likely to decline, dragging budgets another ½ percent of GDP into the red. What can responsible leaders do to bring Australia’s budgets under control? First, they must explain the size and importance of the problem. Second, they must design a package of measures that share the burden of reform fairly across the community. As we showed in our Balancing Budgets report, the most promising reforms include lifting the age of access to Age Pension and superannuation, tightening the Age Pension assets test, paying less for pharmaceuticals with expired patents and asking students to pay a greater share of their tertiary education. However, given the size of the problem, budgets can only be balanced by looking at both expenditure and revenue. The highest priority tax increases should be the withdrawal of poorly targeted tax concessions, particularly superannuation for the wealthy, capital gains discounts, and negative gearing. Sustainable budgets require governments to make tough choices. They are politically difficult, but vital to Australia’s prosperity

Modern Information Systems

The development of modern information systems is a demanding task. New technologies and tools are designed, implemented and presented in the market on a daily bases. User needs change dramatically fast and the IT industry copes to reach the level of efficiency and adaptability for its systems in order to be competitive and up-to-date. Thus, the realization of modern information systems with great characteristics and functionalities implemented for specific areas of interest is a fact of our modern and demanding digital society and this is the main scope of this book. Therefore, this book aims to present a number of innovative and recently developed information systems. It is titled "Modern Information Systems" and includes 8 chapters. This book may assist researchers on studying the innovative functions of modern systems in various areas like health, telematics, knowledge management, etc. It can also assist young students in capturing the new research tendencies of the information systems' development

Novel information and data exchange within power systems using enhanced blockchain technologies

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonCurrent energy systems are primarily designed for centralized power generation and supplying bulk electricity to users with stable and predictable usage patterns. However, with the increasing penetration of renewable energy sources (RES), future energy systems will require greater flexibility and wider distribution of both demand and supply. Integrating RES on a large scale poses challenges to the hosting capacity of distribution systems. To address these challenges, the digitalization of energy systems through novel Information and Communication Technologies (ICT) infrastructure is essential. The shift from centralized to highly distributed systems necessitates increased coordination and communication efforts. This is because a distributed system is composed of multiple independent entities that need to communicate and collaborate effectively to accomplish a shared objective. Coordination and communication are necessary to ensure that the system is operating efficiently and effectively. Traditional centralized cloud-based data exchange schemes depend on a single trusted third party, this may lead to single-point failure and lack of data privacy and access control. To overcome these issues, a novel approach is proposed for exchanging data within power systems using blockchain technology. This approach enables users to securely exchange data while maintaining ownership. The experiments conducted demonstrate that the proposed approach can handle more users and enables information and data exchange within power systems. Secondly, this thesis proposes an Artificial Neural Network (ANN) based prediction model to optimize the performance of the blockchain-enabled data exchange approach. A use case for exchanging data within the power system is implemented on the proposed platform using various performance metrics. The results of the proposed approach are compared to two other schemes: the baseline scheme and an optimized scheme. The evaluation results indicate that the proposed approach can enhance network performance when compared to the baseline and optimized schemes. In summary, the proposed novel approach to ICT infrastructure for successfully exchanging information and data within power systems entities. The performance of the novel approach is evaluated based on the ability to handle multiple users, scalability, reliability, and security

Unfinished business: the governance of the Economic and Monetary Union. Compendium of EPC publications on the future of the Euro

Europe is once again engulfed in crisis. The sheer scale of refugees coming daily is not only a major challenge for the transit and destination countries, it is also exposing distrust between member states (and vis-à-vis the EU institutions). It has also shown that there is an unwillingness to cooperate and compromise within the EU system, in part a collateral damage of the eurocrisis. With a continuing sluggish economy and high unemployment, external challenges such as the conflict in Ukraine and internal ones like the referendum on EU membership in the UK, the EMU crisis looks less urgent at this point, with an agreement with Greece preventing the disastrous consequences of a Grexit, at least for now