SCHEDULING OF UPDATES IN DATA WAREHOUSES

ABSTRACT A stream warehouse enables queries that seamlessly range from realtime alerting and diagnostics to long-term data mining. Continuously loading data from many different and uncontrolled sources into a real-time stream warehouse introduces a new consistency problem: users want results in as timely a fashion as possible, but "stable" results often require lengthy synchronization delays. In this paper we develop a theory of temporal consistency for stream warehouses that allows for multiple consistency levels. We model the streaming warehouse update problem as a scheduling problem, where jobs correspond to processes that load new data into tables, and whose objective is to minimize data staleness over time

Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture

We present the architecture behind Twitter's real-time related query suggestion and spelling correction service. Although these tasks have received much attention in the web search literature, the Twitter context introduces a real-time "twist": after significant breaking news events, we aim to provide relevant results within minutes. This paper provides a case study illustrating the challenges of real-time data processing in the era of "big data". We tell the story of how our system was built twice: our first implementation was built on a typical Hadoop-based analytics stack, but was later replaced because it did not meet the latency requirements necessary to generate meaningful real-time results. The second implementation, which is the system deployed in production, is a custom in-memory processing engine specifically designed for the task. This experience taught us that the current typical usage of Hadoop as a "big data" platform, while great for experimentation, is not well suited to low-latency processing, and points the way to future work on data analytics platforms that can handle "big" as well as "fast" data

New Generation of Educators Initiative: Reform Focus at Comprehensive Grant Sites

This first analysis of the early NGEI work at comprehensive grant campuses shows that collectively campuses are working across points on the pipeline to address the need for teachers who are better prepared to effectively teach to the new standards. While the bulk of the NGEI reform efforts are targeted at teacher preparation program reform, we see NGEI campuses reaching as far back as high school to cultivate early interest in, and preparedness for, teaching in response to local conditions such as limited candidate pools.Within teacher preparation, the early NGEI work of campuses is primarily clustered around the reform of the teacher preparation program coursework and clinical work (reflecting the first and third Key Transformation Elements). Partnerships with districts are at various stages of development and, in several cases, are focused primarily at the school level. A few campuses are reforming the formative feedback process for candidates through their NGEI work (Element 4). Work with district partners on the identification of the key skills, knowledge, and dispositions of well-prepared new teachers (Element 2) and work on continuous improvement based on data on candidates and program completers (Element 5) are less prominent in the NGEI work to date.As campuses clear the hurdle of launching their reforms in the summer and fall and look toward the next phase of NGEI funding, the evaluation (WestEd/SRI) and the facilitation (ConnectEd) teams are poised to provide support to grantees on the Key Transformation Elements that are not yet fully developed across all comprehensive sites, that is:* Partnerships with K–12 district partners to align programming as much as possible.* Shared understandings with K–12 district partners about the key knowledge, skills, and dispositions of a well-prepared new teacher that are used to inform teacher preparation program elements.* Feedback to candidates on their mastery of prioritized skills during preparation.* Data on candidate progress toward mastery of identified knowledge and practices during their training and after program completion.Specifically, ConnectEd is available to assist with implementation coaching and support for comprehensive campus teams and can support the work with K–12 partners.In addition to providing ongoing formative evaluation work across the comprehensive grant sites, the WestEd/SRI team can provide technical support for grantees to assist with the development of high-quality data on candidate progress toward mastery of identified knowledge and practices during their training and after program completion. The data inventories that the evaluation team developed for each campus show that there are opportunities to: a) enhance the quality of existing data, b) improve access to those data, and c) develop new data sources targeted toward the measurement of prioritized skills and knowledge for formative feedback to candidates. In the coming months, the evaluation team will also be seeking opportunities to bridge the system-level work described above in Box 1 with campus efforts to strengthen systems for continuous improvement.

Towards run-time monitoring of web services conformance to business-level agreements

Web service behaviour is currently specified in a mixture of ways, often using methods that are only partially complete. These range from static functional specifications, based on interfaces in WSDL and preconditions in RIF, to business process simulations using executable process-based models such as BPEL, to detailed quality of service (QoS) agreements laid down in a service level agreement (SLA). This paper recognises that something similar to a SLA is required at the higher business level to govern the contract between service producers, brokers and consumers. We call this a business level agreement (BLA) and within this framework, seek to unify disparate aspects of functional specification, QoS and run-time verification. We propose that the method for validating a web service with respect to its advertised BLA should be based on run-time service monitoring. This is a position paper towards defining these goals