A Data-Descriptive Feedback Framework for Data Stream Management Systems

Data Stream Management Systems (DSMSs) provide support for continuous query evaluation over data streams. Data streams provide processing challenges due to their unbounded nature and varying characteristics, such as rate and density fluctuations. DSMSs need to adapt stream processing to these changes within certain constraints, such as available computational resources and minimum latency requirements in producing results. The proposed research develops an inter-operator feedback framework, where opportunities for run-time adaptation of stream processing are expressed in terms of descriptions of substreams and actions applicable to the substreams, called feedback punctuations. Both the discovery of adaptation opportunities and the exploitation of these opportunities are performed in the query operators. DSMSs are also concerned with state management, in particular, state derived from tuple processing. The proposed research also introduces the Contracts Framework, which provides execution guarantees about state purging in continuous query evaluation for systems with and without inter-operator feedback. This research provides both theoretical and design contributions. The research also includes an implementation and evaluation of the feedback techniques in the NiagaraST DSMS, and a reference implementation of the Contracts Framework

Improving Travel Information Products via Robust Estimation Techniques

Traffic-monitoring systems, such as those using loop detectors, are prone to coverage gaps, arising from sensor noise, processing errors and transmission problems. Such gaps adversely affect the accuracy of Advanced Traveler Information Systems. This project will explore models based on historical data that can provide estimates to fill such gaps. We build on an initial study by Mr. Rafael J. Fernandez-Moctezuma, using both a linear model and an artificial neural network (ANN) trained on historical data to estimate values for reporting gaps. These initial models were 80% and 89% accurate, respectively, in estimating the correct speed range, and misclassifications were always between adjacent speed ranges (in particular, the free-flow range and congested range were never confused). Going forward, we will investigate other non-linear models, such as Gaussian Mixtures, that provide further statistical metrics, in contrast to the uninterpreted weights of ANNs. This work will exploit the Portland Transportation Archive Listing (PORTAL) at the Intelligent Transportation Systems Laboratory at PSU. Dr. Tufte helps supervise development of PORTAL, and Mr. Fernandez used PORTAL data in his study. PORTAL holds more than two years of Portland-area freeway-loop-detector data at both detailed and aggregated levels, and is an ideal resource for the proposed work. Initially we will be building and testing estimators in off-line mode. We will select a highway segment (comprising multiple detector stations) that is representative in terms of pattern of outages. We will build models for this segment, then examine their performance on estimates for synthetic gaps (so we can compare estimates to reported values). Later, using live loop-detector data (which PORTAL supports), we will work towards on-line estimation over the local freeway network, which requires computing estimates in a timely manner. Our end target is improvements in end-user travel information products, such as the Portland-Metro Speed Map on ODOT\u27s Trip Check. Our main evaluation metric will be the trade-off curve bewteen accuracy of prediciton and percentage of gaps that can be filled. This research supports national surface-transportation research priorities, including the Systems Management Information area (ITS JPO). Within that area, it relates to (2) Data Management (techniques and guidance for processing and managing data associated with highway and transit monitoring) and (5) Data Dissemination (exchanging information about transportation services and providing that information to travelers). [Page 3-15, U.S. Department of Transportation Research, Development, and Technology Plan, 6th Edition

Support for Schema Evolution in Data Stream Management Systems

Unlike Database Management Systems (DBMSs), Data Stream Management Systems (DSMSs) do not evaluate queries over static data sets — rather, they continuously produce result streams to standing queries, and often operate in a context where any interruption can lead to data loss. Support for schema evolution in such an environment is currently unaddressed. In this work we address evolution in DSMSs by introducing a new element to streams, called an accent, that precedes and describes an evolution. We characterize how a subset of commonly used query operators in DSMS act on and propagate accents with respect to three evolution primitives: Add Attribute, Drop Attribute, and Alter Data