Outlier Detection and Missing Value Estimation in Time Series Traffic Count Data: Final Report of SERC Project GR/G23180.

A serious problem in analysing traffic count data is what to do when missing or extreme values occur, perhaps as a result of a breakdown in automatic counting equipment. The objectives of this current work were to attempt to look at ways of solving this problem by: 1)establishing the applicability of time series and influence function techniques for estimating missing values and detecting outliers in time series traffic data; 2)making a comparative assessment of new techniques with those used by traffic engineers in practice for local, regional or national traffic count systems Two alternative approaches were identified as being potentially useful and these were evaluated and compared with methods currently employed for `cleaning' traffic count series. These were based on evaluating the effect of individual or groups of observations on the estimate of the auto-correlation structure and events influencing a parametric model (ARIMA). These were compared with the existing methods which included visual inspection and smoothing techniques such as the exponentially weighted moving average in which means and variances are updated using observations from the same time and day of week. The results showed advantages and disadvantages for each of the methods. The exponentially weighted moving average method tended to detect unreasonable outliers and also suggested replacements which were consistently larger than could reasonably be expected. Methods based on the autocorrelation structure were reasonably successful in detecting events but the replacement values were suspect particularly when there were groups of values needing replacement. The methods also had problems in the presence of non-stationarity, often detecting outliers which were really a result of the changing level of the data rather than extreme values. In the presence of other events, such as a change in level or seasonality, both the influence function and change in autocorrelation present problems of interpretation since there is no way of distinguishing these events from outliers. It is clear that the outlier problem cannot be separated from that of identifying structural changes as many of the statistics used to identify outliers also respond to structural changes. The ARIMA (1,0,0)(0,1,1)7 was found to describe the vast majority of traffic count series which means that the problem of identifying a starting model can largely be avoided with a high degree of assurance. Unfortunately it is clear that a black-box approach to data validation is prone to error but methods such as those described above lend themselves to an interactive graphics data-validation technique in which outliers and other events are highlighted requiring acceptance or otherwise manually. An adaptive approach to fitting the model may result in something which can be more automatic and this would allow for changes in the underlying model to be accommodated. In conclusion it was found that methods based on the autocorrelation structure are the most computationally efficient but lead to problems of interpretation both between different types of event and in the presence of non-stationarity. Using the residuals from a fitted ARIMA model is the most successful method at finding outliers and distinguishing them from other events, being less expensive than case deletion. The replacement values derived from the ARIMA model were found to be the most accurate

Setar Modelling of Traffic Count Data.

As part of a SERC funded project investigating outlier detection and replacement with transport data, univariate Box-Jenkins (1976) models have already been successfully applied to traffic count series (see Redfern et al, 1992). However, the underlying assumption of normality for ARIMA models implies they are not ideally suited for time series exhibiting certain behavioural characteristics. The limitations of ARIMA models are discussed in some detail by Tong (1983), including problems with time irreversibility, non-normality, cyclicity and asymmetry. Data with irregularly spaced extreme values are unlikely to be modelled well by ARIMA models, which are better suited to data where the probability of a very high value is small. Tong (1983) argues that one way of modelling such non-normal behaviour might be to retain the general ARIMA framework and allow the white noise element to be non-gaussian. As an alternative he proposes abandoning the linearity assumption and defines a group of non linear structures, one of which is the Self-Exciting Threshold Autoregressive (SETAR) model. The model form is described in more detail below but basically consists of two (or more) piecewise linear models, with the time series "tripping" between each model according to its value with respect to a threshold point. The model is called "Self-Exciting" because the indicator variable determining the appropriate linear model for each piece of data is itself a function of the data series. Intuitively this means the mechanism driving the alternation between each model form is not an external input such as a related time series (other models can be defined where this exists), but is actually contained within the series itself. The series is thus Self-Exciting. The three concepts embedded within the SETAR model structure are those of the threshold, limit cycle and time delay, each of which can be illustrated by the diverse applications such models can take. The threshold can be defined as some point beyond which, if the data falls, the series structure changes inherently and so an alternative linear model form would be appropriate. In hydrology this is seen as the non-linearity of soil infiltration, where at the soil saturation point (threshold) a new model for infiltration would become appropriate. Limit cycles describe the stable cyclical phenomena which we sometimes observe within time series. The cyclical behaviour is stationary, ie consists of regular, sustained oscillations and is an intrinsic property of the data. The limit cycle phenomena is physically observable in the field of radio-engineering where a triode valve is used to generate oscillations (see Tong, 1983 for a full description). Essentially the triode value produces self-sustaining oscillations between emitting and collecting electrons, according to the voltage value of a grid placed between the anode and cathode (thereby acting as the threshold indicator). The third essential concept within the SETAR structure is that of the time delay and is perhaps intuitively the easiest to grasp. It can be seen within the field of population biology where many types of non-linear model may apply. For example within the cyclical oscillations of blowfly population data there is an inbuilt "feedback" mechanism given by the hatching period for eggs, which would give rise to a time delay parameter within the model. For some processes this inherent delay may be so small as to be virtually instantaneous and so the delay parameter could be omitted. In general time series Tong (1983) found the SETAR model well suited to the cyclical nature of the Canadian Lynx trapping series and for modelling riverflow systems (Tong, Thanoon & Gudmundsson, 1984). Here we investigate their applicability with time series traffic counts, some of which have exhibited the type of non-linear and cyclical characteristics which could undermine a straightforward linear modelling process

Application of Outlier Detection and Missing Value Estimation Techniques to Various Forms of Traffic Count Data

This paper reports on the application of suitable techniques for detecting outliers and suggesting estimates for missing values in various forrns of traffic count data. The data used in this study came from three sources. The first set was provided by the Department of Transport's (DOT) regional office in Leeds and consists of automatic hourly traffic counts at four sites. The second set was part of a larger database provided by West Yorkshire Highways, Engineering and Technical Services (HETS). This set consists of automatic half hourly traffic counts on a single site. The third and final set was provided by Nottinghan University and consists of automatic five minute traffic counts at 40 locations, in close proximity to each other, from Leicester. Three suitable techniques emerged from pilot studies of such series conducted by Watson et a1 (1992a) and Redfern et a1 (1992). The three techniques are: a) Maintaining an average and variability measure over time; b) ARIMA modelling with detection of large residuals; C) A point's influence on the correlation structure of the series. A fourth technique, by-eye detection and estimation, provides an intuitive comparison for the first three techniques

An Influence Method for Outliers Detection Applied to Time Series Traffic Data

The applicability of an outlier detection statistic developed for standard time series is assessed in estimating missing values and detecting outliers in traffic count data. The work of Chernick, Downing and Pike (1982) is extended to form a quantitive outlier detection statistic for use with time series data. The statistic is formed from the squared elements of the Influence Function Matrix, where each element of the matrix gives the influence on pk, of a pair of observations at time lag k. Approximate first four moments for the statistic are derived and by fitting Johnson curves to those theoretical moments, critical points are also produced. The statistic is also used to form the basis of an adjustment procedure to treat outliers or estimate missing values in the time series. Chernick et al's (1982) nuclear power data and the Department of Transport's traffic count data are used for practical illustration

Instructional Methods that Foster the Reading Development of Students with Significant Intellectual Disabilities

Educational legislation has made reading a priority for students with significant intellectual disabilities (ID) and associated speech, language, sensory, or physical impairments. Historically, reading instruction for students with significant ID has focused on sight word instruction, with limited exposure to other essential reading skills. This article focuses on the evidence-based instructional methods that effectively and efficiently foster the reading development of students with significant ID. The authors reviewed the literature from the past 20 years on reading interventions for students with significant ID. In spite of access and opportunity barriers that have inhibited the reading development of students with significant ID, a synthesis of the empirical research on reading instruction suggests that students with significant ID and associated disabilities can learn phonemic awareness, phonics, vocabulary, fluency, and comprehension skills with direct instruction. Implications for providing reading instruction that effectively promotes reading development are discussed and areas for future research are identified

Development of an Influence Statistic for Outlier Detection With Time Series Traffic Data.

As part of a SERC funded project investigating the detection and treatment of outlying time series transport data, the practical applicability of the Influence Statistic described by Watson et al(1991) is assessed here. Missing or outlying data occur in a variety of transport time series such as traflic counts or journey times for many reasons including broken machinery and recording errors. In practice such data is patched largely by subjective opinion or using simple aggregate methods. In the analysis of non-transport time series several methods have been recently developed to both detect and treat outliers, including work by Kohn and Ansley (1986), Hau and Tong (1984) and Bruce and Martin (1989). These methods use either an intervention modelling approach (where the outlier is modelled as part of an ARIMA structure) or look at the influence an observation exerts on a particular parameter associated with the model. An alternative is the Influence Statistic proposed by Watson (1987) and Watson et al (1992) which examines the influence of an observation on the sample autocorrelation function. Initial research showed the statistic has practical application in a transport context, and a replacement procedure based on the method was found to be effective in treating maverick data. Here we report the results from a wider application of the statistic using traffic count data fmm. the Department of Transport. Further developments are suggested and investigated for the replacement procedure and a comparison is made between possible variations in the method

Modelling Outliers and Missing Values in traffic Count Data Using the ARIMA Model.

This paper considers the application of the methodology to traffic count time series in which both missing values and outliers are present. Intervention analysis and detection using large residuals are shown to he reasonably effective but possible problems that result from non- stationarity in the data are identified. It is shown that despite considerable variabilty in the types of series the model selected from the ARIMA family is surprisingly homogeneous

QCD Down Under: Building Bridges

The strong coupling regime of QCD is responsible for 99% of hadronic phenomena. Though considerable progress has been made in solving QCD in this non-perturbative region, we nevertheless have to rely on a disparate range of models and approximations. If we are to gain an understanding of the underlying physics and not just have numerical answers from computing `` black'' boxes, we must build bridges between the parameter space where models and approximations are valid to the regime describing experiment, and between the different modellings of strong dynamics. We describe here how the Schwinger-Dyson/Bethe-Salpeter approach provides just such a bridge, linking physics, the lattice and experiment.Comment: 8 pages, 10 figures. Opening talk at Workshop on QCD Down Under, March 2004, Barossa Valley and Adelaide (to be published in the Proceedings

Efficacy of pimobendan in the prevention of congestive heart failure or sudden death in doberman pinschers with preclinical dilated cardiomyopathy (the PROTECT study)

<p>Background: The benefit of pimobendan in delaying the progression of preclinical dilated cardiomyopathy (DCM) in Dobermans is not reported.</p> <p>Hypothesis: That chronic oral administration of pimobendan to Dobermans with preclinical DCM will delay the onset of CHF or sudden death and improve survival.</p> <p>Animals: Seventy-six client-owned Dobermans recruited at 10 centers in the UK and North America.</p> <p>Methods: The trial was a randomized, blinded, placebo-controlled, parallel group multicenter study. Dogs were allocated in a 1:1 ratio to receive pimobendan (Vetmedin capsules) or visually identical placebo.</p> <p>The composite primary endpoint was prospectively defined as either onset of CHF or sudden death. Time to death from all causes was a secondary endpoint.</p> <p>Results: The proportion of dogs reaching the primary endpoint was not significantly different between groups (P = .1). The median time to the primary endpoint (onset of CHF or sudden death) was significantly longer in the pimobendan (718 days, IQR 441–1152 days) versus the placebo group (441 days, IQR 151–641 days) (log-rank P = 0.0088). The median survival time was significantly longer in the pimobendan (623 days, IQR 491–1531 days) versus the placebo group (466 days, IQR 236–710 days) (log-rank P = .034).</p> <p>Conclusion and Clinical Importance: The administration of pimobendan to Dobermans with preclinical DCM prolongs the time to the onset of clinical signs and extends survival. Treatment of dogs in the preclinical phase of this common cardiovascular disorder with pimobendan can lead to improved outcome.</p&gt

Religious diversity, empathy, and God images : perspectives from the psychology of religion shaping a study among adolescents in the UK

Major religious traditions agree in advocating and promoting love of neighbour as well as love of God. Love of neighbour is reflected in altruistic behaviour and empathy stands as a key motivational factor underpinning altruism. This study employs the empathy scale from the Junior Eysenck Impulsiveness Questionnaire to assess the association between empathy and God images among a sample of 5993 religiously diverse adolescents (13–15 years old) attending state maintained schools in England, Northern Ireland, Scotland, Wales, and London. The key psychological theory being tested by these data concerns the linkage between God images and individual differences in empathy. The data demonstrate that religious identity (e.g. Christian, Muslim) and religious attendance are less important than the God images which young people hold. The image of God as a God of mercy is associated with higher empathy scores, while the image of God as a God of justice is associated with lower empathy scores