Trend Detection based Regret Minimization for Bandit Problems

We study a variation of the classical multi-armed bandits problem. In this problem, the learner has to make a sequence of decisions, picking from a fixed set of choices. In each round, she receives as feedback only the loss incurred from the chosen action. Conventionally, this problem has been studied when losses of the actions are drawn from an unknown distribution or when they are adversarial. In this paper, we study this problem when the losses of the actions also satisfy certain structural properties, and especially, do show a trend structure. When this is true, we show that using \textit{trend detection}, we can achieve regret of order $\tilde{O} (N \sqrt{TK})$ with respect to a switching strategy for the version of the problem where a single action is chosen in each round and $\tilde{O} (Nm \sqrt{TK})$ when $m$ actions are chosen each round. This guarantee is a significant improvement over the conventional benchmark. Our approach can, as a framework, be applied in combination with various well-known bandit algorithms, like Exp3. For both versions of the problem, we give regret guarantees also for the \textit{anytime} setting, i.e. when the length of the choice-sequence is not known in advance. Finally, we pinpoint the advantages of our method by comparing it to some well-known other strategies

A Bayesian spatio-temporal model of panel design data: airborne particle number concentration in Brisbane, Australia

This paper outlines a methodology for semi-parametric spatio-temporal modelling of data which is dense in time but sparse in space, obtained from a split panel design, the most feasible approach to covering space and time with limited equipment. The data are hourly averaged particle number concentration (PNC) and were collected, as part of the Ultrafine Particles from Transport Emissions and Child Health (UPTECH) project. Two weeks of continuous measurements were taken at each of a number of government primary schools in the Brisbane Metropolitan Area. The monitoring equipment was taken to each school sequentially. The school data are augmented by data from long term monitoring stations at three locations in Brisbane, Australia. Fitting the model helps describe the spatial and temporal variability at a subset of the UPTECH schools and the long-term monitoring sites. The temporal variation is modelled hierarchically with penalised random walk terms, one common to all sites and a term accounting for the remaining temporal trend at each site. Parameter estimates and their uncertainty are computed in a computationally efficient approximate Bayesian inference environment, R-INLA. The temporal part of the model explains daily and weekly cycles in PNC at the schools, which can be used to estimate the exposure of school children to ultrafine particles (UFPs) emitted by vehicles. At each school and long-term monitoring site, peaks in PNC can be attributed to the morning and afternoon rush hour traffic and new particle formation events. The spatial component of the model describes the school to school variation in mean PNC at each school and within each school ground. It is shown how the spatial model can be expanded to identify spatial patterns at the city scale with the inclusion of more spatial locations.Comment: Draft of this paper presented at ISBA 2012 as poster, part of UPTECH projec

Carbon Dioxide Plume Evolution Following Injection into a Depleted Natural Gas Reservoir: Modeling of Conformance Uncertainty Reduction Over Time

The uncertainty in the long-term fate of CO2 injected for geologic carbon sequestration (GCS) is a significant barrier to the adoption of GCS as a greenhouse gas emission mitigation approach for industry and regulatory agencies alike. Here we present a modeling study that demonstrates that the uncertainty in forecasts of GCS site performance decreases over time as monitoring data are used to inform and update operational models. The approach we take is to consider a case study consisting of a depleted natural gas reservoir that is used for GCS with CO2 injection occurring over 20 years, with a 50-year post-injection site care (PISC) period. We constructed a detailed model of the system and ran this model out to 200 years to generate the actual site data. A series of simpler operational models based on limited data and assumptions about how an actual operator would model such a site are then run and compared against the actual model output at various specific monitoring points after one year, two years, etc. The operational model is then updated and improved using the observations (synthetic data from the actual model) at the same time intervals. We found that both model parameter values and model features needed to be added over time to improve matches to the actual system. These kinds of model adjustments are expected to be a normal part of reservoir engineering and site management at GCS sites. We found that the uncertainty in two key measures related to site performance at various locations decreases with time. This overall conclusion should help allay the concerns of industry and regulators about the uncertainty in GCS operations

CO2 plume evolution in a depleted natural gas reservoir: Modeling of conformance uncertainty reduction over time

Uncertainty in the long-term fate of CO2 injected for geologic carbon sequestration (GCS) is a significant barrier to the adoption of GCS as a greenhouse-gas emission-mitigation for industry and regulatory agencies alike. We present a modeling study that demonstrates that the uncertainty in forecasts of GCS site performance decreases over time as monitoring data are used to update operational models. We consider a case study of GCS in a depleted natural gas reservoir, with CO2 injection occurring over 20 years, with a 50-year post-injection site care period. We constructed a detailed model to generate the actual model output, which is considered synthetic observation data. A series of simpler operational models based on limited data and assumptions about how an operator would model such a site are then run and compared against actual model output at specific monitoring points after one year, two years, etc. The operational model is updated and improved using the synthetic observation data from the actual model at the same time intervals. Model parameter values and model features needed to be updated over time to improve matches to the actual model. These kinds of model adjustments would be a normal part of reservoir engineering and site management at GCS sites. Uncertainty in two key measures related to site performance decreases with time: extent of the CO2 plume up-dip migration, and radial extent of the pressure pulse. This conclusion should help allay the concerns of industry and regulators about uncertainty in long-term fate of CO2 at GCS sites

Jump-preserving monitoring of dependent time series using pilot estimators

An important problem of the statistical analysis of time series is to detect change-points in the mean structure. Since this problem is a one-dimensional version of the higher dimensional problem of detecting edges in images, we study detection rules which benefit from results obtained in image processing. For the sigma-filter studied there to detect edges, asymptotic bounds for the normed delay have been established for independent data. These results are considerably extended in two directions. First, we allow for dependent processes satisfying a certain conditional mixing property. Second, we allow for more general pilot estimators, e.g., the median, resulting in better detection properties. A simulation study indicates that our new procedure indeed performs much more better. --Image processing,Nonparametric regression,Quality Control,Structural Change

A Geostatistical Approach to Define Guidelines for Radon Prone Area Identification

Radon is a natural radioactive gas known to be the main contributor to natural background radiation exposure and the major leading cause of lung cancer second to smoking. Indoor radon concentration levels of 200 and 400 Bq/m3 are reference values suggested by the 90/143/Euratom recommendation, above which mitigation measures should be taken in new and old buildings, respectively, to reduce exposure to radon. Despite this international recommendation, Italy still does not have mandatory regulations or guidelines to deal with radon in dwellings. Monitoring surveys have been undertaken in a number of western European countries in order to assess the exposure of people to this radioactive gas and to identify radon prone areas. However, such campaigns provide concentration values in each single dwelling included in the sample, while it is often necessary to provide measures of the pollutant concentration which refer to sub-areas of the region under study. This requires a realignment of the spatial data from the level at which they are collected (points) to the level at which they are necessary (areas). This is known as change of support problem. In this paper, we propose a methodology based on geostatistical simulations in order to solve this problem and to identify radon prone areas which may be suggested for national guidelines.Radon Prone Areas, kriging, geostatistical conditional simulation, change of support problem