67 research outputs found
Intra-hour cloud index forecasting with data assimilation
We introduce a computational framework to forecast cloud index (CI)fields for up to one hour on a spatial domain that covers a city. Such intra-hour CI forecasts are important to produce solar power forecasts of utility scale solar power and distributed rooftop solar. Our method combines a 2D advection model with cloud motion vectors (CMVs)derived from a mesoscale numerical weather prediction (NWP)model and sparse optical flow acting on successive, geostationary satellite images. We use ensemble data assimilation to combine these sources of cloud motion information based on the uncertainty of each data source. Our technique produces forecasts that have similar or lower root mean square error than reference techniques that use only optical flow, NWP CMV fields, or persistence. We describe how the method operates on three representative case studies and present results from 39 cloudy days
MALA-within-Gibbs samplers for high-dimensional distributions with sparse conditional structure
Markov chain Monte Carlo (MCMC) samplers are numerical methods for drawing samples from a given target probability distribution. We discuss one particular MCMC sampler, the MALA-within-Gibbs sampler, from the theoretical and practical perspectives. We first show that the acceptance ratio and step size of this sampler are independent of the overall problem dimension when (i) the target distribution has sparse conditional structure, and (ii) this structure is reflected in the partial updating strategy of MALA-within-Gibbs. If, in addition, the target density is blockwise log-concave, then the sampler's convergence rate is independent of dimension. From a practical perspective, we expect that MALA-within-Gibbs is useful for solving high-dimensional Bayesian inference problems where the posterior exhibits sparse conditional structure at least approximately. In this context, a partitioning of the state that correctly reflects the sparse conditional structure must be found, and we illustrate this process in two numerical examples. We also discuss trade-offs between the block size used for partial updating and computational requirements that may increase with the number of blocks
Recommended from our members
Performance bounds for particle filters using the optimal proposal
Particle filters may suffer from degeneracy of the particle weights. For the simplest "bootstrap" filter, it is known that avoiding degeneracy in large systems requires that the ensemble size must increase exponentially with the variance of the observation log-likelihood. The present article shows first that a similar result applies to particle filters using sequential importance sampling and the optimal proposal distribution and, second, that the optimal proposal yields minimal degeneracy when compared to any other proposal distribution that depends only on the previous state and the most recent observations. Thus, the optimal proposal provides performance bounds for filters using sequential importance sampling and any such proposal. An example with independent and identically distributed degrees of freedom illustrates both the need for exponentially large ensemble size with the optimal proposal as the system dimension increases and the potentially dramatic advantages of the optimal proposal relative to simpler proposals. Those advantages depend crucially on the magnitude of the system noise
Recommended from our members
The equivalent-weights particle filter in a high-dimensional system
In general, particle filters need large numbers of model runs in order to avoid filter degeneracy in high-dimensional systems. The recently proposed, fully nonlinear equivalent-weights particle filter overcomes this requirement by replacing the standard model transition density with two different proposal transition densities. The first proposal density is used to relax all particles towards the high-probability regions of state space as defined by the observations. The crucial second proposal density is then used to ensure that the majority of particles have equivalent weights at observation time. Here, the performance of the scheme in a high, 65 500 dimensional, simplified ocean model is explored. The success of the equivalent-weights particle filter in matching the true model state is shown using the mean of just 32 particles in twin experiments. It is of particular significance that this remains true even as the number and spatial variability of the observations are changed. The results from rank histograms are less easy to interpret and can be influenced considerably by the parameter values used. This article also explores the sensitivity of the performance of the scheme to the chosen parameter values and the effect of using different model error parameters in the truth compared with the ensemble model runs
- …