A Practitioner's Guide to Bayesian Estimation of Discrete Choice Dynamic Programming Models

This paper provides a step-by-step guide to estimating discrete choice dynamic programming (DDP) models using the Bayesian Dynamic Programming algorithm developed by Imai Jain and Ching (2008) (IJC). The IJC method combines the DDP solution algorithm with the Bayesian Markov Chain Monte Carlo algorithm into a single algorithm, which solves the DDP model and estimates its structural parameters simultaneously. The main computational advantage of this estimation algorithm is the efficient use of information obtained from the past iterations. In the conventional Nested Fixed Point algorithm, most of the information obtained in the past iterations remains unused in the current iteration. In contrast, the Bayesian Dynamic Programming algorithm extensively uses the computational results obtained from the past iterations to help solving the DDP model at the current iterated parameter values. Consequently, it significantly alleviates the computational burden of estimating a DDP model. We carefully discuss how to implement the algorithm in practice, and use a simple dynamic store choice model to illustrate how to apply this algorithm to obtain parameter estimates.Bayesian Dynamic Programming, Discrete Choice Dynamic Programming, Markov Chain Monte Carlo

Data Driven Prognosis: A multi-physics approach verified via balloon burst experiment

A multi-physics formulation for Data Driven Prognosis (DDP) is developed. Unlike traditional predictive strategies that require controlled off-line measurements or training for determination of constitutive parameters to derive the transitional statistics, the proposed DDP algorithm relies solely on in situ measurements. It utilizes a deterministic mechanics framework, but the stochastic nature of the solution arises naturally from the underlying assumptions regarding the order of the conservation potential as well as the number of dimensions involved. The proposed DDP scheme is capable of predicting onset of instabilities. Since the need for off-line testing (or training) is obviated, it can be easily implemented for systems where such a priori testing is difficult or even impossible to conduct. The prognosis capability is demonstrated here via a balloon burst experiment where the instability is predicted utilizing only on-line visual observations. The DDP scheme never failed to predict the incipient failure, and no false positives were issued. The DDP algorithm is applicable to others types of datasets. Time horizons of DDP predictions can be adjusted by using memory over different time windows. Thus, a big dataset can be parsed in time to make a range of predictions over varying time horizons

Strong Secrecy on the Binary Erasure Wiretap Channel Using Large-Girth LDPC Codes

For an arbitrary degree distribution pair (DDP), we construct a sequence of low-density parity-check (LDPC) code ensembles with girth growing logarithmically in block-length using Ramanujan graphs. When the DDP has minimum left degree at least three, we show using density evolution analysis that the expected bit-error probability of these ensembles, when passed through a binary erasure channel with erasure probability $\epsilon$, decays as $\mathcal{O}(\exp(-c_1 n^{c_2}))$ with the block-length $n$ for positive constants $c_1$ and $c_2$, as long as $\epsilon$ is lesser than the erasure threshold $\epsilon_\mathrm{th}$ of the DDP. This guarantees that the coset coding scheme using the dual sequence provides strong secrecy over the binary erasure wiretap channel for erasure probabilities greater than $1 - \epsilon_\mathrm{th}$.Comment: 11 pages, 4 figures. Submitted to the IEEE Transactions on Information Forensics and Securit

Beta-Product Poisson-Dirichlet Processes

Time series data may exhibit clustering over time and, in a multiple time series context, the clustering behavior may differ across the series. This paper is motivated by the Bayesian non--parametric modeling of the dependence between the clustering structures and the distributions of different time series. We follow a Dirichlet process mixture approach and introduce a new class of multivariate dependent Dirichlet processes (DDP). The proposed DDP are represented in terms of vector of stick-breaking processes with dependent weights. The weights are beta random vectors that determine different and dependent clustering effects along the dimension of the DDP vector. We discuss some theoretical properties and provide an efficient Monte Carlo Markov Chain algorithm for posterior computation. The effectiveness of the method is illustrated with a simulation study and an application to the United States and the European Union industrial production indexes

On the Benefits of Surrogate Lagrangians in Optimal Control and Planning Algorithms

This paper explores the relationship between numerical integrators and optimal control algorithms. Specifically, the performance of the differential dynamical programming (DDP) algorithm is examined when a variational integrator and a newly proposed surrogate variational integrator are used to propagate and linearize system dynamics. Surrogate variational integrators, derived from backward error analysis, achieve higher levels of accuracy while maintaining the same integration complexity as nominal variational integrators. The increase in the integration accuracy is shown to have a large effect on the performance of the DDP algorithm. In particular, significantly more optimized inputs are computed when the surrogate variational integrator is utilized

Path integral policy improvement with differential dynamic programming

Path Integral Policy Improvement with Covariance Matrix Adaptation (PI2-CMA) is a step-based model free reinforcement learning approach that combines statistical estimation techniques with fundamental results from Stochastic Optimal Control. Basically, a policy distribution is improved iteratively using reward weighted averaging of the corresponding rollouts. It was assumed that PI2-CMA somehow exploited gradient information that was contained by the reward weighted statistics. To our knowledge we are the first to expose the principle of this gradient extraction rigorously. Our findings reveal that PI2-CMA essentially obtains gradient information similar to the forward and backward passes in the Differential Dynamic Programming (DDP) method. It is then straightforward to extend the analogy with DDP by introducing a feedback term in the policy update. This suggests a novel algorithm which we coin Path Integral Policy Improvement with Differential Dynamic Programming (PI2-DDP). The resulting algorithm is similar to the previously proposed Sampled Differential Dynamic Programming (SaDDP) but we derive the method independently as a generalization of the framework of PI2-CMA. Our derivations suggest to implement some small variations to SaDDP so to increase performance. We validated our claims on a robot trajectory learning task

A Guide to Distributed Digital Preservation

This volume is devoted to the broad topic of distributed digital preservation, a still-emerging field of practice for the cultural memory arena. Replication and distribution hold out the promise of indefinite preservation of materials without degradation, but establishing effective organizational and technical processes to enable this form of digital preservation is daunting. Institutions need practical examples of how this task can be accomplished in manageable, low-cost ways."--P. [4] of cove

Skor Pola Pangan Harapan dan Hubungannya dengan Status Gizi Anak Usia 0,5 – 12 Tahun di Indonesia

The prevalence of undernutrition in Indonesia is still high compared to its neighbouring countries. The causes are quantity and quality of dietary intakes that can be assesed by dietary recall and desirable dietary pattern (DDP) respectively. The objective was to measure DDP and its association with nutritional status of 0,5-12-year-old Indonesian children. Data was obtained from SEANUTS Indonesia\u27s research covering 3.600 children in 48 districts. Trained nutritionists collected food intakes and dietary pattern by 1x24 hour dietary recall. Nutrient intakes and DDP were calculated by food composition tables and 9 food groups respectively. DDP score were categorized into lowest (score <55), low (55-70), medium (71-84), and high (>85). Weight, length/height were measured by digital weight scale and length measuring board/microtoise. World Health Organization (WHO) child standard was used to calculate W/A, H/A, W/H Z-scores. Analysis was done to measure DDP and its association with nutritional status. The result showed that DDP child 0,5-1,9 years was 48,7 point, DDP child 2,0-5,9 years was 54,7 point, DD child 6,0-12,9 years was 48,8 point. The overall DDP was 49,9 point, far below the maximum value 100 point. DDP was higher among older age, urban areas, higher father education, and higher socioeconomic status. The risk of stunted was higher in low DDP (OR = 1,24; 95% CI 1,15-1,732) and underweight (OR = 1,27; 95% CI 1,16-1,38) but no risk for wasted. The conclusion DDP of Indonesian children was still low and it was associated significantly with stunting and underweight