Stochastic Reinforcement Learning

In reinforcement learning episodes, the rewards and punishments are often non-deterministic, and there are invariably stochastic elements governing the underlying situation. Such stochastic elements are often numerous and cannot be known in advance, and they have a tendency to obscure the underlying rewards and punishments patterns. Indeed, if stochastic elements were absent, the same outcome would occur every time and the learning problems involved could be greatly simplified. In addition, in most practical situations, the cost of an observation to receive either a reward or punishment can be significant, and one would wish to arrive at the correct learning conclusion by incurring minimum cost. In this paper, we present a stochastic approach to reinforcement learning which explicitly models the variability present in the learning environment and the cost of observation. Criteria and rules for learning success are quantitatively analyzed, and probabilities of exceeding the observation cost bounds are also obtained.Comment: AIKE 201

Likelihood estimation for distributed parameter models for NASA Mini-MAST truss

A maximum likelihood estimation for distributed parameter models of large flexible structures was formulated. Distributed parameter models involve far fewer unknown parameters than independent modal characteristics or finite element models. The closed form solutions for the partial differential equations with corresponding boundary conditions were derived. The closed-form expressions of sensitivity functions led to highly efficient algorithms for analyzing ground or on-orbit test results. For an illustration of this approach, experimental data of the NASA Mini-MAST truss was used. The estimations of modal properties involve lateral bending modes and torsional modes. The results show that distributed parameter models are promising in the parameter estimation of large flexible structures

Preservation of Oil Palm Fruits: Nonoxidative Effects of Ionizing Radiation on Palm Olein and Crude Palm Oil

The effect of gamma-irradiation on palm oil has been investigated. Irradiation doses (0.1 to 1 MGj') caused severe destruction of unsaturated but had little effect on saturated fatty acids. Similar effects were obsemed in irradiated samples stored for 1 and 2 months at room temperature. Radiation (uP to 30 kGy) also caused severe destruction of carotenes in crude palm oil, but had no significant effect on the free fatty acid content. These findings indicate that gamma-radiation may be used for preservation but not for sterilization of palm fruits

Application of Conjugable Oxidation Products Assay in Assessment of Gamma-Irradiated Palm Olein

Samples ojpalm olein were irradiated with y-rays up to 12kGy. The extent ojperoxidation in irradiated samples was determined by conjugable oxidation products (COP) assay and the result were compared with the UV absorbance at 232 nm. The two parameters were poorly correlated (r = 0.6321) within the range ofdoses used. The ifJect ofy-irradiation is mainly to oxidise linoleic acid (C18:2) as this component is the major diunsaturatedJatty acid in palm oil

Data assimilation in a sparsely observed one-dimensional modeled MHD system

International audienceA one dimensional non-linear magneto-hydrodynamic (MHD) system has been introduced to test a sequential optimal interpolation assimilation technique that uses a Monte-Carlo method to calculate the forecast error covariance. An ensemble of 100 model runs with perturbed initial conditions are used to construct the covariance, and the assimilation algorithm is tested using Observation Simulation Experiments (OSE's). The system is run with a variety of observation types (magnetic and/or velocity fields) and a range of observation densities. The impact of cross covariances between velocity and magnetic fields is investigated by running the assimilation with and without these terms. Sets of twin experiments show that while observing both velocity and magnetic fields has the greatest positive impact on the system, observing the magnetic field alone can also effectively constrain the system. Observations of the velocity field are ineffective as a constraint on the magnetic field, even when observations are made at every point. The implications for geomagnetic data assimilation are discussed

Gaussian approximations for stochastic systems with delay: chemical Langevin equation and application to a Brusselator system

We present a heuristic derivation of Gaussian approximations for stochastic chemical reaction systems with distributed delay. In particular we derive the corresponding chemical Langevin equation. Due to the non-Markovian character of the underlying dynamics these equations are integro-differential equations, and the noise in the Gaussian approximation is coloured. Following on from the chemical Langevin equation a further reduction leads to the linear-noise approximation. We apply the formalism to a delay variant of the celebrated Brusselator model, and show how it can be used to characterise noise-driven quasi-cycles, as well as noise-triggered spiking. We find surprisingly intricate dependence of the typical frequency of quasi-cycles on the delay period.Comment: 14 pages, 9 figure

Variational data assimilation for the initial-value dynamo problem

The secular variation of the geomagnetic field as observed at the Earth's surface results from the complex magnetohydrodynamics taking place in the fluid core of the Earth. One way to analyze this system is to use the data in concert with an underlying dynamical model of the system through the technique of variational data assimilation, in much the same way as is employed in meteorology and oceanography. The aim is to discover an optimal initial condition that leads to a trajectory of the system in agreement with observations. Taking the Earth's core to be an electrically conducting fluid sphere in which convection takes place, we develop the continuous adjoint forms of the magnetohydrodynamic equations that govern the dynamical system together with the corresponding numerical algorithms appropriate for a fully spectral method. These adjoint equations enable a computationally fast iterative improvement of the initial condition that determines the system evolution. The initial condition depends on the three dimensional form of quantities such as the magnetic field in the entire sphere. For the magnetic field, conservation of the divergence-free condition for the adjoint magnetic field requires the introduction of an adjoint pressure term satisfying a zero boundary condition. We thus find that solving the forward and adjoint dynamo system requires different numerical algorithms. In this paper, an efficient algorithm for numerically solving this problem is developed and tested for two illustrative problems in a whole sphere: one is a kinematic problem with prescribed velocity field, and the second is associated with the Hall-effect dynamo, exhibiting considerable nonlinearity. The algorithm exhibits reliable numerical accuracy and stability. Using both the analytical and the numerical techniques of this paper, the adjoint dynamo system can be solved directly with the same order of computational complexity as that required to solve the forward problem. These numerical techniques form a foundation for ultimate application to observations of the geomagnetic field over the time scale of centuries

Data assimilation in a sparsely observed one-dimensional modeled MHD system

A one dimensional non-linear magneto-hydrodynamic (MHD) system has been introduced to test a sequential optimal interpolation assimilation technique that uses a Monte-Carlo method to calculate the forecast error covariance. An ensemble of 100 model runs with perturbed initial conditions are used to construct the covariance, and the assimilation algorithm is tested using Observation Simulation Experiments (OSE's). The system is run with a variety of observation types (magnetic and/or velocity fields) and a range of observation densities. The impact of cross covariances between velocity and magnetic fields is investigated by running the assimilation with and without these terms. Sets of twin experiments show that while observing both velocity and magnetic fields has the greatest positive impact on the system, observing the magnetic field alone can also effectively constrain the system. Observations of the velocity field are ineffective as a constraint on the magnetic field, even when observations are made at every point. The implications for geomagnetic data assimilation are discussed

Prospects for detection of Υ(1D)→Υ(1S)ππ\Upsilon(1D) \to \Upsilon(1S) \pi \pi via Υ(3S)→Υ(1D)+X\Upsilon(3S) \to \Upsilon(1D) + X

At least one state in the first family of D-wave $b \bar b$ quarkonium levels has been discovered near the predicted mass of 10.16 GeV/$c^2$. This state is probably the one with J=2. This state and the ones with J=1 and J=3 may contribute a detectable amount to the decay $\Upsilon(1D) \to \Upsilon(1S) \pi \pi$, depending on the partial widths for these decays for which predictions vary considerably. The prospects for detection of the chain $\Upsilon(3S) \to \Upsilon(1D) + X \to \Upsilon \pi \pi + X$ are discussed.Comment: 4 pages, LaTeX, 1 figure, to be published in Phys. Rev. D, comment added after Eq. (2