Stochastic Reinforcement Learning

In reinforcement learning episodes, the rewards and punishments are often non-deterministic, and there are invariably stochastic elements governing the underlying situation. Such stochastic elements are often numerous and cannot be known in advance, and they have a tendency to obscure the underlying rewards and punishments patterns. Indeed, if stochastic elements were absent, the same outcome would occur every time and the learning problems involved could be greatly simplified. In addition, in most practical situations, the cost of an observation to receive either a reward or punishment can be significant, and one would wish to arrive at the correct learning conclusion by incurring minimum cost. In this paper, we present a stochastic approach to reinforcement learning which explicitly models the variability present in the learning environment and the cost of observation. Criteria and rules for learning success are quantitatively analyzed, and probabilities of exceeding the observation cost bounds are also obtained.Comment: AIKE 201

Preservation of Oil Palm Fruits: Nonoxidative Effects of Ionizing Radiation on Palm Olein and Crude Palm Oil

The effect of gamma-irradiation on palm oil has been investigated. Irradiation doses (0.1 to 1 MGj') caused severe destruction of unsaturated but had little effect on saturated fatty acids. Similar effects were obsemed in irradiated samples stored for 1 and 2 months at room temperature. Radiation (uP to 30 kGy) also caused severe destruction of carotenes in crude palm oil, but had no significant effect on the free fatty acid content. These findings indicate that gamma-radiation may be used for preservation but not for sterilization of palm fruits

Application of Conjugable Oxidation Products Assay in Assessment of Gamma-Irradiated Palm Olein

Samples ojpalm olein were irradiated with y-rays up to 12kGy. The extent ojperoxidation in irradiated samples was determined by conjugable oxidation products (COP) assay and the result were compared with the UV absorbance at 232 nm. The two parameters were poorly correlated (r = 0.6321) within the range ofdoses used. The ifJect ofy-irradiation is mainly to oxidise linoleic acid (C18:2) as this component is the major diunsaturatedJatty acid in palm oil

Likelihood estimation for distributed parameter models for NASA Mini-MAST truss

A maximum likelihood estimation for distributed parameter models of large flexible structures was formulated. Distributed parameter models involve far fewer unknown parameters than independent modal characteristics or finite element models. The closed form solutions for the partial differential equations with corresponding boundary conditions were derived. The closed-form expressions of sensitivity functions led to highly efficient algorithms for analyzing ground or on-orbit test results. For an illustration of this approach, experimental data of the NASA Mini-MAST truss was used. The estimations of modal properties involve lateral bending modes and torsional modes. The results show that distributed parameter models are promising in the parameter estimation of large flexible structures

Data assimilation in a sparsely observed one-dimensional modeled MHD system

International audienceA one dimensional non-linear magneto-hydrodynamic (MHD) system has been introduced to test a sequential optimal interpolation assimilation technique that uses a Monte-Carlo method to calculate the forecast error covariance. An ensemble of 100 model runs with perturbed initial conditions are used to construct the covariance, and the assimilation algorithm is tested using Observation Simulation Experiments (OSE's). The system is run with a variety of observation types (magnetic and/or velocity fields) and a range of observation densities. The impact of cross covariances between velocity and magnetic fields is investigated by running the assimilation with and without these terms. Sets of twin experiments show that while observing both velocity and magnetic fields has the greatest positive impact on the system, observing the magnetic field alone can also effectively constrain the system. Observations of the velocity field are ineffective as a constraint on the magnetic field, even when observations are made at every point. The implications for geomagnetic data assimilation are discussed

Data assimilation in a sparsely observed one-dimensional modeled MHD system

A one dimensional non-linear magneto-hydrodynamic (MHD) system has been introduced to test a sequential optimal interpolation assimilation technique that uses a Monte-Carlo method to calculate the forecast error covariance. An ensemble of 100 model runs with perturbed initial conditions are used to construct the covariance, and the assimilation algorithm is tested using Observation Simulation Experiments (OSE's). The system is run with a variety of observation types (magnetic and/or velocity fields) and a range of observation densities. The impact of cross covariances between velocity and magnetic fields is investigated by running the assimilation with and without these terms. Sets of twin experiments show that while observing both velocity and magnetic fields has the greatest positive impact on the system, observing the magnetic field alone can also effectively constrain the system. Observations of the velocity field are ineffective as a constraint on the magnetic field, even when observations are made at every point. The implications for geomagnetic data assimilation are discussed

Variational data assimilation for the initial-value dynamo problem

The secular variation of the geomagnetic field as observed at the Earth's surface results from the complex magnetohydrodynamics taking place in the fluid core of the Earth. One way to analyze this system is to use the data in concert with an underlying dynamical model of the system through the technique of variational data assimilation, in much the same way as is employed in meteorology and oceanography. The aim is to discover an optimal initial condition that leads to a trajectory of the system in agreement with observations. Taking the Earth's core to be an electrically conducting fluid sphere in which convection takes place, we develop the continuous adjoint forms of the magnetohydrodynamic equations that govern the dynamical system together with the corresponding numerical algorithms appropriate for a fully spectral method. These adjoint equations enable a computationally fast iterative improvement of the initial condition that determines the system evolution. The initial condition depends on the three dimensional form of quantities such as the magnetic field in the entire sphere. For the magnetic field, conservation of the divergence-free condition for the adjoint magnetic field requires the introduction of an adjoint pressure term satisfying a zero boundary condition. We thus find that solving the forward and adjoint dynamo system requires different numerical algorithms. In this paper, an efficient algorithm for numerically solving this problem is developed and tested for two illustrative problems in a whole sphere: one is a kinematic problem with prescribed velocity field, and the second is associated with the Hall-effect dynamo, exhibiting considerable nonlinearity. The algorithm exhibits reliable numerical accuracy and stability. Using both the analytical and the numerical techniques of this paper, the adjoint dynamo system can be solved directly with the same order of computational complexity as that required to solve the forward problem. These numerical techniques form a foundation for ultimate application to observations of the geomagnetic field over the time scale of centuries

Azimuth sidelobes suppression using multi-azimuth angle synthetic aperture radar images

A novel method is proposed for azimuth sidelobes suppression using multi-pass squinted (MPS) synthetic aperture radar (SAR) data. For MPS SAR, the radar observes the scene with different squint angles and heights on each pass. The MPS SAR mode acquisition geometry is given first. Then, 2D signals are focused and the images are registered to the master image. Based on the new signal model, elevation processing and incoherent addition are introduced in detail, which are the main parts for azimuth sidelobes suppression. Moreover, parameter design criteria in incoherent addition are derived for the best performance. With the proposed parameter optimization step, the new method has a prominent azimuth sidelobes suppression effect with a slightly better azimuth resolution, as verified by experimental results on both simulated point targets and TerraSAR-X data

General relations of heavy quark-antiquark potentials induced by reparameterization invariance

A set of general relations between the spin-independent and spin-dependent potentials of heavy quark and anti-quark interactions are derived from reparameterization invariance in the Heavy Quark Effective Theory. It covers the Gromes relation and includes some new interesting relations which are useful in understanding the spin-independent and spin-dependent relativistic corrections to the leading order nonrelativistic potential.Comment: 11 pages, TUIMP-TH-93/54, CCAST-93-3