Stochastic Reinforcement Learning

In reinforcement learning episodes, the rewards and punishments are often non-deterministic, and there are invariably stochastic elements governing the underlying situation. Such stochastic elements are often numerous and cannot be known in advance, and they have a tendency to obscure the underlying rewards and punishments patterns. Indeed, if stochastic elements were absent, the same outcome would occur every time and the learning problems involved could be greatly simplified. In addition, in most practical situations, the cost of an observation to receive either a reward or punishment can be significant, and one would wish to arrive at the correct learning conclusion by incurring minimum cost. In this paper, we present a stochastic approach to reinforcement learning which explicitly models the variability present in the learning environment and the cost of observation. Criteria and rules for learning success are quantitatively analyzed, and probabilities of exceeding the observation cost bounds are also obtained.Comment: AIKE 201

Application of Conjugable Oxidation Products Assay in Assessment of Gamma-Irradiated Palm Olein

Samples ojpalm olein were irradiated with y-rays up to 12kGy. The extent ojperoxidation in irradiated samples was determined by conjugable oxidation products (COP) assay and the result were compared with the UV absorbance at 232 nm. The two parameters were poorly correlated (r = 0.6321) within the range ofdoses used. The ifJect ofy-irradiation is mainly to oxidise linoleic acid (C18:2) as this component is the major diunsaturatedJatty acid in palm oil

Preservation of Oil Palm Fruits: Nonoxidative Effects of Ionizing Radiation on Palm Olein and Crude Palm Oil

The effect of gamma-irradiation on palm oil has been investigated. Irradiation doses (0.1 to 1 MGj') caused severe destruction of unsaturated but had little effect on saturated fatty acids. Similar effects were obsemed in irradiated samples stored for 1 and 2 months at room temperature. Radiation (uP to 30 kGy) also caused severe destruction of carotenes in crude palm oil, but had no significant effect on the free fatty acid content. These findings indicate that gamma-radiation may be used for preservation but not for sterilization of palm fruits

Likelihood estimation for distributed parameter models for NASA Mini-MAST truss

A maximum likelihood estimation for distributed parameter models of large flexible structures was formulated. Distributed parameter models involve far fewer unknown parameters than independent modal characteristics or finite element models. The closed form solutions for the partial differential equations with corresponding boundary conditions were derived. The closed-form expressions of sensitivity functions led to highly efficient algorithms for analyzing ground or on-orbit test results. For an illustration of this approach, experimental data of the NASA Mini-MAST truss was used. The estimations of modal properties involve lateral bending modes and torsional modes. The results show that distributed parameter models are promising in the parameter estimation of large flexible structures

Gaussian approximations for stochastic systems with delay: chemical Langevin equation and application to a Brusselator system

We present a heuristic derivation of Gaussian approximations for stochastic chemical reaction systems with distributed delay. In particular we derive the corresponding chemical Langevin equation. Due to the non-Markovian character of the underlying dynamics these equations are integro-differential equations, and the noise in the Gaussian approximation is coloured. Following on from the chemical Langevin equation a further reduction leads to the linear-noise approximation. We apply the formalism to a delay variant of the celebrated Brusselator model, and show how it can be used to characterise noise-driven quasi-cycles, as well as noise-triggered spiking. We find surprisingly intricate dependence of the typical frequency of quasi-cycles on the delay period.Comment: 14 pages, 9 figure

Variational data assimilation for the initial-value dynamo problem

The secular variation of the geomagnetic field as observed at the Earth's surface results from the complex magnetohydrodynamics taking place in the fluid core of the Earth. One way to analyze this system is to use the data in concert with an underlying dynamical model of the system through the technique of variational data assimilation, in much the same way as is employed in meteorology and oceanography. The aim is to discover an optimal initial condition that leads to a trajectory of the system in agreement with observations. Taking the Earth's core to be an electrically conducting fluid sphere in which convection takes place, we develop the continuous adjoint forms of the magnetohydrodynamic equations that govern the dynamical system together with the corresponding numerical algorithms appropriate for a fully spectral method. These adjoint equations enable a computationally fast iterative improvement of the initial condition that determines the system evolution. The initial condition depends on the three dimensional form of quantities such as the magnetic field in the entire sphere. For the magnetic field, conservation of the divergence-free condition for the adjoint magnetic field requires the introduction of an adjoint pressure term satisfying a zero boundary condition. We thus find that solving the forward and adjoint dynamo system requires different numerical algorithms. In this paper, an efficient algorithm for numerically solving this problem is developed and tested for two illustrative problems in a whole sphere: one is a kinematic problem with prescribed velocity field, and the second is associated with the Hall-effect dynamo, exhibiting considerable nonlinearity. The algorithm exhibits reliable numerical accuracy and stability. Using both the analytical and the numerical techniques of this paper, the adjoint dynamo system can be solved directly with the same order of computational complexity as that required to solve the forward problem. These numerical techniques form a foundation for ultimate application to observations of the geomagnetic field over the time scale of centuries

Data assimilation in a sparsely observed one-dimensional modeled MHD system

A one dimensional non-linear magneto-hydrodynamic (MHD) system has been introduced to test a sequential optimal interpolation assimilation technique that uses a Monte-Carlo method to calculate the forecast error covariance. An ensemble of 100 model runs with perturbed initial conditions are used to construct the covariance, and the assimilation algorithm is tested using Observation Simulation Experiments (OSE's). The system is run with a variety of observation types (magnetic and/or velocity fields) and a range of observation densities. The impact of cross covariances between velocity and magnetic fields is investigated by running the assimilation with and without these terms. Sets of twin experiments show that while observing both velocity and magnetic fields has the greatest positive impact on the system, observing the magnetic field alone can also effectively constrain the system. Observations of the velocity field are ineffective as a constraint on the magnetic field, even when observations are made at every point. The implications for geomagnetic data assimilation are discussed

Finite dimensional quantizations of the (q,p) plane : new space and momentum inequalities

We present a N-dimensional quantization a la Berezin-Klauder or frame quantization of the complex plane based on overcomplete families of states (coherent states) generated by the N first harmonic oscillator eigenstates. The spectra of position and momentum operators are finite and eigenvalues are equal, up to a factor, to the zeros of Hermite polynomials. From numerical and theoretical studies of the large $N$ behavior of the product $\lambda\_m(N) \lambda\_M(N)$ of non null smallest positive and largest eigenvalues, we infer the inequality $\delta\_N(Q) \Delta\_N(Q) = \sigma\_N \overset{<}{\underset{N \to \infty}{\to}} 2 \pi$ (resp. $\delta\_N(P) \Delta\_N(P) = \sigma\_N \overset{<}{\underset{N \to \infty}{\to}} 2 \pi$) involving, in suitable units, the minimal ($\delta\_N(Q)$) and maximal ($\Delta\_N(Q)$) sizes of regions of space (resp. momentum) which are accessible to exploration within this finite-dimensional quantum framework. Interesting issues on the measurement process and connections with the finite Chern-Simons matrix model for the Quantum Hall effect are discussed

Search for Bc(ns)B_c(ns) via the Bc(ns)→Bc(ms)π+π−B_c(ns)\to B_c(ms)\pi^+\pi^- transition at LHCb and Z0Z_0 factory

It is interesting to study the characteristics of the whole family of $B_c$ which contains two different heavy flavors. LHC and the proposed $Z^0$ factory provide an opportunity because a large database on the $B_c$ family will be achieved. $B_c$ and its excited states can be identified via their decay modes. As suggested by experimentalists, $B_c^*(ns)\to B_c+\gamma$ is not easy to be clearly measured, instead, the trajectories of $\pi^+$ and $\pi^-$ occurring in the decay of $B_c(ns)\to B_c(ms)+\pi^+\pi^-$ ($n>m$) can be unambiguously identified, thus the measurement seems easier and more reliable, therefore this mode is more favorable at early running stage of LHCb and the proposed $Z^0$ factory. In this work, we calculate the rate of $B_c(ns)\to B_c(ms)+\pi^+\pi^-$ in terms of the QCD multipole-expansion and the numerical results indicate that the experimental measurements with the luminosity of LHC and $Z^0$ factory are feasible.Comment: 12 pages, 1 figures and 4 tables, acceptted by SCIENCE CHINA Physics, Mechanics & Astronomy (Science in China Series G

Discovery From Non-Parties (Third-Party Discovery) in International Arbitration

International arbitration rules and many arbitration laws usually provide procedures that permit tribunals to order parties to disclose documents and other materials to the other parties.1 More complex are the rules that determine opportunities to obtain discovery from persons that are not party to the arbitration (third-party discovery). This article will review third-party discovery under the Federal Arbitration Act (FAA) and the provisions of the US Code s.1782 that authorise US courts to act in aid of actions before foreign tribunals. Section 1782 has unique interest at this time because it figured prominently in the EU antitrust investigation of Intel that was initiated on request from Advanced Micro Devices (AMD). Early in that investigation, AMD filed a s.1782 request in the US District Court to obtain evidence from US sources for submission to the DG-Competition of the European Commission (EC). This request ultimately led to the Supreme Court’s decision in Intel Corp v Advanced Micro Devices Inc2 which appeared to significantly expand the scope of s.1782. Ironically, after AMD won on key legal issues in the Supreme Court, the District Court on remand exercised its discretion and denied the request for judicial assistance. This paper first describes the FAA non-party discovery rules and the split among the federal appellate courts concerning the authority of arbitrators to order prehearing discovery from non-parties. Next, it provides an analysis of the meaning of the terms “interested party” and “tribunal”—terms that were controversially interpreted by the Supreme Court in Intel and are essential to the application of s.1782. Finally, it discusses the “discretionary” factors used by the federal courts in deciding whether to grant a s.1782 request even when the statutory criteria are met. The opportunity to exercise this discretion seems to rebut the argument that the Supreme Court’s interpretation of s.1782 gives participants before foreign tribunals more discovery rights in the United States than are available to the parties in arbitrations covered by the FAA