Discretized Bayesian pursuit – A new scheme for reinforcement learning

The success of Learning Automata (LA)-based estimator algorithms over the classical, Linear Reward-Inaction ( L RI )-like schemes, can be explained by their ability to pursue the actions with the highest reward probability estimates. Without access to reward probability estimates, it makes sense for schemes like the L RI to first make large exploring steps, and then to gradually turn exploration into exploitation by making progressively smaller learning steps. However, this behavior becomes counter-intuitive when pursuing actions based on their estimated reward probabilities. Learning should then ideally proceed in progressively larger steps, as the reward probability estimates turn more accurate. This paper introduces a new estimator algorithm, the Discretized Bayesian Pursuit Algorithm (DBPA), that achieves this. The DBPA is implemented by linearly discretizing the action probability space of the Bayesian Pursuit Algorithm (BPA) [1]. The key innovation is that the linear discrete updating rules mitigate the counter-intuitive behavior of the corresponding linear continuous updating rules, by augmenting them with the reward probability estimates. Extensive experimental results show the superiority of DBPA over previous estimator algorithms. Indeed, the DBPA is probably the fastest reported LA to date

Regularity issues in the problem of fluid structure interaction

We investigate the evolution of rigid bodies in a viscous incompressible fluid. The flow is governed by the 2D Navier-Stokes equations, set in a bounded domain with Dirichlet boundary conditions. The boundaries of the solids and the domain have H\"older regularity $C^{1, \alpha}$, $0 < \alpha \le 1$. First, we show the existence and uniqueness of strong solutions up to collision. A key ingredient is a BMO bound on the velocity gradient, which substitutes to the standard $H^2$ estimate for smoother domains. Then, we study the asymptotic behaviour of one $C^{1, \alpha}$ body falling over a flat surface. We show that collision is possible in finite time if and only if $\alpha < 1/2$

Hypogene Calcitization: Evaporite Diagenesis in the Western Delaware Basin

Evaporite calcitization within the Castile Formation of the Delaware Basin is more widespread and diverse than originally recognized. Coupled field and GIS studies have identified more than 1000 individual occurrences of calcitization within the Castile Formation outcrop area, which includes both calcitized masses (limestone buttes) and laterally extensive calcitized horizons (limestone sheets). Both limestone buttes and sheets commonly contain a central brecciated zone that we attribute to hypogene dissolution. Lithologic fabric of calcitized zones ranges from little alteration of original varved laminae to fabrics showing extensive laminae distortion as well as extensive vuggy and open cavernous porosity. Calcitization is most abundant in the western portion of the Castile outcrop region where surface denudation has been greatest. Calcitization often forms linear trends, indicating fluid migration along fractures, but also occurs as dense clusters indicating focused, ascending, hydrocarbon-rich fluids. Native sulfur, secondary tabular gypsum (i.e. selenite) and hypogene caves are commonly associated with clusters of calcitization. This assemblage suggests that calcium sulfate diagenesis within the Castile Formation is dominated by hypogene speleogemesis

Phenotypic Plasticity and Contemporary Evolution in Introduced Populations: Evidence from Translocated Populations of White Sands Pupfish (Cyrpinodon tularosa)

Contemporary evolution has been shown in a few studies to be an important component of colonization ability, but seldom have researchers considered whether phenotypic plasticity facilitates directional evolution from the invasion event. In the current study, we evaluated body shape divergence of the New Mexico State-threatened White Sands pupfish (Cyprinodon tularosa) that were introduced to brackish, lacustrine habitats at two different time in the recent past (approximately 30 years and 1 year previously) from the same source population (saline river environment). Pupfish body shape is correlated with environmental salinity: fish from saline habitats are characterized by slender body shapes, whereas fish from fresher, yet brackish springs are deep-bodied. In this study, lacustrine populations consisted of an approximately 30-year old population and several 1-year old populations, all introduced from the same source. The body shape divergence of the 30-year old population was significant and greater than any of the divergences of the 1-year old populations (which were for the most part not significant). Nonetheless, all body shape changes exhibited body deepening in less saline environments. We conclude that phenotypic plasticity potentially facilitates directional evolution of body deepening for introduced pupfish populations

A Theoretical Model of a Molecular-Motor-Powered Pump

The motion of a cylindrical bead in a fluid contained within a two-dimensional channel is investigated using the boundary element method as a model of a biomolecular-motor-powered microfluidics pump. The novelty of the pump lies in the use of motor proteins (kinesin) to power the bead motion and the few moving parts comprising the pump. The performance and feasibility of this pump design is investigated using two model geometries: a straight channel, and a curved channel with two concentric circular walls. In the straight channel geometry, it is shown that increasing the bead radius relative to the channel width, increases the flow rate at the expense of increasing the force the kinesins must generate in order to move the bead. Pump efficiency is generally higher for larger bead radii, and larger beads can support higher imposed loads. In the circular channel geometry, it is shown that bead rotation modifies the force required to move the bead and that shifting the bead inward slightly reduces the required force. Bead rotation has a minimal effect on flow rate. Recirculation regions, which can develop between the bead and the channel walls, influence the stresses and force on the bead. These results suggest this pump design is feasible, and the kinesin molecules provide sufficient force to deliver pico- to atto- l/s flows.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/44478/1/10544_2005_Article_6168.pd