Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

Transition-metal interactions in aluminum-rich intermetallics

The extension of the first-principles generalized pseudopotential theory (GPT) to transition-metal (TM) aluminides produces pair and many-body interactions that allow efficient calculations of total energies. In aluminum-rich systems treated at the pair-potential level, one practical limitation is a transition-metal over-binding that creates an unrealistic TM-TM attraction at short separations in the absence of balancing many-body contributions. Even with this limitation, the GPT pair potentials have been used effectively in total-energy calculations for Al-TM systems with TM atoms at separations greater than 4 AA. An additional potential term may be added for systems with shorter TM atom separations, formally folding repulsive contributions of the three- and higher-body interactions into the pair potentials, resulting in structure-dependent TM-TM potentials. Towards this end, we have performed numerical ab-initio total-energy calculations using VASP (Vienna Ab Initio Simulation Package) for an Al-Co-Ni compound in a particular quasicrystalline approximant structure. The results allow us to fit a short-ranged, many-body correction of the form a(r_0/r)^{b} to the GPT pair potentials for Co-Co, Co-Ni, and Ni-Ni interactions.Comment: 18 pages, 5 figures, submitted to PR

The Enigmatic Radio Afterglow of GRB 991216

We present wide-band radio observations spanning from 1.4 GHz to 350 GHz of the afterglow of GRB 991216, taken from 1 to 80 days after the burst. The optical and X-ray afterglow of this burst were fairly typical and are explained by a jet fireball. In contrast, the radio light curve is unusual in two respects: (a) the radio light curve does not show the usual rise to maximum flux on timescales of weeks and instead appears to be declining already on day 1 and (b) the power law indices show significant steepening from the radio through the X-ray bands. We show that the standard fireball model, in which the afterglow is from a forward shock, is unable to account for (b) and we conclude that the bulk of the radio emission must arise from a different source. We consider two models, neither of which can be ruled out with the existing data. In the first (conventional) model, the early radio emission is attributed to emission from the reverse shock as in the case of GRB 990123. We predict that the prompt optical emission would have been as bright (or brighter) than 8th magnitude. In the second (exotic) model, the radio emission originates from the forward shock of an isotropically energetic fireball (10^54 erg) expanding into a tenuous medium (10^-4 cm^-3). The resulting fireball would remain relativistic for months and is potentially resolvable with VLBI techniques. Finally, we note that the near-IR bump of the afterglow is similar to that seen in GRB 971214 and no fireball model can explain this bump.Comment: ApJ, submitte

HI ``Tails'' from Cometary Globules in IC1396

IC 1396 is a relatively nearby (750 pc), large (>2 deg), HII region ionized by a single O6.5V star and containing bright-rimmed cometary globules. We have made the first arcmin resolution images of atomic hydrogen toward IC 1396, and have found remarkable ``tail''-like structures associated with some of the globules and extending up to 6.5 pc radially away from the central ionizing star. These HI ``tails'' may be material which has been ablated from the globule through ionization and/or photodissociation and then accelerated away from the globule by the stellar wind, but which has since drifted into the ``shadow'' of the globules. This report presents the first results of the Galactic Plane Survey Project recently begun by the Dominion Radio Astrophysical Observatory.Comment: 11 pages, 5 postscript figures, uses aaspp4.sty macros, submitted in uuencoded gzipped tar format, accepted for publication in Astrophysical Journal Letters, colour figures available at http://www.drao.nrc.ca/~schieven/news_sep95/ic1396.htm

Large-Area Mapping at 850 Microns. I. Optimum Image Reconstruction from Chop Measurements

We present results on the optimum reconstruction of chop data taken using the Submillimeter Common-User Bolometer Array on the James Clerk Maxwell Telescope. Using an artificial data set with known noise properties, we analyze three techniques for constructing images of the sky from the chop data: Emerson Fourier deconvolution, matrix inversion, and maximum entropy reconstruction. We conclude that a matrix inversion formulation via an iterative procedure produces the best image reconstructions. We apply the three reconstruction techniques to produce maps of the calibration point source CRL 618 and the ? Ophiuchi A core at 850 ?m and use Wiener filtering to remove the high-frequency noise component from the matrix inversion method

The Broadband Afterglow of GRB980329

We present radio observations of the afterglow of the bright gamma-ray burst GRB980329 made between one month and several years after the burst, a re-analysis of previously published submillimeter data, and late-time optical and near-infrared (NIR) observations of the host galaxy. From the absence of a spectral break in the optical/NIR colors of the host galaxy, we exclude the earlier suggestion that GRB980329 lies at a redshift of z >~5. We combine our data with the numerous multi-wavelength observations of the early afterglow, fit a comprehensive afterglow model to the entire broadband dataset, and derive fundamental physical parameters of the blast-wave and its host environment. Models for which the ejecta expand isotropically require both a high circumburst density and extreme radiative losses from the shock. No low density model (n << 10 cm^{-3}) fits the data. A burst with a total energy of ~ 10^{51} erg, with the ejecta narrowly collimated to an opening angle of a few degrees, driven into a surrounding medium with density ~ 20 cm^{-3}, provides a satisfactory fit to the lightcurves over a range of redshifts.Comment: 27 pages, incl. 6 figures, minor revisions (e.g. added/updated references) Accepted by Ap

Curie: Constraining Solar System Bombardment Using In Situ Radiometric Dating

The Curie mission would constrain the existence of the putative cataclysm by determining the age of samples directly sourced from the impact melt sheet of a major pre-Imbrium lunar basin. The measurements would also enable further understanding of lunar evolution by characterizing new lunar lithologies far from the Apollo and Luna landing sites, including the very low-Ti basalts in Mare Crisium and potential olivine rich lithologies in the margins of both Mare Nectaris and Mars Crisium. Equipped with a mass spectrometer and a LIBS, Curie would also be well-placed to survey volatile components of the lunar regolith, including surface-bound hydrogen

The Computational Complexity of the Lorentz Lattice Gas

The Lorentz lattice gas is studied from the perspective of computational complexity theory. It is shown that using massive parallelism, particle trajectories can be simulated in a time that scales logarithmically in the length of the trajectory. This result characterizes the ``logical depth" of the Lorentz lattice gas and allows us to compare it to other models in statistical physics.Comment: 9 pages, LaTeX, to appear in J. Stat. Phy

Discovery of Early Optical Emission from GRB 021211

We report our discovery and early time optical, near-infrared, and radio wavelength follow-up observations of the afterglow of the gamma-ray burst GRB 021211. Our optical observations, beginning 21 min after the burst trigger, demonstrate that the early afterglow of this burst is roughly three magnitudes fainter than the afterglow of GRB 990123 at similar epochs, and fainter than almost all known afterglows at an epoch of 1d after the GRB. Our near-infrared and optical observations indicate that this is not due to extinction. Combining our observations with data reported by other groups, we identify the signature of a reverse shock. This reverse shock is not detected to a 3-sigma limit of 110 uJy in an 8.46-GHz VLA observation at t=0.10d, implying either that the Lorentz factor of the burst gamma <~ 200, or that synchrotron self-absorption effects dominate the radio emission at this time. Our early optical observations, near the peak of the optical afterglow (forward shock), allow us to characterize the afterglow in detail. Comparing our model to flux upper limits from the VLA at later times, t >~ 1 week, we find that the late-time radio flux is suppressed by a factor of two relative to the >~ 80 uJy peak flux at optical wavelengths. This suppression is not likely to be due to synchrotron self-absorption or an early jet break, and we suggest instead that the burst may have suffered substantial radiative corrections.Comment: 13 pages, 2 figures, ApJL accepted; edits for lengt