Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

Ionospheric simulator survey

Evaluation of D and E region ionospheric simulation technique

Atmospheric cloud physics laboratory project study

Engineering studies were performed for the Zero-G Cloud Physics Experiment liquid cooling and air pressure control systems. A total of four concepts for the liquid cooling system was evaluated, two of which were found to closely approach the systems requirements. Thermal insulation requirements, system hardware, and control sensor locations were established. The reservoir sizes and initial temperatures were defined as well as system power requirements. In the study of the pressure control system, fluid analyses by the Atmospheric Cloud Physics Laboratory were performed to determine flow characteristics of various orifice sizes, vacuum pump adequacy, and control systems performance. System parameters predicted in these analyses as a function of time include the following for various orifice sizes: (1) chamber and vacuum pump mass flow rates, (2) the number of valve openings or closures, (3) the maximum cloud chamber pressure deviation from the allowable, and (4) cloud chamber and accumulator pressure

Analysis of severe atmospheric disturbances from airline flight records

Advanced methods were developed to determine time varying winds and turbulence from digital flight data recorders carried aboard modern airliners. Analysis of several cases involving severe clear air turbulence encounters at cruise altitudes has shown that the aircraft encountered vortex arrays generated by destabilized wind shear layers above mountains or thunderstorms. A model was developed to identify the strength, size, and spacing of vortex arrays. This model is used to study the effects of severe wind hazards on operational safety for different types of aircraft. The study demonstrates that small remotely piloted vehicles and executive aircraft exhibit more violent behavior than do large airliners during encounters with high-altitude vortices. Analysis of digital flight data from the accident at Dallas/Ft. Worth in 1985 indicates that the aircraft encountered a microburst with rapidly changing winds embedded in a strong outflow near the ground. A multiple-vortex-ring model was developed to represent the microburst wind pattern. This model can be used in flight simulators to better understand the control problems in severe microburst encounters

Stability of the replica symmetric solution for the information conveyed by by a neural network

The information that a pattern of firing in the output layer of a feedforward network of threshold-linear neurons conveys about the network's inputs is considered. A replica-symmetric solution is found to be stable for all but small amounts of noise. The region of instability depends on the contribution of the threshold and the sparseness: for distributed pattern distributions, the unstable region extends to higher noise variances than for very sparse distributions, for which it is almost nonexistant.Comment: 19 pages, LaTeX, 5 figures. Also available at http://www.mrc-bbc.ox.ac.uk/~schultz/papers.html . Submitted to Phys. Rev. E Minor change

Rugged Metropolis Sampling with Simultaneous Updating of Two Dynamical Variables

The Rugged Metropolis (RM) algorithm is a biased updating scheme, which aims at directly hitting the most likely configurations in a rugged free energy landscape. Details of the one-variable (RM$_1$) implementation of this algorithm are presented. This is followed by an extension to simultaneous updating of two dynamical variables (RM$_2$). In a test with Met-Enkephalin in vacuum RM$_2$ improves conventional Metropolis simulations by a factor of about four. Correlations between three or more dihedral angles appear to prevent larger improvements at low temperatures. We also investigate a multi-hit Metropolis scheme, which spends more CPU time on variables with large autocorrelation times.Comment: 8 pages, 5 figures. Revisions after referee reports. Additional simulations for temperatures down to 220

Ocean Shrimp Report 1975 Season

Statewide Pacific ocean shrimp, Pandalus jordani, landings totaled 4,992,233 lb, a record annual catch that was double the 1974 catch of 2,382,821 lb. Landings for Areas A, B-1, B-2, and C were 3.4, 0.3, 1.2, and 0.06 million lb, respectively. Catch per hour trawled by single gear vessels ranged from 360 to 2,443 lb per hour for the 4 areas. The outlook is for good 1976 seasons in Areas A and B-1 if the moderately strong 1974 year class is complemented by strong recruitment of the 1975 year class. Prospects for Areas B-2 and C are not as promising as those of other areas due to weak 1974 year classes. (20pp.

Intrinsic Low Temperature Paramagnetism in B-DNA

We present experimental study of magnetization in $\lambda$-DNA in conjunction with structural measurements. The results show the surprising interplay between the molecular structures and their magnetic property. In the B-DNA state, $\lambda$-DNA exhibits paramagnetic behaviour below 20 K that is non-linear in applied magnetic field whereas in the A-DNA state, remains diamagnetic down to 2 K. We propose orbital paramagnetism as the origin of the observed phenomena and discuss its relation to the existence of long range coherent transport in B-DNA at low temperature.Comment: 5 pages, 4 figures, submitted to Physical Review Letters October 200