Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

Tests of a full-scale annular ram-induction combustor for a Mach 3 cruise turbojet engine

Full-scale annular ram-induction combustor tests for Mach 3 cruise turbojet engin

Full scale tests of a short length, double annular ram induction turbojet combustor for supersonic flight

Performance tests and characteristics of short length, double annular ram induction turbojet combustion chambers for supersonic fligh

Properties of optically selected BL Lac candidates from the SDSS

\textbf{Context.} Deep optical surveys open the avenue for find large numbers of BL Lac objects that are hard to identify because they lack the unique properties classifying them as such. While radio or X-ray surveys typically reveal dozens of sources, recent compilations based on optical criteria alone have increased the number of BL Lac candidates considerably. However, these compilations are subject to biases and may contain a substantial number of contaminating sources. \textbf{Aims.} In this paper we extend our analysis of 182 optically selected BL Lac object candidates from the SDSS with respect to an earlier study. The main goal is to determine the number of bona fide BL Lac objects in this sample. \textbf{Methods.} We examine their variability characteristics, determine their broad-band radio-UV SEDs, and search for the presence of a host galaxy. In addition we present new optical spectra for 27 targets with improved S/N with respect to the SDSS spectra. \textbf{Results.} At least 59% of our targets have shown variability between SDSS DR2 and our observations by more than 0.1-0.27 mag de- pending on the telescope used. A host galaxy was detected in 36% of our targets. The host galaxy type and luminosities are consistent with earlier studies of BL Lac host galaxies. Simple fits to broad-band SEDS for 104 targets of our sample derived synchrotron peak frequencies between $13.5 \leq \mathrm{log}_{10}(\nu_{\mathrm{peak}}) \leq 16$ with a peak at $\mathrm{log}_{10} \sim 14.5$. Our new optical spectra do not reveal any new redshift for any of our objects. Thus the sample contains a large number of bona fide BL Lac objects and seems to contain a substantial fraction of intermediate-frequency peaked BL Lacs.Comment: Accepted for publication in A\&

Combustion of coal gas fuels in a staged combustor

Gaseous fuels produced from coal resources generally have heating values much lower than natural gas; the low heating value could result in unstable or inefficient combustion. Coal gas fuels may contain ammonia which if oxidized in an uncontrolled manner could result in unacceptable nitrogen oxide exhaust emission levels. Previous investigations indicate that staged, rich-lean combustion represents a desirable approach to achieve stable, efficient, low nitrogen oxide emission operation for coal-derived liquid fuels contaning up to 0.8-wt pct nitrogen. An experimental program was conducted to determine whether this fuel tolerance can be extended to include coal-derived gaseous fuels. The results of tests with three nitrogen-free fuels having heating values of 100, 250, and 350 Btu/scf and a 250 Btu/scf heating value doped to contain 0.7 pct ammonia are presented

Infection of Soybean and Pea Nodules by \u3cem\u3eRhizobium\u3c/em\u3e spp. Purine Auxotrophs in the Presence of 5-aminoimidazole-4-Carboxamide Riboside

Purine auxotrophs of various Rhizobium species are symbiotically defective, usually unable to initiate or complete the infection process. Earlier studies demonstrated that, in the Rhizobium etli-bean symbiosis, infection by purine auxotrophs is partially restored by supplementation of the plant medium with 5-amino-imidazole-4-carboxamide (AICA) riboside, the unphosphorylated form of the purine biosynthetic intermediate AICAR. The addition of purine to the root environment does not have this effect. In this study, purine auxotrophs of Rhizobium fredii HH303 and Rhizobium leguminosarum 128C56 (bv. viciae) were examined. Nutritional and genetic characterization indicated that each mutant was blocked in purine biosynthesis prior to the production of AICAR. R. fredii HH303 and R. leguminosarum 128C56 appeared to be deficient in AICA riboside transport and/or conversion into AICAR, and the auxotrophs derived from them grew very poorly with AICA riboside as a purine source. All of the auxotrophs elicited poorly developed, uninfected nodules on their appropriate hosts. On peas, addition of AICA riboside or purine to the root environment led to enhanced nodulation; however, infection threads were observed only in the presence of AICA riboside. On soybeans, only AICA riboside was effective in enhancing nodulation and promoting infection. Although AICA riboside supplementation of the auxotrophs led to infection thread development on both hosts, the numbers of bacteria recovered from the nodules were still 2 or more orders of magnitude lower than in fully developed nodules populated by wild-type bacteria. The ability to AICA riboside to promote infection by purine auxotrophs, despite serving as a very poor purine source for these strains, supports the hypothesis that AICAR plays a role in infection other than merely promoting bacterial growth

Reversible strain effect on the magnetization of LaCoO3 films

The magnetization of ferromagnetic LaCoO3 films grown epitaxially on piezoelectric substrates has been found to systematically decrease with the reduction of tensile strain. The magnetization change induced by the reversible strain variation reveals an increase of the Co magnetic moment with tensile strain. The biaxial strain dependence of the Curie temperature is estimated to be below 4K/% in the as-grown tensile strain state of our films. This is in agreement with results from statically strained films on various substrates

Spectroscopic studies of individual plasmon resonant nanoparticles

We present a detailed description of the apparatus and techniques that we have utilized in our experimental study of individual plasmon resonant nanoparticles, along with a brief description of some major results. The apparatus consists of a spectroscopic system combined with a modified darkfield microscope, which enables the user to sequentially select individual resonant nanostructures in the microscopic field of view for spectroscopic study. Plasmon resonant nanostructures scatter light elastically, and typically have very large scattering cross-sections at their resonant optical wavelengths. In general, spectra can be obtained with acquisition times between .1 to 30 seconds, and color images can be captured using consumer digital color cameras. Spheres, tetrahedrons, and pentagonal platelets were fabricated using colloidal chemistry techniques. To produce highly anisotropic structures such as nanorods and "barbells", templates were used. Many of these nanostructures have been individually spectroscopically characterized, and their spectra correlated with their shape and size as determined by transmission electron microscope (TEM). The unique shape, size, composition, and dielectric surroundings of the individual plasmon resonant nanostructures determine their plasmon resonant behavior. We will show how the composition of the substrate on which the particles are immobilized and the dielectric of the surrounding medium have a significant effect on the plasmon resonance of the individual particles

The effect of parallel static and microwave electric fields on excited hydrogen atoms

Motivated by recent experiments we analyse the classical dynamics of a hydrogen atom in parallel static and microwave electric fields. Using an appropriate representation and averaging approximations we show that resonant ionisation is controlled by a separatrix, and provide necessary conditions for a dynamical resonance to affect the ionisation probability. The position of the dynamical resonance is computed using a high-order perturbation series, and estimate its radius of convergence. We show that the position of the dynamical resonance does not coincide precisely with the ionisation maxima, and that the field switch-on time can dramatically affect the ionisation signal which, for long switch times, reflects the shape of an incipient homoclinic. Similarly, the resonance ionisation time can reflect the time-scale of the separatrix motion, which is therefore longer than conventional static field Stark ionisation. We explain why these effects should be observed in the quantum dynamics. PACs: 32.80.Rm, 33.40.+f, 34.10.+x, 05.45.Ac, 05.45.MtComment: 47 pages, 20 figure