Adaptive Critic Designs

We discuss a variety of adaptive critic designs (ACDs) for neurocontrol. These are suitable for learning in noisy, nonlinear, and nonstationary environments. They have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Our discussion of these origins leads to an explanation of three design families: heuristic dynamic programming, dual heuristic programming, and globalized dual heuristic programming (GDHP). The main emphasis is on DHP and GDHP as advanced ACDs. We suggest two new modifications of the original GDHP design that are currently the only working implementations of GDHP. They promise to be useful for many engineering applications in the areas of optimization and optimal control. Based on one of these modifications, we present a unified approach to all ACDs. This leads to a generalized training procedure for ACD

Electron-electron interaction and charging effects in graphene quantum dots

We analyze charging effects in graphene quantum dots. Using a simple model, we show that, when the Fermi level is far from the neutrality point, charging effects lead to a shift in the electrostatic potential and the dot shows standard Coulomb blockade features. Near the neutrality point, surface states are partially occupied and the Coulomb interaction leads to a strongly correlated ground state which can be approximated by either a Wigner crystal or a Laughlin like wave function. The existence of strong correlations modify the transport properties which show non equilibrium effects, similar to those predicted for tunneling into other strongly correlated systems.Comment: Extended version accepted for publication at Phys. Rev.

Training Winner-Take-All Simultaneous Recurrent Neural Networks

The winner-take-all (WTA) network is useful in database management, very large scale integration (VLSI) design, and digital processing. The synthesis procedure of WTA on single-layer fully connected architecture with sigmoid transfer function is still not fully explored. We discuss the use of simultaneous recurrent networks (SRNs) trained by Kalman filter algorithms for the task of finding the maximum among N numbers. The simulation demonstrates the effectiveness of our training approach under conditions of a shared-weight SRN architecture. A more general SRN also succeeds in solving a real classification application on car engine data

Conservative Thirty Calendar Day Stock Prediction Using a Probabilistic Neural Network

We describe a system that predicts significant short-term price movement in a single stock utilizing conservative strategies. We use preprocessing techniques, then train a probabilistic neural network to predict only price gains large enough to create a significant profit opportunity. Our primary objective is to limit false predictions (known in the pattern recognition literature as false alarms). False alarms are more significant than missed opportunities, because false alarms acted upon lead to losses. We can achieve false alarm rates as low as 5.7% with the correct system design and parameterization

Adaptive Critic Design in Learning to Play Game of Go

This paper examines the performance of an HDP-type adaptive critic design (ACD) of the game Go. The game Go is an ideal problem domain for exploring machine learning; it has simple rules but requires complex strategies to play well. All current commercial Go programs are knowledge based implementations; they utilize input feature and pattern matching along with minimax type search techniques. But the extremely high branching factor puts a limit on their capabilities, and they are very weak compared to the relative strengths of other game programs like chess. In this paper, the Go-playing ACD consists of a critic network and an action network. The HDP type critic network learns to predict the cumulative utility function of the current board position from training games, and, the action network chooses a next move which maximizes critics next step cost-to-go. After about 6000 different training games against a public domain program, WALLY, the network (playing WHITE) began to win in some of the games and showed slow but steady improvements on test game

Neurocontroller Alternatives for Fuzzy Ball-and-Beam Systems with Nonuniform Nonlinear Friction

The ball-and-beam problem is a benchmark for testing control algorithms. Zadeh proposed (1994) a twist to the problem, which, he suggested, would require a fuzzy logic controller. This experiment uses a beam, partially covered with a sticky substance, increasing the difficulty of predicting the ball\u27s motion. We complicated this problem even more by not using any information concerning the ball\u27s velocity. Although it is common to use the first differences of the ball\u27s consecutive positions as a measure of velocity and explicit input to the controller, we preferred to exploit recurrent neural networks, inputting only consecutive positions instead. We have used truncated backpropagation through time with the node-decoupled extended Kalman filter (NDEKF) algorithm to update the weights in the networks. Our best neurocontroller uses a form of approximate dynamic programming called an adaptive critic design. A hierarchy of such designs exists. Our system uses dual heuristic programming (DHP), an upper-level design. To our best knowledge, our results are the first use of DHP to control a physical system. It is also the first system we know of to respond to Zadeh\u27s challenge. We do not claim this neural network control algorithm is the best approach to this problem, nor do we claim it is better than a fuzzy controller. It is instead a contribution to the scientific dialogue about the boundary between the two overlapping disciplines

Comparative Study of Stock Trend Prediction using Time Delay, Recurrent and Probabilistic Neural Networks

Three networks are compared for low false alarm stock trend predictions. Short-term trends, particularly attractive for neural network analysis, can be used profitably in scenarios such as option trading, but only with significant risk. Therefore, we focus on limiting false alarms, which improves the risk/reward ratio by preventing losses. To predict stock trends, we exploit time delay, recurrent, and probabilistic neural networks (TDNN, RNN, and PNN, respectively), utilizing conjugate gradient and multistream extended Kalman filter training for TDNN and RNN. We also discuss different predictability analysis techniques and perform an analysis of predictability based on a history of daily closing price. Our results indicate that all the networks are feasible, the primary preference being one of convenienc

A Statistical Solution to a Text Decoding Challenge Problem

Given an encoded unknown text message in the form of a three dimensional spatial series generated by the use of four smooth nonlinear functions, we use a method based on simple statistical reasoning to pick up samples for rebuilding the four functions. The estimated functions are then used to decode the sequence. The experimental results show that our method gives a nearly perfect decoding, enabling us to submit a 100% accurate solution to the IJCNN challenge proble

Nonlinear electromagnetic response of graphene: Frequency multiplication and the self-consistent-field effects

Graphene is a recently discovered carbon based material with unique physical properties. This is a monolayer of graphite, and the two-dimensional electrons and holes in it are described by the effective Dirac equation with a vanishing effective mass. As a consequence, electromagnetic response of graphene is predicted to be strongly non-linear. We develop a quasi-classical kinetic theory of the non-linear electromagnetic response of graphene, taking into account the self-consistent-field effects. Response of the system to both harmonic and pulse excitation is considered. The frequency multiplication effect, resulting from the non-linearity of the electromagnetic response, is studied under realistic experimental conditions. The frequency up-conversion efficiency is analysed as a function of the applied electric field and parameters of the samples. Possible applications of graphene in terahertz electronics are discussed.Comment: 14 pages, 7 figures, invited paper written for a special issue of JPCM "Terahertz emitters

The ocean's saltiness and its overturning

Here we explore the relationship between the mean salinity urn:x-wiley:grl:media:grl55555:grl55555-math-0001 of the ocean and the strength of its Atlantic and Pacific Meridional Overturning Circulations (AMOC and PMOC). We compare simulations performed with a realistically configured coarse‐grained ocean model, spanning a range of mean salinities. We find that the AMOC strength increases approximately linearly with urn:x-wiley:grl:media:grl55555:grl55555-math-0002. In contrast, the PMOC strength declines approximately linearly with urn:x-wiley:grl:media:grl55555:grl55555-math-0003 until it reaches a small background value similar to the present‐day ocean. Well‐established scaling laws for the overturning circulation explain both of these dependencies on urn:x-wiley:grl:media:grl55555:grl55555-math-0004