The Importance of Clipping in Neurocontrol by Direct Gradient Descent on the Cost-to-Go Function and in Adaptive Dynamic Programming

In adaptive dynamic programming, neurocontrol and reinforcement learning, the objective is for an agent to learn to choose actions so as to minimise a total cost function. In this paper we show that when discretized time is used to model the motion of the agent, it can be very important to do "clipping" on the motion of the agent in the final time step of the trajectory. By clipping we mean that the final time step of the trajectory is to be truncated such that the agent stops exactly at the first terminal state reached, and no distance further. We demonstrate that when clipping is omitted, learning performance can fail to reach the optimum; and when clipping is done properly, learning performance can improve significantly. The clipping problem we describe affects algorithms which use explicit derivatives of the model functions of the environment to calculate a learning gradient. These include Backpropagation Through Time for Control, and methods based on Dual Heuristic Dynamic Programming. However the clipping problem does not significantly affect methods based on Heuristic Dynamic Programming, Temporal Differences or Policy Gradient Learning algorithms. Similarly, the clipping problem does not affect fixed-length finite-horizon problems

An experimental facility for sequential decoding Technical report no. 450

Experimental facility system design for sequential decoding used in communication theor

Zoom invariant vision of figural shape: The mathematics of cores

Believing that figural zoom invariance and the cross-figural boundary linking implied by medial loci are important aspects of object shape, we present the mathematics of and algorithms for the extraction of medial loci directly from image intensities. The medial loci called cores are defined as generalized maxima in scale space of a form of medial information that is invariant to translation, rotation, and in particular, zoom. These loci are very insensitive to image disturbances, in strong contrast to previously available medial loci, as demonstrated in a companion paper. Core-related geometric properties and image object representations are laid out which, together with the aforementioned insensitivities, allow the core to be used effectively for a variety of image analysis objectives.

A New Method for Structural Simulation

In this paper structural change is defined and a tool to simulate structural changes is introduced which consists of a new simulation language which allows to deal separately with quantitative changes and structural qualitative changes. Two strategies of structural simulation are described. In the first one, the user defines the possible structures and conditions of change. In this case, the simulation process finds the structural paths through successive structures. In the second strategy, the structures are generated by the simulation process based on the model of creative thinking proposed by Poincare and Hadamard. AI and genetic programming techniques are used to implement the model. A simple example is given to illustrate the method of the second strategy

Vlsi Implementation of Olfactory Cortex Model

This thesis attempts to implement the building blocks required for the realization of the biologically motivated olfactory neural model in silicon as the special purpose hardware. The olfactory model is originally developed by R. Granger, G. Lynch, and Ambros-Ingerson. CMOS analog integrated circuits were used for this purpose. All of the building blocks were fabricated using the MOSIS service and tested at our site. The results of this study can be used to realize a system level integration of the olfactory model.Electrical Engineerin

Development of a color display capability

Color display capabilities for digital computer

The development and evaluation of computer vision algorithms for the control of an autonomous horticultural vehicle

Economic and environmental pressures have led to a demand for reduced chemical use in crop production. In response to this, precision agriculture techniques have been developed that aim to increase the efficiency of farming operations by more targeted application of chemical treatment. The concept of plant scale husbandry (PSH) has emerged as the logical extreme of precision techniques, where crop and weed plants are treated on an individual basis. To investigate the feasibility of PSH, an autonomous horticultural vehicle has been developed at the Silsoe Research Institute. This thesis describes the development of computer vision algorithms for the experimental vehicle which aim to aid navigation in the field and also allow differential treatment of crop and weed. The algorithm, based upon an extended Kalman filter, exploits the semi-structured nature of the field environment in which the vehicle operates, namely the grid pattern formed by the crop planting. By tracking this grid pattern in the images captured by the vehicles camera as it traverses the field, it is possible to extract information to aid vehicle navigation, such as bearing and offset from the grid of plants. The grid structure can also act as a cue for crop/weed discrimination on the basis of plant position on the ground plane. In addition to tracking the grid pattern, the Kalman filter also estimates the mean distances between the rows of lines and plants in the grid, to cater for variations in the planting procedure. Experiments are described which test the localisation accuracy of the algorithms in offline trials with data captured from the vehicle's camera, and on-line in both a simplified testbed environment and the field. It is found that the algorithms allow safe navigation along the rows of crop. Further experiments demonstrate the crop/weed discrimination performance of the algorithm, both off-line and on-line in a crop treatment experiment performed in the field where all of the crop plants are correctly targeted and no weeds are mistakenly treated