Contrastive learning and neural oscillations

The concept of Contrastive Learning (CL) is developed as a family of possible learning algorithms for neural networks. CL is an extension of Deterministic Boltzmann Machines to more general dynamical systems. During learning, the network oscillates between two phases. One phase has a teacher signal and one phase has no teacher signal. The weights are updated using a learning rule that corresponds to gradient descent on a contrast function that measures the discrepancy between the free network and the network with a teacher signal. The CL approach provides a general unified framework for developing new learning algorithms. It also shows that many different types of clamping and teacher signals are possible. Several examples are given and an analysis of the landscape of the contrast function is proposed with some relevant predictions for the CL curves. An approach that may be suitable for collective analog implementations is described. Simulation results and possible extensions are briefly discussed together with a new conjecture regarding the function of certain oscillations in the brain. In the appendix, we also examine two extensions of contrastive learning to time-dependent trajectories

Neural Computing in Quaternion Algebra

兵庫県立大学201

Modeling the self-organization of color selectivity in the visual cortex

How does the visual cortex represent and process color? Experimental evidence from macaque monkey suggests that cells selective for color are organized into small, spatially separated blobs in V1, and stripes in V2. This organization is strikingly different from that of orientation and ocular dominance maps, which consist of large, spatially contiguous patterns. In this dissertation, a self-organizing model of the early visual cortex is constructed using natural color image input. The modeled V1 develops realistic color-selective receptive fields, ocular dominance stripes, orientation maps, and color-selective regions, while the modeled V2 also creates realistic colorselective and orientation-selective neurons. V1 color-selective regions are generally located in the center of ocular dominance stripes as they are in biological maps; the model predicts that color-selective regions become more widespread in both cortical regions when the amount of color in the training images is increased. The model also predicts that in V1 there are three types of color-selective regions (red-selective, greenselective, and blue-selective), and that a unique cortical activation pattern exists for each of the HSV colors. In both V1 and V2, when regions of different color-selectivity are located nearby, bands of color form with gradually changing color preferences. The model also develops lateral connections between cells that are selective for similar orientations, matching previous experimental results, and predicts that cells selective for color primarily connect to other cells with similar chromatic preferences. Thus the model replicates the known data on the organization of color preferences in V1 and V2, provides a detailed explanation for how this structure develops and functions, and leads to concrete predictions to test in future experiments.Computer Science

Learning FCMs with multi-local and balanced memetic algorithms for forecasting industrial drying processes

In this paper, we propose a Fuzzy Cognitive Map (FCM) learning approach with a multi-local search in balanced memetic algorithms for forecasting industrial drying processes. The first contribution of this paper is to propose a FCM model by an Evolutionary Algorithm (EA), but the resulted FCM model is improved by a multi-local and balanced local search algorithm. Memetic algorithms can be tuned with different local search strategies (CMA-ES, SW, SSW and Simplex) and the balance of the effort between global and local search. To do this, we applied the proposed approach to the forecasting of moisture loss in industrial drying process. The thermal drying process is a relevant one used in many industrial processes such as food industry, biofuels production, detergents and dyes in powder production, pharmaceutical industry, reprography applications, textile industries, and others. This research also shows that exploration of the search space is more relevant than finding local optima in the FCM models tested

Accelerated convergence of neural network system identification algorithms via principal component analysis

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/76898/1/AIAA-1998-4440-926.pd