Adaptive Regularization in Neural Network Modeling

. In this paper we address the important problem of optimizing regularization parameters in neural network modeling. The suggested optimization scheme is an extended version of the recently presented algorithm [24]. The idea is to minimize an empirical estimate -- like the cross-validation estimate -- of the generalization error with respect to regularization parameters. This is done by employing a simple iterative gradient descent scheme using virtually no additional programming overhead compared to standard training. Experiments with feed-forward neural network models for time series prediction and classification tasks showed the viability and robustness of the algorithm. Moreover, we provided some simple theoretical examples in order to illustrate the potential and limitations of the proposed regularization framework. 1 Introduction Neural networks are flexible tools for time series processing and pattern recognition. By increasing the number of hidden neurons in a 2-layer architec..

EEG source imaging assists decoding in a face recognition task

EEG based brain state decoding has numerous applications. State of the art decoding is based on processing of the multivariate sensor space signal, however evidence is mounting that EEG source reconstruction can assist decoding. EEG source imaging leads to high-dimensional representations and rather strong a priori information must be invoked. Recent work by Edelman et al. (2016) has demonstrated that introduction of a spatially focal source space representation can improve decoding of motor imagery. In this work we explore the generality of Edelman et al. hypothesis by considering decoding of face recognition. This task concerns the differentiation of brain responses to images of faces and scrambled faces and poses a rather difficult decoding problem at the single trial level. We implement the pipeline using spatially focused features and show that this approach is challenged and source imaging does not lead to an improved decoding. We design a distributed pipeline in which the classifier has access to brain wide features which in turn does lead to a 15% reduction in the error rate using source space features. Hence, our work presents supporting evidence for the hypothesis that source imaging improves decoding

Juvenile methylphenidate reduces prefrontal cortex plasticity via D3 receptor and BDNF in adulthood

Background:: Early drug intervention in childhood disorders aims to maximize individual potential in the short- and long-term. Consistently, juvenile exposure to psychostimulants, such as methylphenidate (MPH), reduces risk for substance use in animals and sub-populations of individuals with attention deficit hyperactivity disorder (ADHD). We investigated the effects of MPH on brain plasticity via dopamine receptor D3 (D3R) and brain-derived neurotrophic factor (BDNF) expression in developing rats. Methods:: Between postnatal days 20–35, rat pups were administered saline vehicle (Veh) or MPH (2 mg/kg), the D3R-preferring agonist ±7-OHDPAT, or the antagonist nafadotride (0.05 mg/kg) alone, or in combination with MPH twice a day. In adulthood, subjects were challenged to Veh or cocaine (10 mg/kg for two days). The prefrontal cortex was analyzed for protein and mRNA levels of total BDNF, its splice variants I, IIc, III/IV, and IV/VI, and D3 receptors. A separate group of subjects was assessed for splice variants at 20, 35, 40, and 60 days of age. Results:: Across age strong correlations were evident between Drd3 and Bdnf mRNA levels (r = 0.65) and a negative relationship between Drd3 and exon IIc after MPH treatment (r = −0.73). BDNF protein levels did not differ between Veh- and MPH subjects at baseline, but were significantly lower in MPH-treated and cocaine challenged subjects (30.3 ± 9.7%). Bdnf mRNA was significantly higher in MPH-treated subjects, and reversed upon exposure to cocaine. This effect was blocked by nafadotride. Furthermore, Bdnftotal and Bdnf splice variants I, IIc, III/IV, and IV/VI changed across the transitions between juvenility and late adolescence. Conclusions:: These data suggest a sensitive window of vulnerability to modulation of BDNF expression around adolescence, and that compared to normal animals, juvenile exposure to MPH permanently reduces prefrontal BDNF transcription and translation upon cocaine exposure in adulthood by a D3R-mediated mechanism

Revisiting Boltzmann learning: parameter estimation in Markov random fields

This contribution concerns a generalization of the Boltzmann Machine that allows us to use the learning rule for a much wider class of maximum likelihood and maximum a posteriori problems, including both supervised and unsupervised learning. Furthermore, the approach allows us to discuss regularization and generalization in the context of Boltzmann Machines. We provide an illustrative example concerning parameter estimation in an inhomogeneous Markov Field

Sonar discrimination of cylinders from different angles using neural networks neural networks

This paper describes an underwater object discrimination system applied to recognize cylinders of various compositions from different angles. The system is based on a new combination of simulated dolphin clicks, simulated auditory filters and artificial neural networks. The model demonstrates its potential on real data collected from four different cylinders in an environment where the angles were controlled in order to evaluate the models capabilities to recognize cylinders independent of angles. 1. INTRODUCTION Dolphins possess an excellent sonar system for solving underwater target discrimination and recognition tasks in shallow water (see e.g., [2]). This has inspired research in new sonar systems based on biological knowledge, i.e. modeling the dolphins discrimination capabilities (see e.g., [4] and [5]). The fact that the inner ear of the dolphin has many similarities with the human inner ear makes it tempting to use knowledge from simulations of the human auditory system when t..

Design of Robust Neural Network Classifiers

This paper addresses a new framework for designing robust neural network classifiers. The network is optimized using the maximum a posteriori technique, i.e., the cost function is the sum of the log-likelihood and a regularization term (prior). In order to perform robust classification, we present a modified likelihood function which incorporate the potential risk of outliers in the data. This leads to introduction of a new parameter, the outlier probability. Designing the neural classifier involves optimization of network weights as well as outlier probability and regularization parameters. We suggest to adapt the outlier probability and regularization parameters by minimizing the error on a validation set, and a simple gradient descent scheme is derived. In addition, the framework allows for constructing a simple outlier detector. Experiments with artificial data demonstrates the potential of the suggested framework. 1. INTRODUCTION Neural networks are flexible tools for pattern rec..