Avoiding overfitting of multilayer perceptrons by training derivatives

Resistance to overfitting is observed for neural networks trained with extended backpropagation algorithm. In addition to target values, its cost function uses derivatives of those up to the $4^{\mathrm{th}}$ order. For common applications of neural networks, high order derivatives are not readily available, so simpler cases are considered: training network to approximate analytical function inside 2D and 5D domains and solving Poisson equation inside a 2D circle. For function approximation, the cost is a sum of squared differences between output and target as well as their derivatives with respect to the input. Differential equations are usually solved by putting a multilayer perceptron in place of unknown function and training its weights, so that equation holds within some margin of error. Commonly used cost is the equation's residual squared. Added terms are squared derivatives of said residual with respect to the independent variables. To investigate overfitting, the cost is minimized for points of regular grids with various spacing, and its root mean is compared with its value on much denser test set. Fully connected perceptrons with six hidden layers and $2\cdot10^{4}$, $1\cdot10^{6}$ and $5\cdot10^{6}$ weights in total are trained with Rprop until cost changes by less than 10% for last 1000 epochs, or when the $10000^{\mathrm{th}}$ epoch is reached. Training the network with $5\cdot10^{6}$ weights to represent simple 2D function using 10 points with 8 extra derivatives in each produces cost test to train ratio of $1.5$, whereas for classical backpropagation in comparable conditions this ratio is $2\cdot10^{4}$

The plant traits that drive ecosystems: Evidence from three continents.

Question: A set of easily‐measured (‘soft’) plant traits has been identified as potentially useful predictors of ecosystem functioning in previous studies. Here we aimed to discover whether the screening techniques remain operational in widely contrasted circumstances, to test for the existence of axes of variation in the particular sets of traits, and to test for their links with ‘harder’ traits of proven importance to ecosystem functioning. Location: central‐western Argentina, central England, northern upland Iran, and north‐eastern Spain. Recurrent patterns of ecological specialization: Through ordination of a matrix of 640 vascular plant taxa by 12 standardized traits, we detected similar patterns of specialization in the four floras. The first PCA axis was identified as an axis of resource capture, usage and release. PCA axis 2 appeared to be a size‐related axis. Individual PCA for each country showed that the same traits remained valuable as predictors of resource capture and utilization in all of them, despite their major differences in climate, biogeography and land‐use. The results were not significantly driven by particular taxa: the main traits determining PCA axis 1 were very similar in eudicotyledons and monocotyledons and Asteraceae, Fabaceae and Poaceae. Links between recurrent suites of ‘soft’ traits and ‘hard’ traits: The validity of PCA axis 1 as a key predictor of resource capture and utilization was tested by comparisons between this axis and values of more rigorously established predictors (‘hard’ traits) for the floras of Argentina and England. PCA axis 1 was correlated with variation in relative growth rate, leaf nitrogen content, and litter decomposition rate. It also coincided with palatability to model generalist herbivores. Therefore, location on PCA axis 1 can be linked to major ecosystem processes in those habitats where the plants are dominant. Conclusion: We confirm the existence at the global scale of a major axis of evolutionary specialization, previously recognised in several local floras. This axis reflects a fundamental trade‐off between rapid acquisition of resources and conservation of resources within well‐protected tissues. These major trends of specialization were maintained across different environmental situations (including differences in the proximate causes of low productivity, i.e. drought or mineral nutrient deficiency). The trends were also consistent across floras and major phylogenetic groups, and were linked with traits directly relevant to ecosystem processes.Fil: Díaz, Sandra Myrna. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto Multidisciplinario de Biología Vegetal. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas Físicas y Naturales. Instituto Multidisciplinario de Biología Vegetal; ArgentinaFil: Hodgson, J.G.. The University. Department of Animal and Plant Sciences. Unit of Comparative Plant Ecology; Reino UnidoFil: Thompson, K.. The University. Department of Animal and Plant Sciences. Unit of Comparative Plant Ecology; Reino UnidoFil: Cabido, Marcelo Ruben. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto Multidisciplinario de Biología Vegetal. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas Físicas y Naturales. Instituto Multidisciplinario de Biología Vegetal; ArgentinaFil: Cornelissen, Johannes H. C.. Free University. Faculty Earth and Life Sciences. Department of Systems Ecology; Países BajosFil: Funes, Guillermo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto Multidisciplinario de Biología Vegetal. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas Físicas y Naturales. Instituto Multidisciplinario de Biología Vegetal; ArgentinaFil: Pérez Harguindeguy, Natalia. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto Multidisciplinario de Biología Vegetal. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas Físicas y Naturales. Instituto Multidisciplinario de Biología Vegetal; ArgentinaFil: Vendramini, Fernanda. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto Multidisciplinario de Biología Vegetal. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas Físicas y Naturales. Instituto Multidisciplinario de Biología Vegetal; ArgentinaFil: Falczuk, Valeria. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto Multidisciplinario de Biología Vegetal. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas Físicas y Naturales. Instituto Multidisciplinario de Biología Vegetal; ArgentinaFil: Zak, Marcelo Román. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto Multidisciplinario de Biología Vegetal. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas Físicas y Naturales. Instituto Multidisciplinario de Biología Vegetal; ArgentinaFil: Khoshnevi, M.. Research Institute of Forests and Rangelands; IránFil: Pérez Rontomé, M. C.. Instituto Pirenaico de Ecología; EspañaFil: Shirvani, F. A.. Research Institute of Forests and Rangelands; IránFil: Yazdani, S.. Research Institute of Forests and Rangelands; IránFil: Abbas Azimi, R. Research Institute of Forests and Rangelands; IránFil: Bogaard, A. The University. Department of Archaeology and Prehistory; Reino UnidoFil: Boustani, S.. Research Institute of Forests and Rangelands; IránFil: Charles, M.. The University. Department of Archaeology and Prehistory; Reino UnidoFil: Dehghan, M.. Research Institute of Forests and Rangelands; IránFil: de Torres Espuny, L.. Instituto Pirenaico de Ecología; EspañaFil: Guerrero Campo, J.. Instituto Pirenaico de Ecología; EspañaFil: Hynd, A.. The University. Department of Archaeology and Prehistory; Reino UnidoFil: Jones, G.. The University. Department of Archaeology and Prehistory; Reino UnidoFil: Kowsary, E.. Research Institute of Forests and Rangelands; Irán. Instituto Pirenaico de Ecología; EspañaFil: Kazemi Saeed, F.. Research Institute of Forests and Rangelands; IránFil: Maestro Martínez, M.. Instituto Pirenaico de Ecología; EspañaFil: Romo Diez, A.. Instituto Botanico de Barcelona; EspañaFil: Shaw, S.. Research Institute of Forests and Rangelands; Irán. The University. Department of Animal and Plant Sciences; Reino UnidoFil: Siavash, B.. Research Institute of Forests and Rangelands; IránFil: Villar Salvador, P.. Instituto Pirenaico de Ecología; Españ

Particle swarm optimization applied to EEG source localization of somatosensory evoked potentials

One of the most important steps in presurgical diagnosis of medically intractable epilepsy is to find the precise location of the epileptogenic foci. Electroencephalography (EEG) is a noninvasive tool commonly used at epilepsy surgery centers for presurgical diagnosis. In this paper, a modified particle swarm optimization (MPSO) method is used to solve the EEG source localization problem. The method is applied to noninvasive EEG recording of somatosensory evoked potentials (SEPs) for a healthy subject. A 1 mm hexahedra finite element volume conductor model of the subject's head was generated using T1-weighted magnetic resonance imaging data. Special consideration was made to accurately model the skull and cerebrospinal fluid. An exhaustive search pattern and the MPSO method were then applied to the peak of the averaged SEP data and both identified the same region of the somatosensory cortex as the location of the SEP source. A clinical expert independently identified the expected source location, further corroborating the source analysis methods. The MPSO converged to the global minima with significantly lower computational complexity compared to the exhaustive search method that required almost 3700 times more evaluations

On the fully automatic construction of a realistic head model for EEG source localization

Accurate multi-tissue segmentation of magnetic resonance (MR) images is an essential first step in the construction of a realistic finite element head conductivity model (FEHCM) for electroencephalography (EEG) source localization. All of the segmentation approaches proposed to date for this purpose require manual intervention or correction and are thus laborious, time-consuming, and subjective. In this paper we propose and evaluate a fully automatic method based on a hierarchical segmentation approach (HSA) incorporating Bayesian-based adaptive mean-shift segmentation (BAMS). An evaluation of HSA-BAMS, as well as two reference methods, in terms of both segmentation accuracy and the source localization accuracy of the resulting FEHCM is also presented. The evaluation was performed using (i) synthetic 2D multi-modal MRI head data and synthetic EEG (generated for a prescribed source), and (ii) real 3D T1-weighted MRI head data and real EEG data (with expert determined sou rce localization). Expert manual segmentation served as segmentation ground truth. The results show that HSA-BAMS outperforms the two reference methods and that it can be used as a surrogate for manual segmentation for the construction of a realistic FEHCM for EEG source localization