6,305 research outputs found
Theoretical Properties of Projection Based Multilayer Perceptrons with Functional Inputs
Many real world data are sampled functions. As shown by Functional Data
Analysis (FDA) methods, spectra, time series, images, gesture recognition data,
etc. can be processed more efficiently if their functional nature is taken into
account during the data analysis process. This is done by extending standard
data analysis methods so that they can apply to functional inputs. A general
way to achieve this goal is to compute projections of the functional data onto
a finite dimensional sub-space of the functional space. The coordinates of the
data on a basis of this sub-space provide standard vector representations of
the functions. The obtained vectors can be processed by any standard method. In
our previous work, this general approach has been used to define projection
based Multilayer Perceptrons (MLPs) with functional inputs. We study in this
paper important theoretical properties of the proposed model. We show in
particular that MLPs with functional inputs are universal approximators: they
can approximate to arbitrary accuracy any continuous mapping from a compact
sub-space of a functional space to R. Moreover, we provide a consistency result
that shows that any mapping from a functional space to R can be learned thanks
to examples by a projection based MLP: the generalization mean square error of
the MLP decreases to the smallest possible mean square error on the data when
the number of examples goes to infinity
Representation of Functional Data in Neural Networks
Functional Data Analysis (FDA) is an extension of traditional data analysis
to functional data, for example spectra, temporal series, spatio-temporal
images, gesture recognition data, etc. Functional data are rarely known in
practice; usually a regular or irregular sampling is known. For this reason,
some processing is needed in order to benefit from the smooth character of
functional data in the analysis methods. This paper shows how to extend the
Radial-Basis Function Networks (RBFN) and Multi-Layer Perceptron (MLP) models
to functional data inputs, in particular when the latter are known through
lists of input-output pairs. Various possibilities for functional processing
are discussed, including the projection on smooth bases, Functional Principal
Component Analysis, functional centering and reduction, and the use of
differential operators. It is shown how to incorporate these functional
processing into the RBFN and MLP models. The functional approach is illustrated
on a benchmark of spectrometric data analysis.Comment: Also available online from:
http://www.sciencedirect.com/science/journal/0925231
On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: an application of flexible sampling methods using neural networks
Likelihoods and posteriors of instrumental variable regression models with strongendogeneity and/or weak instruments may exhibit rather non-elliptical contours inthe parameter space. This may seriously affect inference based on Bayesian crediblesets. When approximating such contours using Monte Carlo integration methods likeimportance sampling or Markov chain Monte Carlo procedures the speed of the algorithmand the quality of the results greatly depend on the choice of the importance orcandidate density. Such a density has to be `close' to the target density in order toyield accurate results with numerically efficient sampling. For this purpose we introduce neural networks which seem to be natural importance or candidate densities, as they have a universal approximation property and are easy to sample from.A key step in the proposed class of methods is the construction of a neural network that approximates the target density accurately. The methods are tested on a set ofillustrative models. The results indicate the feasibility of the neural networkapproach.Markov chain Monte Carlo;Bayesian inference;credible sets;importance sampling;instrumental variables;neural networks;reduced rank
Considerations of Accuracy and Uncertainty with Kriging Surrogate Models in Single-Objective Electromagnetic Design Optimization
The high computational cost of evaluating objective functions in electromagnetic optimal design problems necessitates the use of cost-effective techniques. This paper discusses the use of one popular technique, surrogate modelling, with emphasis placed on the importance of considering both the accuracy of, and uncertainty in, the surrogate model. After briefly reviewing how such considerations have been made in the single-objective optimization of electromagnetic devices, their use with kriging surrogate models is investigated. Traditionally, space-filling experimental designs are used to construct the initial kriging model, with the aim to maximize the accuracy of the initial surrogate model, from which the optimization search will start. Utility functions, which balance the predictions made by this model with its uncertainty, are often used to select the next point to be evaluated. In this paper, the performances of several different utility functions are examined using experimental designs which yield initial kriging models of varying degrees of accuracy. It is found that no advantage is necessarily achieved through searching for optima using utility functions on initial kriging models of higher accuracy, and that a reduction in the total number of objective function evaluations may be achieved by starting the iterative optimization search earlier with utility functions on kriging models of lower accuracy. The implications for electromagnetic optimal design are discussed
M[pi]log, Macromodeling via parametric identification of logic gates
This paper addresses the development of computational models of digital integrated circuit input and output buffers via the identification of nonlinear parametric models. The obtained models run in standard circuit simulation environments, offer improved accuracy and good numerical efficiency, and do not disclose information on the structure of the modeled devices. The paper reviews the basics of the parametric identification approach and illustrates its most recent extensions to handle temperature and supply voltage variations as well as power supply ports and tristate devices
A practical Bayesian framework for backpropagation networks
A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible (1) objective comparisons between solutions using alternative network architectures, (2) objective stopping rules for network pruning or growing procedures, (3) objective choice of magnitude and type of weight decay terms or additive regularizers (for penalizing large weights, etc.), (4) a measure of the effective number of well-determined parameters in a model, (5) quantified estimates of the error bars on network parameters and on network output, and (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian "evidence" automatically embodies "Occam's razor," penalizing overflexible and overcomplex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalization ability and the Bayesian evidence is obtained
SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
Deep learning has the potential to revolutionize quantum chemistry as it is
ideally suited to learn representations for structured data and speed up the
exploration of chemical space. While convolutional neural networks have proven
to be the first choice for images, audio and video data, the atoms in molecules
are not restricted to a grid. Instead, their precise locations contain
essential physical information, that would get lost if discretized. Thus, we
propose to use continuous-filter convolutional layers to be able to model local
correlations without requiring the data to lie on a grid. We apply those layers
in SchNet: a novel deep learning architecture modeling quantum interactions in
molecules. We obtain a joint model for the total energy and interatomic forces
that follows fundamental quantum-chemical principles. This includes
rotationally invariant energy predictions and a smooth, differentiable
potential energy surface. Our architecture achieves state-of-the-art
performance for benchmarks of equilibrium molecules and molecular dynamics
trajectories. Finally, we introduce a more challenging benchmark with chemical
and structural variations that suggests the path for further work
- âŠ