149 research outputs found
Recommended from our members
ReLEx: Regularisation for Linear Extrapolation in Neural Networks with Rectified Linear Units
Despite the great success of neural networks in recent years, they are not providing useful extrapolation. In regression tasks, the popular Rectified Linear Units do enable unbounded linear extrapolation by neural networks, but their extrapolation behaviour varies widely and is largely independent of the training data. Our goal is instead to continue the local linear trend at the margin of the training data. Here we introduce ReLEx, a regularising method composed of a set of loss terms design to achieve this goal and reduce the variance of the extrapolation. We present a ReLEx implementation for single input, single output, and single hidden layer feed-forward networks. Our results demonstrate that ReLEx has little cost in terms of standard learning, i.e. interpolation, but enables controlled univariate linear extrapolation with ReLU neural networks
{\sc CosmoNet}: fast cosmological parameter estimation in non-flat models using neural networks
We present a further development of a method for accelerating the calculation
of CMB power spectra, matter power spectra and likelihood functions for use in
cosmological Bayesian inference. The algorithm, called {\sc CosmoNet}, is based
on training a multilayer perceptron neural network. We compute CMB power
spectra (up to ) and matter transfer functions over a hypercube in
parameter space encompassing the confidence region of a selection of
CMB (WMAP + high resolution experiments) and large scale structure surveys (2dF
and SDSS). We work in the framework of a generic 7 parameter non-flat
cosmology. Additionally we use {\sc CosmoNet} to compute the WMAP 3-year, 2dF
and SDSS likelihoods over the same region. We find that the average error in
the power spectra is typically well below cosmic variance for spectra, and
experimental likelihoods calculated to within a fraction of a log unit. We
demonstrate that marginalised posteriors generated with {\sc CosmoNet} spectra
agree to within a few percent of those generated by {\sc CAMB} parallelised
over 4 CPUs, but are obtained 2-3 times faster on just a \emph{single}
processor. Furthermore posteriors generated directly via {\sc CosmoNet}
likelihoods can be obtained in less than 30 minutes on a single processor,
corresponding to a speed up of a factor of . We also demonstrate the
capabilities of {\sc CosmoNet} by extending the CMB power spectra and matter
transfer function training to a more generic 10 parameter cosmological model,
including tensor modes, a varying equation of state of dark energy and massive
neutrinos. {\sc CosmoNet} and interfaces to both {\sc CosmoMC} and {\sc
Bayesys} are publically available at {\tt
www.mrao.cam.ac.uk/software/cosmonet}.Comment: 8 pages, submitted to MNRA
Infering Air Quality from Traffic Data using Transferable Neural Network Models
This work presents a neural network based model for inferring air quality from traffic measurements.
It is important to obtain information on air quality in urban environments in order to meet legislative and policy requirements. Measurement equipment tends to be expensive to purchase and maintain. Therefore, a model based approach capable of accurate determination of pollution levels is highly beneficial.
The objective of this study was to develop a neural network model to accurately infer pollution levels from existing data sources in Leicester, UK.
Neural Networks are models made of several highly interconnected processing elements. These elements process information by their dynamic state response to inputs. Problems which were not solvable by traditional algorithmic approaches frequently can be solved using neural networks.
This paper shows that using a simple neural network with traffic and meteorological data as inputs, the air quality can be estimated with a good level of generalisation and in near real-time.
By applying these models to links rather than nodes, this methodology can directly be used to inform traffic engineers and direct traffic management decisions towards enhancing local air quality and traffic management simultaneously.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech
Genetic Classification of Populations using Supervised Learning
There are many instances in genetics in which we wish to determine whether
two candidate populations are distinguishable on the basis of their genetic
structure. Examples include populations which are geographically separated,
case--control studies and quality control (when participants in a study have
been genotyped at different laboratories). This latter application is of
particular importance in the era of large scale genome wide association
studies, when collections of individuals genotyped at different locations are
being merged to provide increased power. The traditional method for detecting
structure within a population is some form of exploratory technique such as
principal components analysis. Such methods, which do not utilise our prior
knowledge of the membership of the candidate populations. are termed
\emph{unsupervised}. Supervised methods, on the other hand are able to utilise
this prior knowledge when it is available.
In this paper we demonstrate that in such cases modern supervised approaches
are a more appropriate tool for detecting genetic differences between
populations. We apply two such methods, (neural networks and support vector
machines) to the classification of three populations (two from Scotland and one
from Bulgaria). The sensitivity exhibited by both these methods is considerably
higher than that attained by principal components analysis and in fact
comfortably exceeds a recently conjectured theoretical limit on the sensitivity
of unsupervised methods. In particular, our methods can distinguish between the
two Scottish populations, where principal components analysis cannot. We
suggest, on the basis of our results that a supervised learning approach should
be the method of choice when classifying individuals into pre-defined
populations, particularly in quality control for large scale genome wide
association studies.Comment: Accepted PLOS On
Development of appropriateness explicit criteria for cataract extraction by phacoemulsification
BACKGROUND: Consensus development techniques were used in the late 1980s to create explicit criteria for the appropriateness of cataract extraction. We developed a new appropriateness of indications tool for cataract following the RAND method. We tested the validity of our panel results. METHODS: Criteria were developed using a modified Delphi panel judgment process. A panel of 12 ophthalmologists was assembled. Ratings were analyzed regarding the level of agreement among panelists. We studied the influence of all variables on the final panel score using linear and logistic regression models. The explicit criteria developed were summarized by classification and regression tree analysis. RESULTS: Of the 765 indications evaluated by the main panel in the second round, 32.9% were found appropriate, 30.1% uncertain, and 37% inappropriate. Agreement was found in 53% of the indications and disagreement in 0.9%. Seven variables were considered to create the indications and divided into three groups: simple cataract, with diabetic retinopathy, or with other ocular pathologies. The preoperative visual acuity in the cataractous eye and visual function were the variables that best explained the panel scoring. The panel results were synthesized and presented in three decision trees. Misclassification error in the decision trees, as compared with the panel original criteria, was 5.3%. CONCLUSION: The parameters tested showed acceptable validity for an evaluation tool. These results support the use of this indication algorithm as a screening tool for assessing the appropriateness of cataract extraction in field studies and for the development of practice guidelines
Neural Network Parameterizations of Electromagnetic Nucleon Form Factors
The electromagnetic nucleon form-factors data are studied with artificial
feed forward neural networks. As a result the unbiased model-independent
form-factor parametrizations are evaluated together with uncertainties. The
Bayesian approach for the neural networks is adapted for chi2 error-like
function and applied to the data analysis. The sequence of the feed forward
neural networks with one hidden layer of units is considered. The given neural
network represents a particular form-factor parametrization. The so-called
evidence (the measure of how much the data favor given statistical model) is
computed with the Bayesian framework and it is used to determine the best form
factor parametrization.Comment: The revised version is divided into 4 sections. The discussion of the
prior assumptions is added. The manuscript contains 4 new figures and 2 new
tables (32 pages, 15 figures, 2 tables
Constraints on fNL from Wilkinson Microwave Anisotropy Probe 7-year data using a neural network classifier
We present a multi-class neural network (NN) classifier as a method to
measure nonGaussianity, characterised by the local non-linear coupling
parameter fNL, in maps of the cosmic microwave background (CMB) radiation. The
classifier is trained on simulated non-Gaussian CMB maps with a range of known
fNL values by providing it with wavelet coefficients of the maps; we consider
both the HealPix (HW) wavelet and the spherical Mexican hat wavelet (SMHW).
When applied to simulated test maps, the NN classfier produces results in very
good agreement with those obtained using standard chi2 minimization. The
standard deviations of the fNL estimates for WMAPlike simulations were {\sigma}
= 22 and {\sigma} = 33 for the SMHW and the HW, respectively, which are
extremely close to those obtained using classical statistical methods in Curto
et al. and Casaponsa et al. Moreover, the NN classifier does not require the
inversion of a large covariance matrix, thus avoiding any need to regularise
the matrix when it is not directly invertible, and is considerably faster.Comment: Accepted for publication in MNRAS, 9 pages, 5 figures, 1 tabl
On the Bounds of Function Approximations
Within machine learning, the subfield of Neural Architecture Search (NAS) has
recently garnered research attention due to its ability to improve upon
human-designed models. However, the computational requirements for finding an
exact solution to this problem are often intractable, and the design of the
search space still requires manual intervention. In this paper we attempt to
establish a formalized framework from which we can better understand the
computational bounds of NAS in relation to its search space. For this, we first
reformulate the function approximation problem in terms of sequences of
functions, and we call it the Function Approximation (FA) problem; then we show
that it is computationally infeasible to devise a procedure that solves FA for
all functions to zero error, regardless of the search space. We show also that
such error will be minimal if a specific class of functions is present in the
search space. Subsequently, we show that machine learning as a mathematical
problem is a solution strategy for FA, albeit not an effective one, and further
describe a stronger version of this approach: the Approximate Architectural
Search Problem (a-ASP), which is the mathematical equivalent of NAS. We
leverage the framework from this paper and results from the literature to
describe the conditions under which a-ASP can potentially solve FA as well as
an exhaustive search, but in polynomial time.Comment: Accepted as a full paper at ICANN 2019. The final, authenticated
publication will be available at https://doi.org/10.1007/978-3-030-30487-4_3
- …