29,250 research outputs found
A unified wavelet-based modelling framework for non-linear system identification: the WANARX model structure
A new unified modelling framework based on the superposition of additive submodels, functional components, and
wavelet decompositions is proposed for non-linear system identification. A non-linear model, which is often represented
using a multivariate non-linear function, is initially decomposed into a number of functional components via the wellknown
analysis of variance (ANOVA) expression, which can be viewed as a special form of the NARX (non-linear
autoregressive with exogenous inputs) model for representing dynamic input–output systems. By expanding each functional
component using wavelet decompositions including the regular lattice frame decomposition, wavelet series and
multiresolution wavelet decompositions, the multivariate non-linear model can then be converted into a linear-in-theparameters
problem, which can be solved using least-squares type methods. An efficient model structure determination
approach based upon a forward orthogonal least squares (OLS) algorithm, which involves a stepwise orthogonalization
of the regressors and a forward selection of the relevant model terms based on the error reduction ratio (ERR), is
employed to solve the linear-in-the-parameters problem in the present study. The new modelling structure is referred to
as a wavelet-based ANOVA decomposition of the NARX model or simply WANARX model, and can be applied to
represent high-order and high dimensional non-linear systems
Connectionist Inference Models
The performance of symbolic inference tasks has long been a challenge to connectionists. In this paper, we present an extended survey of this area. Existing connectionist inference systems are reviewed, with particular reference to how they perform variable binding and rule-based reasoning, and whether they involve distributed or localist representations. The benefits and disadvantages of different representations and systems are outlined, and conclusions drawn regarding the capabilities of connectionist inference systems when compared with symbolic inference systems or when used for cognitive modeling
Representations of molecules and materials for interpolation of quantum-mechanical simulations via machine learning
Computational study of molecules and materials from first principles is a cornerstone of physics, chemistry and materials science, but limited by the cost of accurate and precise simulations. In settings involving many simulations, machine learning can reduce these costs, sometimes by orders of magnitude, by interpolating between reference simulations. This requires representations that describe any molecule or material and support interpolation. We review, discuss and benchmark state-of-the-art representations and relations between them, including smooth overlap of atomic positions, many-body tensor representation, and symmetry functions. For this, we use a unified mathematical framework based on many-body functions, group averaging and tensor products, and compare energy predictions for organic molecules, binary alloys and Al-Ga-In sesquioxides in numerical experiments controlled for data distribution, regression method and hyper-parameter optimization
Deep Reinforcement Learning for Swarm Systems
Recently, deep reinforcement learning (RL) methods have been applied
successfully to multi-agent scenarios. Typically, these methods rely on a
concatenation of agent states to represent the information content required for
decentralized decision making. However, concatenation scales poorly to swarm
systems with a large number of homogeneous agents as it does not exploit the
fundamental properties inherent to these systems: (i) the agents in the swarm
are interchangeable and (ii) the exact number of agents in the swarm is
irrelevant. Therefore, we propose a new state representation for deep
multi-agent RL based on mean embeddings of distributions. We treat the agents
as samples of a distribution and use the empirical mean embedding as input for
a decentralized policy. We define different feature spaces of the mean
embedding using histograms, radial basis functions and a neural network learned
end-to-end. We evaluate the representation on two well known problems from the
swarm literature (rendezvous and pursuit evasion), in a globally and locally
observable setup. For the local setup we furthermore introduce simple
communication protocols. Of all approaches, the mean embedding representation
using neural network features enables the richest information exchange between
neighboring agents facilitating the development of more complex collective
strategies.Comment: 31 pages, 12 figures, version 3 (published in JMLR Volume 20
Optical implementations of radial basis classifiers
We describe two optical systems based on the radial basis function approach to pattern classification. An optical-disk-based system for handwritten character recognition is demonstrated. The optical system computes the Euclidean distance between an unknown input and 650 stored patterns at a demonstrated rate of 26,000 pattern comparisons/s. The ultimate performance of this system is limited by optical-disk resolution to 10^11 binary operations/s. An adaptive system is also presented that facilitates on-line learning and provides additional robustness
Optical network for real-time face recognition
An optical network is described that is capable of recognizing at standard video rates the identity of faces for which it has been trained. The faces are presented under a wide variety of conditions to the system and the classification performance is measured. The system is trained by gradually adapting photorefractive holograms
End-to-End Differentiable Proving
We introduce neural networks for end-to-end differentiable proving of queries
to knowledge bases by operating on dense vector representations of symbols.
These neural networks are constructed recursively by taking inspiration from
the backward chaining algorithm as used in Prolog. Specifically, we replace
symbolic unification with a differentiable computation on vector
representations of symbols using a radial basis function kernel, thereby
combining symbolic reasoning with learning subsymbolic vector representations.
By using gradient descent, the resulting neural network can be trained to infer
facts from a given incomplete knowledge base. It learns to (i) place
representations of similar symbols in close proximity in a vector space, (ii)
make use of such similarities to prove queries, (iii) induce logical rules, and
(iv) use provided and induced logical rules for multi-hop reasoning. We
demonstrate that this architecture outperforms ComplEx, a state-of-the-art
neural link prediction model, on three out of four benchmark knowledge bases
while at the same time inducing interpretable function-free first-order logic
rules.Comment: NIPS 2017 camera-ready, NIPS 201
Forecasting the geomagnetic activity of the Dst Index using radial basis function networks
The Dst index is a key parameter which characterises the disturbance of the geomagnetic field in magnetic storms. Modelling of the Dst index is thus very important for the analysis of the geomagnetic field. A data-based modelling approach, aimed at obtaining efficient models based on limited input-output observational data, provides a powerful tool for analysing and forecasting geomagnetic activities including the prediction of the Dst index. Radial basis function (RBF) networks are an important and popular network model for nonlinear system identification and dynamical modelling. A novel generalised multiscale RBF (MSRBF) network is introduced for Dst index modelling. The proposed MSRBF network can easily be converted into a linear-in-the-parameters form and the training of the linear network model can easily be implemented using an orthogonal least squares (OLS) type algorithm. One advantage of the new MSRBF network, compared with traditional single scale RBF networks, is that the new network is more flexible for describing complex nonlinear dynamical systems
Dimensions of Neural-symbolic Integration - A Structured Survey
Research on integrated neural-symbolic systems has made significant progress
in the recent past. In particular the understanding of ways to deal with
symbolic knowledge within connectionist systems (also called artificial neural
networks) has reached a critical mass which enables the community to strive for
applicable implementations and use cases. Recent work has covered a great
variety of logics used in artificial intelligence and provides a multitude of
techniques for dealing with them within the context of artificial neural
networks. We present a comprehensive survey of the field of neural-symbolic
integration, including a new classification of system according to their
architectures and abilities.Comment: 28 page
- …