379 research outputs found
Storage capacity of correlated perceptrons
We consider an ensemble of single-layer perceptrons exposed to random
inputs and investigate the conditions under which the couplings of these
perceptrons can be chosen such that prescribed correlations between the outputs
occur. A general formalism is introduced using a multi-perceptron costfunction
that allows to determine the maximal number of random inputs as a function of
the desired values of the correlations. Replica-symmetric results for and
are compared with properties of two-layer networks of tree-structure and
fixed Boolean function between hidden units and output. The results show which
correlations in the hidden layer of multi-layer neural networks are crucial for
the value of the storage capacity.Comment: 16 pages, Latex2
A Parallel Incremental Learning Algorithm for Neural Networks with Fault Tolerance
URL : http://vecpar.fe.up.pt/2008/papers/46.pdfInternational audienceThis paper presents a parallel and fault tolerant version of an incremental learning algorithm for feed-forward neural networks used as function approximators. It has been shown in previous works that our incremental algorithm builds networks of reduced size while providing high quality approximations for real data sets. However, for very large sets, the use of our learning process on a single machine may be quite long and even sometimes impossible, due to memory limitations. The parallel algorithm presented in this paper is usable in any parallel system, and in particular, with large dynamical systems such as clusters and grids in which faults may occur. Finally, the quality and performances (without and with faults) of that algorithm are experimentally evaluated
Multilayer neural networks with extensively many hidden units
The information processing abilities of a multilayer neural network with a
number of hidden units scaling as the input dimension are studied using
statistical mechanics methods. The mapping from the input layer to the hidden
units is performed by general symmetric Boolean functions whereas the hidden
layer is connected to the output by either discrete or continuous couplings.
Introducing an overlap in the space of Boolean functions as order parameter the
storage capacity if found to scale with the logarithm of the number of
implementable Boolean functions. The generalization behaviour is smooth for
continuous couplings and shows a discontinuous transition to perfect
generalization for discrete ones.Comment: 4 pages, 2 figure
A weakly stable algorithm for general Toeplitz systems
We show that a fast algorithm for the QR factorization of a Toeplitz or
Hankel matrix A is weakly stable in the sense that R^T.R is close to A^T.A.
Thus, when the algorithm is used to solve the semi-normal equations R^T.Rx =
A^Tb, we obtain a weakly stable method for the solution of a nonsingular
Toeplitz or Hankel linear system Ax = b. The algorithm also applies to the
solution of the full-rank Toeplitz or Hankel least squares problem.Comment: 17 pages. An old Technical Report with postscript added. For further
details, see http://wwwmaths.anu.edu.au/~brent/pub/pub143.htm
On-line learning with adaptive back-propagation in two-layer networks
An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent
Evolutionary optimisation of neural network models for fish collective behaviours in mixed groups of robots and zebrafish
Animal and robot social interactions are interesting both for ethological
studies and robotics. On the one hand, the robots can be tools and models to
analyse animal collective behaviours, on the other hand, the robots and their
artificial intelligence are directly confronted and compared to the natural
animal collective intelligence. The first step is to design robots and their
behavioural controllers that are capable of socially interact with animals.
Designing such behavioural bio-mimetic controllers remains an important
challenge as they have to reproduce the animal behaviours and have to be
calibrated on experimental data. Most animal collective behavioural models are
designed by modellers based on experimental data. This process is long and
costly because it is difficult to identify the relevant behavioural features
that are then used as a priori knowledge in model building. Here, we want to
model the fish individual and collective behaviours in order to develop robot
controllers. We explore the use of optimised black-box models based on
artificial neural networks (ANN) to model fish behaviours. While the ANN may
not be biomimetic but rather bio-inspired, they can be used to link perception
to motor responses. These models are designed to be implementable as robot
controllers to form mixed-groups of fish and robots, using few a priori
knowledge of the fish behaviours. We present a methodology with multilayer
perceptron or echo state networks that are optimised through evolutionary
algorithms to model accurately the fish individual and collective behaviours in
a bounded rectangular arena. We assess the biomimetism of the generated models
and compare them to the fish experimental behaviours.Comment: 10 pages, 4 figure
Una pretensión desmedida, pero necesaria
Presentación del número 11 de la revista Renglones con el tema de Cultura. La cultura como concepto ha sido estudiada desde diferentes enfoques, ramas del saber o tradiciones cientÃficas, todas contribuyendo con su propia versión de lo que es cultura. Se revisan algunas definiciones realizadas por Edward Burnet Tylor, Antonio Gramsci, Gilberto Giménez y Pierre Bourdieu. Se enfatiza en la importancia de la producción y del estudio de la cultura desde la universidad y se da una introducción a los textos presentados en esta edición
The detection of fraud activities on the stock market through forward analysis methodology of financial discussion boards
Financial discussion boards (FDBs) have been widely used for a variety of financial knowledge exchange activities through the posting of comments on the FDBs. Popular public FDBs are prone to be used as a medium to spread false financial information due to having a larger group of audiences. Although online forums, in general, are usually integrated with anti-spam tools such as Akismet, moderation of posted contents heavily relies on human moderators. Unfortunately, popular FDBs attract many comments per day which realistically prevents human moderators from continuously monitoring and moderating possibly fraudulent contents. Such manual moderation can be extremely time-consuming. Moreover, due to the absence of useful tools, no relevant authorities are actively monitoring and handling potential financial crimes on FDBs. This paper presents a novel forward analysis methodology implemented in an Information Extraction (IE) prototype system named FDBs Miner (FDBM). This methodology aims to detect potentially illegal comments on FDBs while integrating share prices in the detection process as this helps to categorise the potentially illegal comments into different risk levels for investigation priority. The IE prototype system will first extract the public comments and per minute share prices from FDBs for the selected listed companies on London Stock Exchange (LSE). In the forward analysis process, the comments are flagged using a predefined Pump and Dump financial crime related keyword template. By only flagging the comments against the keyword template yields an average of 9.82% potentially illegal comments. It is unrealistic and unaffordable for human moderators to read these comments on a daily basis in long run. Hence, by integrating the share prices’ hikes and falls to categorise the flagged comments based on risk levels, it saves time and allows relevant authorities to prioritise and investigate into the higher risk flagged comments as it can potentially indicate real Pump and Dump crimes on FDBs
On the Bounds of Function Approximations
Within machine learning, the subfield of Neural Architecture Search (NAS) has
recently garnered research attention due to its ability to improve upon
human-designed models. However, the computational requirements for finding an
exact solution to this problem are often intractable, and the design of the
search space still requires manual intervention. In this paper we attempt to
establish a formalized framework from which we can better understand the
computational bounds of NAS in relation to its search space. For this, we first
reformulate the function approximation problem in terms of sequences of
functions, and we call it the Function Approximation (FA) problem; then we show
that it is computationally infeasible to devise a procedure that solves FA for
all functions to zero error, regardless of the search space. We show also that
such error will be minimal if a specific class of functions is present in the
search space. Subsequently, we show that machine learning as a mathematical
problem is a solution strategy for FA, albeit not an effective one, and further
describe a stronger version of this approach: the Approximate Architectural
Search Problem (a-ASP), which is the mathematical equivalent of NAS. We
leverage the framework from this paper and results from the literature to
describe the conditions under which a-ASP can potentially solve FA as well as
an exhaustive search, but in polynomial time.Comment: Accepted as a full paper at ICANN 2019. The final, authenticated
publication will be available at https://doi.org/10.1007/978-3-030-30487-4_3
- …