1,906 research outputs found
Metaheuristic design of feedforward neural networks: a review of two decades of research
Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era
Data mining as a tool for environmental scientists
Over recent years a huge library of data mining algorithms has been developed to tackle a variety of problems in fields such as medical imaging and network traffic analysis. Many of these techniques are far more flexible than more classical modelling approaches and could be usefully applied to data-rich environmental problems. Certain techniques such as Artificial Neural Networks, Clustering, Case-Based Reasoning and more recently Bayesian Decision Networks have found application in environmental modelling while other methods, for example classification and association rule extraction, have not yet been taken up on any wide scale. We propose that these and other data mining techniques could be usefully applied to difficult problems in the field. This paper introduces several data mining concepts and briefly discusses their application to environmental modelling, where data may be sparse, incomplete, or heterogenous
Generalization Error in Deep Learning
Deep learning models have lately shown great performance in various fields
such as computer vision, speech recognition, speech translation, and natural
language processing. However, alongside their state-of-the-art performance, it
is still generally unclear what is the source of their generalization ability.
Thus, an important question is what makes deep neural networks able to
generalize well from the training set to new data. In this article, we provide
an overview of the existing theory and bounds for the characterization of the
generalization error of deep neural networks, combining both classical and more
recent theoretical and empirical results
Applying Deep Learning to Fast Radio Burst Classification
Upcoming Fast Radio Burst (FRB) surveys will search 10\, beams on
sky with very high duty cycle, generating large numbers of single-pulse
candidates. The abundance of false positives presents an intractable problem if
candidates are to be inspected by eye, making it a good application for
artificial intelligence (AI). We apply deep learning to single pulse
classification and develop a hierarchical framework for ranking events by their
probability of being true astrophysical transients. We construct a tree-like
deep neural network (DNN) that takes multiple or individual data products as
input (e.g. dynamic spectra and multi-beam detection information) and trains on
them simultaneously. We have built training and test sets using false-positive
triggers from real telescopes, along with simulated FRBs, and single pulses
from pulsars. Training of the DNN was independently done for two radio
telescopes: the CHIME Pathfinder, and Apertif on Westerbork. High accuracy and
recall can be achieved with a labelled training set of a few thousand events.
Even with high triggering rates, classification can be done very quickly on
Graphical Processing Units (GPUs). That speed is essential for selective
voltage dumps or issuing real-time VOEvents. Next, we investigate whether
dedispersion back-ends could be completely replaced by a real-time DNN
classifier. It is shown that a single forward propagation through a moderate
convolutional network could be faster than brute-force dedispersion; but the
low signal-to-noise per pixel makes such a classifier sub-optimal for this
problem. Real-time automated classification may prove useful for bright,
unexpected signals, both now and in the era of radio astronomy when data
volumes and the searchable parameter spaces further outgrow our ability to
manually inspect the data, such as for SKA and ngVLA
Manipulating Attributes of Natural Scenes via Hallucination
In this study, we explore building a two-stage framework for enabling users
to directly manipulate high-level attributes of a natural scene. The key to our
approach is a deep generative network which can hallucinate images of a scene
as if they were taken at a different season (e.g. during winter), weather
condition (e.g. in a cloudy day) or time of the day (e.g. at sunset). Once the
scene is hallucinated with the given attributes, the corresponding look is then
transferred to the input image while preserving the semantic details intact,
giving a photo-realistic manipulation result. As the proposed framework
hallucinates what the scene will look like, it does not require any reference
style image as commonly utilized in most of the appearance or style transfer
approaches. Moreover, it allows to simultaneously manipulate a given scene
according to a diverse set of transient attributes within a single model,
eliminating the need of training multiple networks per each translation task.
Our comprehensive set of qualitative and quantitative results demonstrate the
effectiveness of our approach against the competing methods.Comment: Accepted for publication in ACM Transactions on Graphic
Specification-Driven Predictive Business Process Monitoring
Predictive analysis in business process monitoring aims at forecasting the
future information of a running business process. The prediction is typically
made based on the model extracted from historical process execution logs (event
logs). In practice, different business domains might require different kinds of
predictions. Hence, it is important to have a means for properly specifying the
desired prediction tasks, and a mechanism to deal with these various prediction
tasks. Although there have been many studies in this area, they mostly focus on
a specific prediction task. This work introduces a language for specifying the
desired prediction tasks, and this language allows us to express various kinds
of prediction tasks. This work also presents a mechanism for automatically
creating the corresponding prediction model based on the given specification.
Differently from previous studies, instead of focusing on a particular
prediction task, we present an approach to deal with various prediction tasks
based on the given specification of the desired prediction tasks. We also
provide an implementation of the approach which is used to conduct experiments
using real-life event logs.Comment: This article significantly extends the previous work in
https://doi.org/10.1007/978-3-319-91704-7_7 which has a technical report in
arXiv:1804.00617. This article and the previous work have a coauthor in
commo
- …