Location of Repository

Towards a multipurpose neural network approach to novelty detection

By Simon J. Haggett

Abstract

Novelty detection, the identification of data that is unusual or different in some way, is relevant in a wide number of real-world scenarios, ranging from identifying unusual weather conditions to detecting evidence of damage in mechanical systems. However, utilising novelty detection approaches in a particular scenario presents significant challenges to the non-expert user. They must first select an appropriate approach from the novelty detection literature for their scenario. Then, suitable values must be determined for any parameters of the chosen approach. These challenges are at best time consuming and at worst prohibitively difficult for the user. Worse still, if no suitable approach can be found from the literature, then the user is left with the impossible task of designing a novelty detector themselves. In order to make novelty detection more accessible, an approach is required which does not pose the above challenges. This thesis presents such an approach, which aims to automatically construct novelty detectors for specific applications. The approach combines a neural network model, recently proposed to explain a phenomenon observed in the neural pathways of the retina, with an evolutionary algorithm that is capable of simultaneously evolving the structure and weights of a neural network in order to optimise its performance in a particular task. The proposed approach was evaluated over a number of very different novelty detection tasks. It was found that, in each task, the approach successfully evolved novelty detectors which outperformed a number of existing techniques from the literature. A number of drawbacks with the approach were also identified, and suggestions were given on ways in which these may potentially be overcome

Topics: QA76
Publisher: University of Kent
OAI identifier: oai:kar.kent.ac.uk:24133

Suggested articles

Preview

Citations

  1. (2004). A Basic Course in Statistics. doi
  2. (1995). A bootstrap-like rejection mechanism for multilayer perceptron networks. doi
  3. (2006). A cosine similarity-based negative selection algorithm for time series novelty detection. doi
  4. (2004). A framework for discovering anomalous regimes in multivariate time-series data with local models. http://www.isle.org/ sbay/papers/darts.pdf,
  5. (1985). A learning algorithm for Boltzmann machines. doi
  6. (2000). A linear programming approach to novelty detection.
  7. (1987). A massively parallel architecture for a self-organizing neural pattern recognition machine. doi
  8. (1995). A method for improving classi reliability of multilayer perceptrons. doi
  9. (1995). A method for improving classification reliability of multilayer perceptrons. doi
  10. (1995). A novelty detection approach to classi
  11. (1995). A novelty detection approach to classification.
  12. (2001). A robot implementation of a biologically inspired method for novelty detection.
  13. (2004). A survey of outlier detection methodologies. doi
  14. (2002). A tale of two - on-line novelty detection. doi
  15. (2002). A tale of two filters - on-line novelty detection. doi
  16. (2004). A tutorial on support vector regression. doi
  17. (2006). A unifying framework for detecting outliers and change points from time series. doi
  18. (1991). A User's Guide to Principal Components. doi
  19. (1991). A User’s Guide to Principal Components. doi
  20. (2003). Accurate on-line support vector regression. doi
  21. (1975). Adaptation in Natural and Arti Systems.
  22. (1975). Adaptation in Natural and Artificial Systems. doi
  23. (2007). Adaptive anomaly detection with evolving connectionist systems. doi
  24. (1976). Adaptive pattern classi and universal recoding: doi
  25. (1976). Adaptive pattern classification and universal recoding: doi
  26. (2004). An approach to novelty detection applied to the classi of image regions. doi
  27. (2004). An approach to novelty detection applied to the classification of image regions. doi
  28. (1999). An Introduction to Genetic Algorithms. doi
  29. (2005). An online kernel change detection algorithm. doi
  30. (2002). Anomaly detection in embedded systems. doi
  31. (2003). Anomaly detection using real-valued negative selection.
  32. (2003). Anomaly detection using real-valued negative selection. Genetic Programming and Evolvable Machines,
  33. (2009). Architecture Department. Architecture department weather data. http://www.caed.calpoly.edu/sites/ehhf/weatherstation.html. (URL correct as of
  34. (1991). ART 2-A: An adaptive resonance algorithm for rapid category learning and recognition. doi
  35. (1987). ART 2: Self-organization of stable category recognition codes for analog input patterns. doi
  36. (1990). ART 3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures. doi
  37. (1966). Arti Intelligence through Simulated Evolution.
  38. (1966). Artificial Intelligence through Simulated Evolution. doi
  39. (1997). ARTMAP-FD: Familiarity discrimination applied to radar target recognition. doi
  40. (2003). Automatic change detection of driving environments in a vision-based driver assistance system. doi
  41. (1995). Boosting the performance of RBF networks with dynamic decay adjustment.
  42. (2003). Clustering of time series subsequences is meaningless: implications for past and future research. doi
  43. (2005). Clustering of time-series subsequences is meaningless: implications for previous and future research. doi
  44. (2002). Combining negative selection and classi techniques for anomaly detection. doi
  45. (2002). Combining negative selection and classification techniques for anomaly detection. doi
  46. (2000). Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain. doi
  47. (2007). Condensed Nearest Neighbor Data Domain Description. doi
  48. (2009). Control System Toolbox: Hard-disk read/write head controller. http://www.mathworks.com/access/helpdesk/help/toolbox/control/casestudies/f0-1000868.html. (URL correct as of
  49. (2000). Data description in subspaces. doi
  50. (1999). Data domain description using support vectors. doi
  51. (2002). Data Mining and Knowledge Discovery with Evolutionary Algorithms. doi
  52. (2001). Data Mining: Concepts and Techniques. doi
  53. (2000). Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. doi
  54. (1953). Discharge patterns and functional organisation of mammalian retina.
  55. (2005). Dynamic predictive coding by the retina. doi
  56. (2002). Ecient evolution of neural network topologies. doi
  57. (2004). Ecient Evolution of Neural Networks through Complexi
  58. (2002). Efficient evolution of neural network topologies. doi
  59. (2004). Efficient Evolution of Neural Networks through Complexification.
  60. (2004). Electric power system anomaly detection using neural networks. doi
  61. (1987). Elementary Linear Algebra. doi
  62. (1999). Elliptical novelty grouping for on-line short-turn detection of excited running rotors. doi
  63. (2001). Estimating the support of a high-dimensional distribution. doi
  64. (2006). Estimation of distribution algorithms. doi
  65. (2006). Evolutionary neural networks for anomaly detection based on the behavior of a program. doi
  66. (2003). Evolutionary optimization of radial basis function networks for intrusion detection. doi
  67. (2007). Evolutionary reinforcement learning of arti neural networks. doi
  68. (2007). Evolutionary reinforcement learning of artificial neural networks. doi
  69. (2008). Evolving a dynamic predictive coding mechanism for novelty detection. Knowledge-Based Systems, doi
  70. (1999). Evolving arti neural networks. doi
  71. (1999). Evolving artificial neural networks. doi
  72. (1952). Experiments with linear prediction in television. doi
  73. (1982). Exposition of statistical graphics technology.
  74. (1999). Familiarity discrimination of radar pulses.
  75. (1996). From recombination of genes to the estimation of distributions I. Binary parameters.
  76. (1994). Fundamentals of Neural Networks. doi
  77. (1991). Fuzzy ART: An adaptive resonance algorithm for rapid, stable classi of analog patterns. doi
  78. (1991). Fuzzy ART: An adaptive resonance algorithm for rapid, stable classification of analog patterns. doi
  79. Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. doi
  80. (1992). Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. doi
  81. (2004). Gear fault detection using arti neural networks and support vector machines with genetic algorithms. doi
  82. (2004). Gear fault detection using artificial neural networks and support vector machines with genetic algorithms. doi
  83. (1995). Genetic algorithms and neural networks. doi
  84. (1987). Genetic algorithms with sharing for multimodal function optimization.
  85. (1992). Genetic Programming: On the Programming of Computers by Means of Natural Selection. doi
  86. (1993). Genetic set recombination and its application to neural network topology optimisation. doi
  87. (1965). Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. doi
  88. (1999). High capacity neural networks for familiarity discrimination. doi
  89. (2005). HOT SAX: Eciently the most unusual time series subsequence. doi
  90. (2005). HOT SAX: Efficiently finding the most unusual time series subsequence. doi
  91. (2002). Implicit learning in autoencoder novelty assessment. doi
  92. (2004). Improving novelty detection in short time series through RBF-DDA parameter adjustment. doi
  93. (2002). Improving the performance of radial basis function classi in condition monitoring and fault diagnosis applications where 'unknown' faults may occur. doi
  94. (2002). Improving the performance of radial basis function classifiers in condition monitoring and fault diagnosis applications where ’unknown’ faults may occur. doi
  95. (2001). Incremental and decremental support vector machine learning. doi
  96. (2004). Induction machine fault detection using SOM-based RBF neural networks. doi
  97. (2001). Intelligent Systems for Engineers and Scientists. doi
  98. (2006). Introduction to Data Mining. doi
  99. (2003). Introduction to Evolutionary Computing. doi
  100. (2005). Is negative selection appropriate for anomaly detection? doi
  101. (2000). Kernel method for percentile feature extraction.
  102. (1986). Learning internal representations by error propagation. doi
  103. (2001). LIBSVM: A library for support vector machines, doi
  104. (2001). Linear Algebra with Applications.
  105. (2007). Machine learning approaches to network anomaly detection.
  106. (2003). Model selection in one-class -SVMs using RBF kernels.
  107. (2007). Modeling changing dependency structure in multivariate time series. doi
  108. (1995). Multiple hypothesis testing.
  109. (1990). Multisurface method of pattern separation for medical diagnosis applied to breast cytology. doi
  110. (2007). Multivariate online anomaly detection using kernel recursive least squares. doi
  111. (2006). Negative Selection Algorithms: from the Thymus to V-detector.
  112. Neural networks and physical systems with emergent collective computational abilities. doi
  113. (1995). Neural Networks for Pattern Recognition. doi
  114. Neural Networks: A Comprehensive Foundation. doi
  115. (1991). Nonlinear principal component analysis using autoassociative neural networks. doi
  116. (2003). Novelty detection for short time series with neural networks. doi
  117. (2003). Novelty detection in a Kohonen-like network with a long-term depression learning rule. doi
  118. Novelty detection under changing environmental conditions. doi
  119. (2003). Novelty detection: a review - part 1: statistical approaches. doi
  120. (2003). Novelty detection: a review - part 2: neural network based approaches. doi
  121. (2001). On-line novelty detection through self-organisation, with application to inspection robotics.
  122. (2003). Online novelty detection on temporal sequences. doi
  123. (1977). Oscillation and chaos in physiological control systems. doi
  124. (2001). Pattern Classi doi
  125. (2001). Pattern Classification. doi
  126. (2000). PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. doi
  127. (1955). Predictive coding - parts I and II. doi
  128. (1982). Predictive Coding: A fresh view of inhibition in the retina. doi
  129. (1950). Predictive Coding. doi
  130. (2003). ROC graphs: Notes and practical considerations for researchers.
  131. (1994). Self-nonself discrimination in a computer. doi
  132. (2003). Self-organized maps of sensory events. doi
  133. Self-Organizing Maps. doi
  134. (2001). Self-Organizing Maps. Springer, 3rd edition, doi
  135. (2002). Statistical fraud detection: A review. doi
  136. Statistics of natural time-varying images. doi
  137. (1999). Support vector domain description. doi
  138. (2007). The Bonferroni and Sidak corrections for multiple comparisons.
  139. (2004). The kernel recursive least-squares algorithm. doi
  140. (2000). The Nature of Statistical Learning Theory. doi
  141. (2006). The novelty detection approach for dierent degrees of class imbalance. doi
  142. (2006). The novelty detection approach for different degrees of class imbalance. doi
  143. (1958). The perceptron: A probabilistic model for information storage in the brain. doi
  144. (1936). The use of multiple measurements in taxonomic problems. doi
  145. (2003). Time-series novelty detection using one-class support vector machines. doi
  146. (2002). Topic-conditioned novelty detection. doi
  147. (2007). Visual novelty detection with automatic scale selection. Robotics and Autonomous Systems, doi
  148. (2007). WAIRS: Improving classi accuracy by weighting attributes in the AIRS classi doi
  149. (2007). WAIRS: Improving classification accuracy by weighting attributes in the AIRS classifier. doi

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.