237 research outputs found
Neural visualization of network traffic data for intrusion detection
This study introduces and describes a novel intrusion detection system (IDS) called MOVCIDS (mobile visualization connectionist IDS). This system applies neural projection architectures to detect anomalous situations taking place in a computer network. By its advanced visualization facilities, the proposed IDS allows providing an overview of the network traffic as well as identifying anomalous situations tackled by computer networks, responding to the challenges presented by volume, dynamics and diversity of the traffic, including novel (0-day) attacks. MOVCIDS provides a novel point of view in the field of IDSs by enabling the most interesting projections (based on the fourth order statistics; the kurtosis index) of a massive traffic dataset to be extracted. These projections are then depicted through a functional and mobile visualization interface, providing visual information of the internal structure of the traffic data. The interface makes MOVCIDS accessible from any mobile device to give more accessibility to network administrators, enabling continuous visualization, monitoring and supervision of computer networks. Additionally, a novel testing technique has been developed to evaluate MOVCIDS and other IDSs employing numerical datasets. To show the performance and validate the proposed IDS, it has been tested in different real domains containing several attacks and anomalous situations. In addition, the importance of the temporal dimension on intrusion detection, and the ability of this IDS to process it, are emphasized in this workJunta de Castilla and Leon project BU006A08, Business intelligence for production within the framework of the Instituto Tecnologico de Cas-tilla y Leon (ITCL) and the Agencia de Desarrollo Empresarial (ADE), and the Spanish Ministry of Education and Innovation project CIT-020000-2008-2. The authors would also like to thank the vehicle interior manufacturer, Grupo Antolin Ingenieria S. A., within the framework of the project MAGNO2008-1028-CENIT Project funded by the Spanish Government
A stigmergy-based analysis of city hotspots to discover trends and anomalies in urban transportation usage
A key aspect of a sustainable urban transportation system is the
effectiveness of transportation policies. To be effective, a policy has to
consider a broad range of elements, such as pollution emission, traffic flow,
and human mobility. Due to the complexity and variability of these elements in
the urban area, to produce effective policies remains a very challenging task.
With the introduction of the smart city paradigm, a widely available amount of
data can be generated in the urban spaces. Such data can be a fundamental
source of knowledge to improve policies because they can reflect the
sustainability issues underlying the city. In this context, we propose an
approach to exploit urban positioning data based on stigmergy, a bio-inspired
mechanism providing scalar and temporal aggregation of samples. By employing
stigmergy, samples in proximity with each other are aggregated into a
functional structure called trail. The trail summarizes relevant dynamics in
data and allows matching them, providing a measure of their similarity.
Moreover, this mechanism can be specialized to unfold specific dynamics.
Specifically, we identify high-density urban areas (i.e hotspots), analyze
their activity over time, and unfold anomalies. Moreover, by matching activity
patterns, a continuous measure of the dissimilarity with respect to the typical
activity pattern is provided. This measure can be used by policy makers to
evaluate the effect of policies and change them dynamically. As a case study,
we analyze taxi trip data gathered in Manhattan from 2013 to 2015.Comment: Preprin
AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments
This report considers the application of Articial Intelligence (AI) techniques to
the problem of misuse detection and misuse localisation within telecommunications
environments. A broad survey of techniques is provided, that covers inter alia
rule based systems, model-based systems, case based reasoning, pattern matching,
clustering and feature extraction, articial neural networks, genetic algorithms, arti
cial immune systems, agent based systems, data mining and a variety of hybrid
approaches. The report then considers the central issue of event correlation, that
is at the heart of many misuse detection and localisation systems. The notion of
being able to infer misuse by the correlation of individual temporally distributed
events within a multiple data stream environment is explored, and a range of techniques,
covering model based approaches, `programmed' AI and machine learning
paradigms. It is found that, in general, correlation is best achieved via rule based approaches,
but that these suffer from a number of drawbacks, such as the difculty of
developing and maintaining an appropriate knowledge base, and the lack of ability
to generalise from known misuses to new unseen misuses. Two distinct approaches
are evident. One attempts to encode knowledge of known misuses, typically within
rules, and use this to screen events. This approach cannot generally detect misuses
for which it has not been programmed, i.e. it is prone to issuing false negatives.
The other attempts to `learn' the features of event patterns that constitute normal
behaviour, and, by observing patterns that do not match expected behaviour, detect
when a misuse has occurred. This approach is prone to issuing false positives,
i.e. inferring misuse from innocent patterns of behaviour that the system was not
trained to recognise. Contemporary approaches are seen to favour hybridisation,
often combining detection or localisation mechanisms for both abnormal and normal
behaviour, the former to capture known cases of misuse, the latter to capture
unknown cases. In some systems, these mechanisms even work together to update
each other to increase detection rates and lower false positive rates. It is concluded
that hybridisation offers the most promising future direction, but that a rule or state
based component is likely to remain, being the most natural approach to the correlation
of complex events. The challenge, then, is to mitigate the weaknesses of
canonical programmed systems such that learning, generalisation and adaptation
are more readily facilitated
Unsupervised Anomaly Detection in Stream Data with Online Evolving Spiking Neural Networks
Unsupervised anomaly discovery in stream data is a research topic with many practical applications. However, in many cases, it is not easy to collect enough training data with labeled anomalies for supervised learning of an anomaly detector in order to deploy it later for identification of real anomalies in streaming data. It is thus important to design anomalies detectors that can correctly detect anomalies without access to labeled training data. Our idea is to adapt the Online evolving Spiking Neural Network (OeSNN) classifier to the anomaly detection task. As a result, we offer an Online evolving Spiking Neural Network for Unsupervised Anomaly Detection algorithm (OeSNN-UAD), which, unlike OeSNN, works in an unsupervised way and does not separate output neurons into disjoint decision classes. OeSNN-UAD uses our proposed new two-step anomaly detection method. Also, we derive new theoretical properties of neuronal model and input layer encoding of OeSNN, which enable more effective and efficient detection of anomalies in our OeSNN-UAD approach. The proposed OeSNN-UAD detector was experimentally compared with state-of-the-art unsupervised and semi-supervised detectors of anomalies in stream data from the Numenta Anomaly Benchmark and Yahoo Anomaly Datasets repositories. Our approach outperforms the other solutions provided in the literature in the case of data streams from the Numenta Anomaly Benchmark repository. Also, in the case of real data files of the Yahoo Anomaly Benchmark repository, OeSNN-UAD outperforms other selected algorithms, whereas in the case of Yahoo Anomaly Benchmark synthetic data files, it provides competitive results to the results recently reported in the literature.P. Maciąg acknowledges financial Support of the Faculty of the Electronics and Information Technology of the Warsaw University of Technology, Poland (Grant No. II/2019/GD/1). J.L. Lobo and J. Del Ser would like to thank the Basque Government, Spain for their support through the ELKARTEK and EMAITEK funding programs. J. Del Ser also acknowledges funding support from the Consolidated Research Group MATHMODE (IT1294-19) given by the Department of Education of the Basque Governmen
Comparative analysis of classification techniques for network anomalies management
Bugün, teknolojideki hızlı gelişme milyarlarca cihazın birbiriyle iletişim
kurmasını sağlıyor. Bu gelişme, tüm bu cihazların ağa kolayca bağlanabilmesi için
yeni ağ teknolojilerini gerektirir. Son yıllarda, siber saldırılar hükümetler, işletmeler
ve bireyler için ciddi bir tehdit oluşturuyor. Bu siber saldırıları önlemek için tasarlanan
birçok saldırı tespit sistemi başarısız oldu. Saldırı Tespit Sistemleri (IDS) saldırıları ve
saldırganların kullandığı kurnazca yollarını yeterince tanıyamadığından yetersiz IDS
çözümü ve savunmasız ağlarla sonuçlandı. Veri madenciliği ve istatistiğin bir sonucu
olan makine öğrenmesi tabanlı sistemler kullanmak saldırıları önlemek için çok daha
akıllıca bir çözüm olacaktır. Bu yaklaşım, saldırı tanıma tekniklerine dayanan klasik
IDS çözümüne kıyasla daha verimli bir IDS çözümü getirecektir. Bu tezin amacı, ağ
sorun giderme işlemlerini geliştirmek ve bakım işlemlerinin verimliliğini artırmak
amacıyla makine öğrenmesini kullanarak Ağ Tabanlı Anomali Tespit Sistemi (NADS)
için bir yöntem önermektir. Bu çalışma, seçilen dört makine öğrenme sınıflandırma
algoritmasının performansını birbiriyle karşılaştırmaktadır. Seçilen algoritmalar
şunlardır: K-En Yakın Komşular (KNN), K-Means, Naïve Bayes ve Random Forest.
Bu karşılaştırma ağ anomalisini tespit etmek ve sınıflandırma çerçevesinin
performansını analiz etmek içindir. Bu karşılaştırma, çerçeve seçimi ile ilgili öneriler
sunmak için yapılmıştır. Yukarıda belirtilen algoritmalar, izinsiz giriş tespit
prototiplerini değerlendirmek için yaygın olarak kullanılan KDD CUP99 izinsiz giriş
tespit veri setinde uygulanır ve test edilir. Deneysel sonuçlar KNN algoritmasının
doğruluk ve hesaplama süresi açısından iyi çalıştığını göstermektedir. Ayrıca,
KNN'nin bilinen tüm saldırıların % 98.0379’luk potansiyel tehdidin başarılı bir şekilde
tespit ettiğini göstermiştir
Proceedings of the Second Joint Technology Workshop on Neural Networks and Fuzzy Logic, volume 2
Documented here are papers presented at the Neural Networks and Fuzzy Logic Workshop sponsored by NASA and the University of Texas, Houston. Topics addressed included adaptive systems, learning algorithms, network architectures, vision, robotics, neurobiological connections, speech recognition and synthesis, fuzzy set theory and application, control and dynamics processing, space applications, fuzzy logic and neural network computers, approximate reasoning, and multiobject decision making
Anomalous behaviour detection using heterogeneous data
Anomaly detection is one of the most important methods to process and find abnormal data, as this method can distinguish between normal and abnormal behaviour. Anomaly detection has been applied in many areas such as the medical sector, fraud detection in finance, fault detection in machines, intrusion detection in networks, surveillance systems for security, as well as forensic investigations. Abnormal behaviour can give information or answer questions when an investigator is performing an investigation. Anomaly detection is one way to simplify big data by focusing on data that have been grouped or clustered by the anomaly detection method. Forensic data usually consists of heterogeneous data which have several data forms or types such as qualitative or quantitative, structured or unstructured, and primary or secondary. For example, when a crime takes place, the evidence can be in the form of various types of data. The combination of all the data types can produce rich information insights. Nowadays, data has become ‘big’ because it is generated every second of every day and processing has become time-consuming and tedious. Therefore, in this study, a new method to detect abnormal behaviour is proposed using heterogeneous data and combining the data using data fusion technique. Vast challenge data and image data are applied to demonstrate the heterogeneous data. The first contribution in this study is applying the heterogeneous data to detect an anomaly. The recently introduced anomaly detection technique which is known as Empirical Data Analytics (EDA) is applied to detect the abnormal behaviour based on the data sets. Standardised eccentricity (a newly introduced within EDA measure offering a new simplified form of the well-known Chebyshev Inequality) can be applied to any data distribution. Then, the second contribution is applying image data. The image data is processed using pre-trained deep learning network, and classification is done using a support vector machine (SVM). After that, the last contribution is combining anomaly result from heterogeneous data and image recognition using new data fusion technique. There are five types of data with three different modalities and different dimensionalities. The data cannot be simply combined and integrated. Therefore, the new data fusion technique first analyses the abnormality in each data type separately and determines the degree of suspicious between 0 and 1 and sums up all the degrees of suspicion data afterwards. This method is not intended to be a fully automatic system that resolves investigations, which would likely be unacceptable in any case. The aim is rather to simplify the role of the humans so that they can focus on a small number of cases to be looked in more detail. The proposed approach does simplify the processing of such huge amounts of data. Later, this method can assist human experts in their investigations and making final decisions
- …