118,309 research outputs found
An experimental study on network intrusion detection systems
A signature database is the key component of an elaborate intrusion detection system. The efficiency of signature generation for an intrusion detection system is a crucial requirement because of the rapid appearance of new attacks on the World Wide Web. However, in the commercial applications, signature generation is still a manual process, which requires professional skills and heavy human effort. Knowledge Discovery and Data Mining methods may be a solution to this problem. Data Mining and Machine Learning algorithms can be applied to the network traffic databases, in order to automatically generate signatures.
The purpose of this thesis and the work related to it is to construct a feasible architecture for building a database of network traffic data. This database can then be used to generate signatures automatically. This goal is achieved using network traffic data captured on the data communication network at the New Jersey Institute of Technology (NJIT)
Understanding High Resolution Aerial Imagery Using Computer Vision Techniques
Computer vision can make important contributions to the analysis of remote sensing satellite or aerial imagery. However, the resolution of early satellite imagery was not sufficient to provide useful spatial features. The situation is changing with the advent of very-high-spatial-resolution (VHR) imaging sensors. This change makes it possible to use computer vision techniques to perform analysis of man-made structures. Meanwhile, the development of multi-view imaging techniques allows the generation of accurate point clouds as ancillary knowledge.
This dissertation aims at developing computer vision and machine learning algorithms for high resolution aerial imagery analysis in the context of application problems including debris detection, building detection and roof condition assessment. High resolution aerial imagery and point clouds were provided by Pictometry International for this study.
Debris detection after natural disasters such as tornadoes, hurricanes or tsunamis, is needed for effective debris removal and allocation of limited resources. Significant advances in aerial image acquisition have greatly enabled the possibilities for rapid and automated detection of debris. In this dissertation, a robust debris detection algorithm is proposed. Large scale aerial images are partitioned into homogeneous regions by interactive segmentation. Debris areas are identified based on extracted texture features.
Robust building detection is another important part of high resolution aerial imagery understanding. This dissertation develops a 3D scene classification algorithm for building detection using point clouds derived from multi-view imagery. Point clouds are divided into point clusters using Euclidean clustering. Individual point clusters are identified based on extracted spectral and 3D structural features.
The inspection of roof condition is an important step in damage claim processing in the insurance industry. Automated roof condition assessment from remotely sensed images is proposed in this dissertation. Initially, texture classification and a bag-of-words model were applied to assess the roof condition using features derived from the whole rooftop. However, considering the complexity of residential rooftop, a more sophisticated method is proposed to divide the task into two stages: 1) roof segmentation, followed by 2) classification of segmented roof regions. Deep learning techniques are investigated for both segmentation and classification. A deep learned feature is proposed and applied in a region merging segmentation algorithm. A fine-tuned deep network is adopted for roof segment classification and found to achieve higher accuracy than traditional methods using hand-crafted features.
Contributions of this study include the development of algorithms for debris detection using 2D images and building detection using 3D point clouds. For roof condition assessment, the solutions to this problem are explored in two directions: features derived from the whole rooftop and features extracted from each roof segments. Through our research, roof segmentation followed by segments classification was found to be a more promising method and the workflow processing developed and tested. Deep learning techniques are also investigated for both roof segmentation and segments classification. More unsupervised feature extraction techniques using deep learning can be explored in future work
Evaluation of Machine Learning Algorithms for Intrusion Detection System
Intrusion detection system (IDS) is one of the implemented solutions against
harmful attacks. Furthermore, attackers always keep changing their tools and
techniques. However, implementing an accepted IDS system is also a challenging
task. In this paper, several experiments have been performed and evaluated to
assess various machine learning classifiers based on KDD intrusion dataset. It
succeeded to compute several performance metrics in order to evaluate the
selected classifiers. The focus was on false negative and false positive
performance metrics in order to enhance the detection rate of the intrusion
detection system. The implemented experiments demonstrated that the decision
table classifier achieved the lowest value of false negative while the random
forest classifier has achieved the highest average accuracy rate
Detection of Dispersed Radio Pulses: A machine learning approach to candidate identification and classification
Searching for extraterrestrial, transient signals in astronomical data sets
is an active area of current research. However, machine learning techniques are
lacking in the literature concerning single-pulse detection. This paper
presents a new, two-stage approach for identifying and classifying dispersed
pulse groups (DPGs) in single-pulse search output. The first stage identified
DPGs and extracted features to characterize them using a new peak
identification algorithm which tracks sloping tendencies around local maxima in
plots of signal-to-noise ratio vs. dispersion measure. The second stage used
supervised machine learning to classify DPGs. We created four benchmark data
sets: one unbalanced and three balanced versions using three different
imbalance treatments.We empirically evaluated 48 classifiers by training and
testing binary and multiclass versions of six machine learning algorithms on
each of the four benchmark versions. While each classifier had advantages and
disadvantages, all classifiers with imbalance treatments had higher recall
values than those with unbalanced data, regardless of the machine learning
algorithm used. Based on the benchmarking results, we selected a subset of
classifiers to classify the full, unlabelled data set of over 1.5 million DPGs
identified in 42,405 observations made by the Green Bank Telescope. Overall,
the classifiers using a multiclass ensemble tree learner in combination with
two oversampling imbalance treatments were the most efficient; they identified
additional known pulsars not in the benchmark data set and provided six
potential discoveries, with significantly less false positives than the other
classifiers.Comment: 13 pages, accepted for publication in MNRAS, ref. MN-15-1713-MJ.R
Monitoring spatial sustainable development: Semi-automated analysis of satellite and aerial images for energy transition and sustainability indicators
Solar panels are installed by a large and growing number of households due to
the convenience of having cheap and renewable energy to power house appliances.
In contrast to other energy sources solar installations are distributed very
decentralized and spread over hundred-thousands of locations. On a global level
more than 25% of solar photovoltaic (PV) installations were decentralized. The
effect of the quick energy transition from a carbon based economy to a green
economy is though still very difficult to quantify. As a matter of fact the
quick adoption of solar panels by households is difficult to track, with local
registries that miss a large number of the newly built solar panels. This makes
the task of assessing the impact of renewable energies an impossible task.
Although models of the output of a region exist, they are often black box
estimations. This project's aim is twofold: First automate the process to
extract the location of solar panels from aerial or satellite images and
second, produce a map of solar panels along with statistics on the number of
solar panels. Further, this project takes place in a wider framework which
investigates how official statistics can benefit from new digital data sources.
At project completion, a method for detecting solar panels from aerial images
via machine learning will be developed and the methodology initially developed
for BE, DE and NL will be standardized for application to other EU countries.
In practice, machine learning techniques are used to identify solar panels in
satellite and aerial images for the province of Limburg (NL), Flanders (BE) and
North Rhine-Westphalia (DE).Comment: This document provides the reader with an overview of the various
datasets which will be used throughout the project. The collection of
satellite and aerial images as well as auxiliary information such as the
location of buildings and roofs which is required to train, test and validate
the machine learning algorithm that is being develope
- …