An Effective and Efficient Intrusion Detection System of Network Attacks Using Stacked CNN and Voting Technique

IDS are crucial to network security because they can identify malicious activity and halt it in its tracks. Network intrusion data is often masked by a sea of benign data, making it difficult to train a model or perform a detection with a high FPR. This is because networks are inherently dynamic and change over time. In this research, we offer a ML & DL model-based method to ID, and we demonstrate how to deal with the issue of data imbalance by using a hybrid sampling technique. Conventional firewalls and data encryption technologies are unable to provide the level of security required by current networks. As a result, IDSs have been endorsed for use against network threats. Recent mainstream ID approaches have benefited from ML, but they have low detection rates & need a lot of feature engineering to be truly useful. Using layered CNN and Voting classifier (XGBoost and LGBM), this study introduces ML-DL-NIDS to address the issue of subpar detection precision. Using a publicly available NSL-KDD & UNSW-15 benchmark datasets for network intrusion detection, we find that this model outperforms competing methods according to accuracy and F1-score obtained from experimental evaluations

Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms

Many different machine learning algorithms exist; taking into account each algorithm's hyperparameters, there is a staggeringly large number of possible alternatives overall. We consider the problem of simultaneously selecting a learning algorithm and setting its hyperparameters, going beyond previous work that addresses these issues in isolation. We show that this problem can be addressed by a fully automated approach, leveraging recent innovations in Bayesian optimization. Specifically, we consider a wide range of feature selection techniques (combining 3 search and 8 evaluator methods) and all classification approaches implemented in WEKA, spanning 2 ensemble methods, 10 meta-methods, 27 base classifiers, and hyperparameter settings for each classifier. On each of 21 popular datasets from the UCI repository, the KDD Cup 09, variants of the MNIST dataset and CIFAR-10, we show classification performance often much better than using standard selection/hyperparameter optimization methods. We hope that our approach will help non-expert users to more effectively identify machine learning algorithms and hyperparameter settings appropriate to their applications, and hence to achieve improved performance.Comment: 9 pages, 3 figure

Machine Learning For In-Region Location Verification In Wireless Networks

In-region location verification (IRLV) aims at verifying whether a user is inside a region of interest (ROI). In wireless networks, IRLV can exploit the features of the channel between the user and a set of trusted access points. In practice, the channel feature statistics is not available and we resort to machine learning (ML) solutions for IRLV. We first show that solutions based on either neural networks (NNs) or support vector machines (SVMs) and typical loss functions are Neyman-Pearson (N-P)-optimal at learning convergence for sufficiently complex learning machines and large training datasets . Indeed, for finite training, ML solutions are more accurate than the N-P test based on estimated channel statistics. Then, as estimating channel features outside the ROI may be difficult, we consider one-class classifiers, namely auto-encoders NNs and one-class SVMs, which however are not equivalent to the generalized likelihood ratio test (GLRT), typically replacing the N-P test in the one-class problem. Numerical results support the results in realistic wireless networks, with channel models including path-loss, shadowing, and fading

A Prediction Modeling Framework For Noisy Welding Quality Data

Numerous and various research projects have been conducted to utilize historical manufacturing process data in product design. These manufacturing process data often contain data inconsistencies, and it causes challenges in extracting useful information from the data. In resistance spot welding (RSW), data inconsistency is a well-known issue. In general, such inconsistent data are treated as noise data and removed from the original dataset before conducting analyses or constructing prediction models. This may not be desirable for every design and manufacturing applications since every data can contain important information to further explain the process. In this research, we propose a prediction modeling framework, which employs bootstrap aggregating (bagging) with support vector regression (SVR) as the base learning algorithm to improve the prediction accuracy on such noisy data. Optimal hyper-parameters for SVR are selected by particle swarm optimization (PSO) with meta-modeling. Constructing bagging models require 114 more computational costs than a single model. Also, evolutionary computation algorithms, such as PSO, generally require a large number of candidate solution evaluations to achieve quality solutions. These two requirements greatly increase the overall computational cost in constructing effective bagging SVR models. Meta-modeling can be employed to reduce the computational cost when the fitness or constraints functions are associated with computationally expensive tasks or analyses. In our case, the objective function is associated with constructing bagging SVR models with candidate sets of hyper-parameters. Therefore, in regards to PSO, a large number of bagging SVR models have to be constructed and evaluated, which is computationally expensive. The meta-modeling approach, called MUGPSO, developed in this research assists PSO in evaluating these candidate solutions (i.e., sets of hyper-parameters). MUGPSO approximates the fitness function of candidate solutions. Through this method, the numbers of real fitness function evaluations (i.e., constructing bagging SVR models) are reduced, which also reduces the overall computational costs. Using the Meta2 framework, one can expect an improvement in the prediction accuracy with reduced computational time. Experiments are conducted on three artificially generated noisy datasets and a real RSW quality dataset. The results indicate that Meta2 is capable of providing promising solutions with noticeably reduced computational costs

Ensemble based on randomised neural networks for online data stream regression in presence of concept drift

The big data paradigm has posed new challenges for the Machine Learning algorithms, such as analysing continuous flows of data, in the form of data streams, and dealing with the evolving nature of the data, which cause a phenomenon often referred to in the literature as concept drift. Concept drift is caused by inconsistencies between the optimal hypotheses in two subsequent chunks of data, whereby the concept underlying a given process evolves over time, which can happen due to several factors including change in consumer preference, economic dynamics, or environmental conditions. This thesis explores the problem of data stream regression with the presence of concept drift. This problem requires computationally efficient algorithms that are able to adapt to the various types of drift that may affect the data. The development of effective algorithms for data streams with concept drift requires several steps that are discussed in this research. The first one is related to the datasets required to assess the algorithms. In general, it is not possible to determine the occurrence of concept drift on real-world datasets; therefore, synthetic datasets where the various types of concept drift can be simulated are required. The second issue is related to the choice of the algorithm. The ensemble algorithms show many advantages to deal with concept drifting data streams, which include flexibility, computational efficiency and high accuracy. For the design of an effective ensemble, this research analyses the use of randomised Neural Networks as base models, along with their optimisation. The optimisation of the randomised Neural Networks involves design and tuning hyperparameters which may substantially affect its performance. The optimisation of the base models is an important aspect to build highly accurate and computationally efficient ensembles. To cope with the concept drift, the existing methods either require setting fixed updating points, which may result in unnecessary computations or slow reaction to concept drift, or rely on drifting detection mechanism, which may be ineffective due to the difficulty to detect drift in real applications. Therefore, the research contributions of this thesis include the development of a new approach for synthetic dataset generation, development of a new hyperparameter optimisation algorithm that reduces the search effort and the need of prior assumptions compared to existing methods, the analysis of the effects of randomised Neural Networks hyperparameters, and the development of a new ensemble algorithm based on bagging meta-model that reduces the computational effort over existing methods and uses an innovative updating mechanism to cope with concept drift. The algorithms have been tested on synthetic datasets and validated on four real-world datasets from various application domains

Evaluating an automated procedure of machine learning parameter tuning for software effort estimation

Software effort estimation requires accurate prediction models. Machine learning algorithms have been used to create more accurate estimation models. However, these algorithms are sensitive to factors such as the choice of hyper-parameters. To reduce this sensitivity, automated approaches for hyper-parameter tuning have been recently investigated. There is a need for further research on the effectiveness of such approaches in the context of software effort estimation. These evaluations could help understand which hyper-parameter settings can be adjusted to improve model accuracy, and in which specific contexts tuning can benefit model performance. The goal of this work is to develop an automated procedure for machine learning hyper-parameter tuning in the context of software effort estimation. The automated procedure builds and evaluates software effort estimation models to determine the most accurate evaluation schemes. The methodology followed in this work consists of first performing a systematic mapping study to characterize existing hyper-parameter tuning approaches in software effort estimation, developing the procedure to automate the evaluation of hyper-parameter tuning, and conducting controlled quasi experiments to evaluate the automated procedure. From the systematic literature mapping we discovered that effort estimation literature has favored the use of grid search. The results we obtained in our quasi experiments demonstrated that fast, less exhaustive tuners were viable in place of grid search. These results indicate that randomly evaluating 60 hyper-parameters can be as good as grid search, and that multiple state-of-the-art tuners were only more effective than this random search in 6% of the evaluated dataset-model combinations. We endorse random search, genetic algorithms, flash, differential evolution, and tabu and harmony search as effective tuners.Los algoritmos de aprendizaje automático han sido utilizados para crear modelos con mayor precisión para la estimación del esfuerzo del desarrollo de software. Sin embargo, estos algoritmos son sensibles a factores, incluyendo la selección de hiper parámetros. Para reducir esto, se han investigado recientemente algoritmos de ajuste automático de hiper parámetros. Es necesario evaluar la efectividad de estos algoritmos en el contexto de estimación de esfuerzo. Estas evaluaciones podrían ayudar a entender qué hiper parámetros se pueden ajustar para mejorar los modelos, y en qué contextos esto ayuda el rendimiento de los modelos. El objetivo de este trabajo es desarrollar un procedimiento automatizado para el ajuste de hiper parámetros para algoritmos de aprendizaje automático aplicados a la estimación de esfuerzo del desarrollo de software. La metodología seguida en este trabajo consta de realizar un estudio de mapeo sistemático para caracterizar los algoritmos de ajuste existentes, desarrollar el procedimiento automatizado, y conducir cuasi experimentos controlados para evaluar este procedimiento. Mediante el mapeo sistemático descubrimos que la literatura en estimación de esfuerzo ha favorecido el uso de la búsqueda en cuadrícula. Los resultados obtenidos en nuestros cuasi experimentos demostraron que algoritmos de estimación no-exhaustivos son viables para la estimación de esfuerzo. Estos resultados indican que evaluar aleatoriamente 60 hiper parámetros puede ser tan efectivo como la búsqueda en cuadrícula, y que muchos de los métodos usados en el estado del arte son solo más efectivos que esta búsqueda aleatoria en 6% de los escenarios. Recomendamos el uso de la búsqueda aleatoria, algoritmos genéticos y similares, y la búsqueda tabú y harmónica.Escuela de Ciencias de la Computación e InformáticaCentro de Investigaciones en Tecnologías de la Información y ComunicaciónUCR::Vicerrectoría de Investigación::Sistema de Estudios de Posgrado::Ingeniería::Maestría Académica en Computación e Informátic

Intelligent Multi-Attribute Decision Making Applications: Decision Support Systems for Performance Measurement, Evaluation and Benchmarking

Efficiency has been and continues to be an important attribute of competitive business environments where limited resources exist. Owing to growing complexity of organizations and more broadly, to global economic growth, efficiency considerations are expected to remain a top priority for organizations. Continuous performance evaluations play a significant role in sustaining efficient and effective business processes. Consequently, the literature offers a wide range of performance evaluation methodologies to assess the operational efficiency of various industries. Majority of these models focus solely on quantitative criteria omitting qualitative data. However, a thorough performance measurement and benchmarking require consideration of all available information since accurately describing and defining complex systems require utilization of both data types. Most evaluation models also function under the unrealistic assumption of evaluation criteria being dependent on one another. Furthermore, majority of these methodologies tend to utilize discrete and contemporary information eliminating historical performance data from the model environment. These shortcomings hinder the reliability of evaluation outcomes leading to inadequate performance evaluations for many businesses. This problem gains more significance for business where performance evaluations are tied in to important decisions relating to business expansion, investment, promotion and compensation. The primary purpose of this research is to present a thorough, equitable and accurate evaluation framework for operations management while filling the existing gaps in the literature. Service industry offers a more suitable platform for this study since the industry tend to accommodate both qualitative and quantitative performance evaluation factors relatively with more ease compared to manufacturing due to the intensity of customer (consumer) interaction. Accordingly, a U.S. based food franchise company is utilized for data acquisition and as a case study to demonstrate the applications of the proposed models. Compatible with their multiple criteria nature, performance measurement, evaluation and benchmarking systems require heavy utilization of Multi-Attribute Decision Making (MADM) approaches which constitute the core of this research. In order to be able to accommodate the vagueness in decision making, fuzzy values are also utilized in all proposed models. In the first phase of the study, the main and sub-criteria in the evaluation are considered independently in a hierarchical order and contemporary data is utilized in a holistic approach combining three different multi-criteria decision making methods. The cross-efficiency approach is also introduced in this phase. Building on this approach, the second phase considered the influence of the main and sub-criteria over one another. That is, in the proposed models, the main and sub-criteria form a network with dependencies rather than having a hierarchical relationship. The decision making model is built to extract the influential weights for the evaluation criteria. Furthermore, Group Decision Making (GDM) is introduced to integrate different perspectives and preferences of multiple decision makers who are responsible for different functions in the organization with varying levels of impact on decisions. Finally, an artificial intelligence method is applied to utilize the historical data and to obtain the final performance ranking. Owing to large volumes of data emanating from digital sources, current literature offers a variety of artificial intelligence and machine learning methods for big data analytics applications. Comparing the results generated by the ANNs, three additional well-established methods, viz., Adaptive Neuro Fuzzy Inference System (ANFIS), Least Squares Support Vector Machine (LSSVM) and Extreme Learning Machine (ELM), are also employed for the same problem. In order to test the prediction capability of these methods, the most influencing criteria are obtained from the data set via Pearson Correlation Analysis and grey relational analysis. Subsequently, the corresponding parameters in each method are optimized via Particle Swarm Optimization to improve the prediction accuracy. The accuracy of artificial intelligence and machine learning methods are heavily reliant on large volumes of data. Despite the fact that several businesses, especially business that utilize social media data or on-line real-time operational data, there are organizations which lack adequate amount of data required for their performance evaluations simply due to the nature of their business. Grey Modeling (GM) technique addresses this issue and provides higher forecasting accuracy in presence of uncertain and limited data. With this motivation, a traditional multi-variate grey model is applied to predict the performance scores. Improved grey models are also applied to compare the results. Finally, the integration of the fractional order accumulation along with the background value coefficient optimization are proposed to improve accuracy