Prediction of traffic flow based on deep learning

Deep neural networks (DNNs) have recently demonstrated the capability to predict traffic flow with big data. Although existing DNN models can provide better performance than shallow models, it is still an open question to make full use of the spatio-temporal characteristics of traffic flows to improve performance. We propose a novel deep architecture combining CNN and LSTM for traffic flow (RCF) predictio. The model uses CNN to explore temporal correlation and LSTM to explore spatial correlation . Factors such as weather and historical period data are also added to the feature. Its advantage lies in making full use of the spatial-temporal correlation of traffic data and more comprehensively considered the impact of multiple related factors. Aiming at the difficult problem of obtaining spatial features, a feature selection method based on Random Forests is proposed. We use the gini score to represent the spatial connection between intersections to form a network graph constructed based on data.  The experimental results show that based on the random forest feature selection and RCF model, the accuracy of traffic prediction reaches 90%

Effective Features and Machine Learning Methods for Document Classification

Document classification has been involved in a variety of applications, such as phishing and fraud detection, news categorisation, and information retrieval. This thesis aims to provide novel solutions to several important problems presented by document classification. First, an improved Principal Components Analysis (PCA), based on similarity and correlation criteria instead of covariance, is proposed, which aims to capture low-dimensional feature subset that facilitates improved performance in text classification. The experimental results have demonstrated the advantages and usefulness of the proposed method for text classification in high-dimensional feature space in terms of the number of features required to achieve the best classification accuracy. Second, two hybrid feature-subset selection methods are proposed based on the combination (via either union or intersection) of the results of both supervised (in one method) and unsupervised (in the other method) filter approaches prior to the use of a wrapper, leading to low-dimensional feature subset that can achieve both high classification accuracy and good interpretability, and spend less processing time than most current methods. The experimental results have demonstrated the effectiveness of the proposed methods for feature subset selection in high-dimensional feature space in terms of the number of selected features and the processing time spent to achieve the best classification accuracy. Third, a class-specific (supervised) pre-trained approach based on a sparse autoencoder is proposed for acquiring low-dimensional interesting structure of relevant features, which can be used for high-performance document classification. The experimental results have demonstrated the merit of this proposed method for document classification in high-dimensional feature space, in terms of the limited number of features required to achieve good classification accuracy. Finally, deep classifier structures associated with a stacked autoencoder (SAE) for higher-level feature extraction are investigated, aiming to overcome the difficulties experienced in training deep neural networks with limited training data in high-dimensional feature space, such as overfitting and vanishing/exploding gradients. This investigation has resulted in a three-stage learning algorithm for training deep neural networks. In comparison with support vector machines (SVMs) combined with SAE and Deep Multilayer Perceptron (DMLP) with random weight initialisation, the experimental results have shown the advantages and effectiveness of the proposed three-stage learning algorithm

Feature selection using mutual information in network intrusion detection system

University of Technology Sydney. Faculty of Engineering and Information Technology.Network technologies have made significant progress in development, while the security issues alongside these technologies have not been well addressed. Current research on network security mainly focuses on developing preventative measures, such as security policies and secure communication protocols. Meanwhile, attempts have been made to protect computer systems and networks against malicious behaviours by deploying Intrusion Detection Systems (IDSs). The collaboration of IDSs and preventative measures can provide a safe and secure communication environment. Intrusion detection systems are now an essential complement to security project infrastructure of most organisations. However, current IDSs suffer from three significant issues that severely restrict their utility and performance. These issues are: a large number of false alarms, very high volume of network traffic and the classification problem when the class labels are not available. In this thesis, these three issues are addressed and efficient intrusion detection systems are developed which are effective in detecting a wide variety of attacks and result in very few false alarms and low computational cost. The principal contribution is the efficient and effective use of mutual information, which offers a solid theoretical framework for quantifying the amount of information that two random variables share with each other. The goal of this thesis is to develop an IDS that is accurate in detecting attacks and fast enough to make real-time decisions. First, a nonlinear correlation coefficient-based similarity measure to help extract both linear and nonlinear correlations between network traffic records is used. This measure is based on mutual information. The extracted information is used to develop an IDS to detect malicious network behaviours. However, the current network traffic data, which consist of a great number of traffic patterns, create a serious challenge to IDSs. Therefore, to address this issue, two feature selection methods are proposed; filter-based feature selection and hybrid feature selection algorithms, added to our current IDS for supervised classification. These methods are used to select a subset of features from the original feature set and use the selected subset to build our IDS and enhance the detection performance. The filter-based feature selection algorithm, named Flexible Mutual Information Feature Selection (FMIFS), uses the theoretical analyses of mutual information as evaluation criteria to measure the relevance between the input features and the output classes. To eliminate the redundancy among selected features, FMIFS introduces a new criterion to estimate the redundancy of the current selected features with respect to the previously selected subset of features. The hybrid feature selection algorithm is a combination of filter and wrapper algorithms. The filter method searches for the best subset of features using mutual information as a measure of relevance between the input features and the output class. The wrapper method is used to further refine the selected subset from the previous phase and select the optimal subset of features that can produce better accuracy. In addition to the supervised feature selection methods, the research is extended to unsupervised feature selection methods, and an Extended Laplacian score EL and a Modified Laplacian score ML methods are proposed which can select features in unsupervised scenarios. More specifically, each of EL and ML consists of two main phases. In the first phase, the Laplacian score algorithm is applied to rank the features by evaluating the power of locality preservation for each feature in the initial data. In the second phase, a new redundancy penalization technique uses mutual information to remove the redundancy among the selected features. The final output of these algorithms is then used to build the detection model. The proposed IDSs are then tested on three publicly available datasets, the KDD Cup 99, NSL-KDD and Kyoto dataset. Experimental results confirm the effectiveness and feasibility of these proposed solutions in terms of detection accuracy, false alarm rate, computational complexity and the capability of utilising unlabelled data. The unsupervised feature selection methods have been further tested on five more well-known datasets from the UCI Machine Learning Repository. These newly added datasets are frequently used in literature to evaluate the performance of feature selection methods. Furthermore, these datasets have different sample sizes and various numbers of features, so they are a lot more challenging for comprehensively testing feature selection algorithms. The experimental results show that ML performs better than EL and four other state-of-art methods (including the Variance score algorithm and the Laplacian score algorithm) in terms of the classification accuracy

Metode Deteksi Intrusi Menggunakan Algoritme Extreme Learning Machine dengan Correlation-based Feature Selection

Deteksi intrusi pada jaringan komputer merupakan kegiatan yang sangat penting dilakukan untuk menjaga keamanan data dan informasi. Deteksi intrusi merupakan proses monitor traffic pada sebuah jaringan untuk mendeteksi adanya pola data yang dianggap mencurigakan, yang memungkinkan terjadinya serangan jaringan. Penelitian ini melakukan analisis pada traffic jaringan untuk mengetahui apakah paket tersebut mengandung intrusi atau merupakan paket normal. Data traffic yang digunakan untuk deteksi intrusi pada penelitian ini diambil dari dataset KDD Cup. Metode yang digunakan untuk melakukan deteksi intrusi dengan cara klasifikasi yaitu dengan menggunakan metode Extreme Learning Machine (ELM). Namun, dengan menggunakan metode ELM saja tidak mampu untuk menghasilkan akurasi yang baik maka, pada metode ELM perlu ditambahkan metode seleksi fitur Correlation-Based Feature Selection (CFS) untuk meningkatkan hasil akurasi dan waktu komputasi. Hasil penelitian yang dilakukan dengan menggunakan metode ELM menunjukkan tingkat akurasi mencapai 81,97% dengan waktu komputasi 3,39 detik. Setelah ditambahkan metode seleksi fitur CFS pada ELM tingkat akurasi meningkat secara signifikan menjadi 98,00% dengan waktu komputasi 2,32 detik. AbstractIntrusion detection of computer networks is a very important activity carried out to maintain data and information security. Intrusion detection is the process of monitoring traffic on a network to detect any data patterns that are considered suspicious, which allows network attacks. This research analyzes the network traffic to find out whether the packet contains intrusion or is a normal packet. Traffic data used for intrusion detection in this study were taken from the KDD Cup dataset. The method used to do intrusion detection by classification is using the Extreme Learning Machine (ELM) method. However, using the ELM method alone is not able to produce good accuracy, so the ELM method needs to be added to the Correlation-Based Feature Selection (CFS) feature selection method to improve the accuracy and computational time. The results of the research conducted using the ELM method showed an accuracy rate of 81.97% with a computation time of 3.39 seconds. After adding the CFS feature selection method to ELM the accuracy level increased significantly to 98.00% with a computing time of 2.32 seconds

An optimized Deep Neural Network Approach for Vehicular Traffic Noise Trend Modelling

Vehicular traffic plays a significant role in terms of economic development; however, it is also a major source of noise pollution. Therefore, it is highly imperative to model traffic noise, especially for expressways due to their high traffic volume and speed, which produce very-high level of traffic noise. Previous traffic prediction models are mostly based on the regression approach and the artificial neural network (ANN), which often fail to describe the trends of noise. In this paper, a deep neural network-based optimization approach is implemented in two ways: i) using different algorithms for training and activation, and ii) integrating with feature selection methods such as correlation-based feature selection (CFS) and wrapper for feature-subset selection (WFS) methods. These methods are integrated to produce traffic noise maps for different time of the day on weekdays, including morning, afternoon, evening, and night. The novelty of this study is the integration of the feature selection method with the deep neural network for vehicular traffic noise modelling. New Klang Valley Expressway (NKVE) in Malaysia was used as a case study due to its increasing heavy and light vehicles, and the motorbike during peak hours, which result in high traffic noise. The results from the models indicate that the WFS-DNN model has the least mean-absolute-deviation (MAD) of 2.28, and the least root-mean-square-error (RMSE) of 3.97. Also, this model shows the best result compared to the other models such as DNN without integration with feature selection methods, CFS-DNN and the ANN networks (MLP and RBF). MAD improvement of 27% - 47% and RMSE improvement of 25% - 38% was achieved compared to other methods. The study provides a generic approach to key parameter selection and dimension reduction with novel trend descriptor which could be useful for future such modelling applications

Feature Ranking for Text Classifiers

Feature selection based on feature ranking has received much attention by researchers in the field of text classification. The major reasons are their scalability, ease of use, and fast computation. %, However, compared to the search-based feature selection methods such as wrappers and filters, they suffer from poor performance. This is linked to their major deficiencies, including: (i) feature ranking is problem-dependent; (ii) they ignore term dependencies, including redundancies and correlation; and (iii) they usually fail in unbalanced data. While using feature ranking methods for dimensionality reduction, we should be aware of these drawbacks, which arise from the function of feature ranking methods. In this thesis, a set of solutions is proposed to handle the drawbacks of feature ranking and boost their performance. First, an evaluation framework called feature meta-ranking is proposed to evaluate ranking measures. The framework is based on a newly proposed Differential Filter Level Performance (DFLP) measure. It was proved that, in ideal cases, the performance of text classifier is a monotonic, non-decreasing function of the number of features. Then we theoretically and empirically validate the effectiveness of DFLP as a meta-ranking measure to evaluate and compare feature ranking methods. The meta-ranking framework is also examined by a stopword extraction problem. We use the framework to select appropriate feature ranking measure for building domain-specific stoplists. The proposed framework is evaluated by SVM and Rocchio text classifiers on six benchmark data. The meta-ranking method suggests that in searching for a proper feature ranking measure, the backward feature ranking is as important as the forward one. Second, we show that the destructive effect of term redundancy gets worse as we decrease the feature ranking threshold. It implies that for aggressive feature selection, an effective redundancy reduction should be performed as well as feature ranking. An algorithm based on extracting term dependency links using an information theoretic inclusion index is proposed to detect and handle term dependencies. The dependency links are visualized by a tree structure called a term dependency tree. By grouping the nodes of the tree into two categories, including hub and link nodes, a heuristic algorithm is proposed to handle the term dependencies by merging or removing the link nodes. The proposed method of redundancy reduction is evaluated by SVM and Rocchio classifiers for four benchmark data sets. According to the results, redundancy reduction is more effective on weak classifiers since they are more sensitive to term redundancies. It also suggests that in those feature ranking methods which compact the information in a small number of features, aggressive feature selection is not recommended. Finally, to deal with class imbalance in feature level using ranking methods, a local feature ranking scheme called reverse discrimination approach is proposed. The proposed method is applied to a highly unbalanced social network discovery problem. In this case study, the problem of learning a social network is translated into a text classification problem using newly proposed actor and relationship modeling. Since social networks are usually sparse structures, the corresponding text classifiers become highly unbalanced. Experimental assessment of the reverse discrimination approach validates the effectiveness of the local feature ranking method to improve the classifier performance when dealing with unbalanced data. The application itself suggests a new approach to learn social structures from textual data

Data-driven methods for situation awareness and operational adjustment of sustainable energy integration into power systems

In the context of increasing complexity in power system operations due to the integration of renewable energy sources, two main challenges arise: accurate short-term wind power forecasting and power flow convergence control. Accurate wind power forecasting plays a crucial role in power system scheduling, while controlling power flow convergence is essential for system stability. This study proposes a concise short-term wind power generation prediction model that combines a feature selection-based convolutional neural network-bidirectional long short-term memory network (CNN-BiLSTM) model. By effectively screening multidimensional feature datasets, the model optimizes the selection of highly correlated feature parameters and assigns weights to input data based on feature correlation. The CNN-BiLSTM combination model is then employed to establish a predictive model for wind power generation based on multiple features. Additionally, this study introduces an automatic adjustment model for power flow convergence using the D3QN (Double Dueling Q Network) reinforcement learning algorithm. This addresses the challenge of power imbalance leading to flow non-convergence, enabling effective control of power flow convergence and adaptive adjustment of operating modes. Experiments conducted using the KDD Cup 2022 wind power prediction dataset validate the wind power prediction method. The results demonstrate that the CNN-BiLSTM model effectively utilizes time-series data, surpassing other neural networks in prediction accuracy. Simulation results based on the PYPOWER case39 standard case reveal that the reinforcement learning model’s reward value increases with training rounds and stabilizes at 40. Remarkably, more than 72% of abnormal flow samples achieve rapid convergence within 10 steps, affirming the proposed method's efficacy and computational efficiency. The findings of this study contribute to enhancing the accurate awareness of new energy integration into power systems and provide a novel adaptive control method for power flow

Improving Clustering-Based Forecasting of Aggregated Distribution Transformer Loadings With Gradient Boosting and Feature Selection

Load forecasting is more important than ever to enable new monitor and control functionalities of distribution networks aiming to mitigate the impact of the energy transition. Load forecasting at medium voltage (MV) level is becoming more challenging, because these load profiles become more stochastic due to the increasing penetration of photovoltaic (PV) generation in distribution networks. This work combines medium to low voltage (MV/LV) transformer loadings measured with advanced metering infrastructure (AMI) and machine learning (ML) algorithms to propose a new clustering based day-ahead aggregated load forecasting approach. This four-step approach improves the day-ahead load forecast of a city. First, MV/LV transformer loadings are clustered based on the shape of their load pattern. Second, a gradient boosting algorithm is used to forecast the load of each cluster and calculate the related feature importance. Third, feature selection is applied to improve the forecast accuracy of each cluster. Finally, the day-ahead load forecast of all clusters are aggregated. The case study presented uses 519 measured MV/LV transformer loadings in a city to perform 30 day-ahead load forecasts. Compared against the day-ahead aggregated load forecast without clustering, the average normalized root mean squared error (NRMSE) reduced 12.7 %, the average mean absolute percentage error (MAPE) reduced 18.2 % and the average Pearson Correlation Coefficient (PCC) increased 0.37 %. The 95 % confidence interval of the difference between the average NRMSE, MAPE and PCC without clustering and with the proposed method indicates a statistically significant improvement in accuracy

Using Kinect to classify Parkinson's disease stages related to severity of gait impairment

Background: Parkinson’s Disease (PD) is a chronic neurodegenerative disease associated with motor problems such as gait impairment. Different systems based on 3D cameras, accelerometers or gyroscopes have been used in related works in order to study gait disturbances in PD. Kinect has also been used to build these kinds of systems, but contradictory results have been reported: some works conclude that Kinect does not provide an accurate method of measuring gait kinematics variables, but others, on the contrary, report good accuracy results. Methods: In this work, we have built a Kinect-based system that can distinguish between different PD stages, and have performed a clinical study with 30 patients suffering from PD belonging to three groups: early PD patients without axial impairment, more evolved PD patients with higher gait impairment but without Freezing of Gait (FoG), and patients with advanced PD and FoG. Those patients were recorded by two Kinect devices when they were walking in a hospital corridor. The datasets obtained from the Kinect were preprocessed, 115 features identified, some methods were applied to select the relevant features (correlation based feature selection, information gain, and consistency subset evaluation), and different classification methods (decision trees, Bayesian networks, neural networks and K-nearest neighbours classifiers) were evaluated with the goal of finding the most accurate method for PD stage classification. Results: The classifier that provided the best results is a particular case of a Bayesian Network classifier (similar to a Naïve Bayesian classifier) built from a set of 7 relevant features selected by the correlation-based on feature selection method. The accuracy obtained for that classifier using 10-fold cross validation is 93.40%. The relevant features are related to left shin angles, left humerus angles, frontal and lateral bents, left forearm angles and the number of steps during spin. Conclusions: In this paper, it is shown that using Kinect is adequate to build a inexpensive and comfortable system that classifies PD into three different stages related to FoG. Compared to the results of previous works, the obtained accuracy (93.40%) can be considered high. The relevant features for the classifier are: a) movement and position of the left arm, b) trunk position for slightly displaced walking sequences, and c) left shin angle, for straight walking sequences. However, we have obtained a better accuracy (96.23%) for a classifier that only uses features extracted from slightly displaced walking steps and spin walking steps. Finally, the obtained set of relevant features may lead to new rehabilitation therapies for PD patients with gait problems

NEW STRATEGIES FOR IMPROVING NETWORK SECURITY AGAINST CYBER ATTACK BASED ON INTELLIGENT ALGORITHMS

Gradually, since the number of linked computer systems that use networks linked to the Internet is raised the information that is delivered through those systems becomes more vulnerable to cyber threats. This article presents proposed algorithms based on Machine Learning (ML) that ensure early detection of cyber threats that cause network breaking through the use of the Correlation Ranking Filter feature selection method. These proposed algorithms were applied to the Multi-Step Cyber-Attack Dataset (MSCAD) which consists of 66 features. The proposed strategy will apply machine learning algorithms like Adaptive Boosting-Deep Learning (AdaBoost-Deep Learning) or (ABDL), Multi-Layer Perceptron (MLP), Bayesian Networks Model (BNM), and Random Forest (RF), the feature would be decreased to high valuable of 46 features were included with a threshold of 0.1 or higher. The accuracy would be increased when the no. of features decreased to 46 with a threshold of ≥ 0.1 with the ABDL algorithm producing an accuracy of 99.7076%. The obtained results showed that the proposed algorithm delivered a suitable accuracy of 99.6791% with the ABDL algorithm even with a higher number of features