An Overview on Application of Machine Learning Techniques in Optical Networks

Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, Machine Learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude the paper proposing new possible research directions

Data analytic approach for manipulation detection in stock market

The term “price manipulation” is used to describe the actions of “rogue” traders who employ carefully designed trading tactics to incur equity prices up or down to make profit. Such activities damage the proper functioning, integrity, and stability of the financial markets. In response to that, the regulators proposed new regulatory guidance to prohibit such activities on the financial markets. However, due to the lack of existing research and the implementation complexity, the application of those regulatory guidance, i.e. MiFID II in EU, is postponed to 2018. The existing studies exploring this issue either focus on empirical analysis of such cases, or propose detection models based on certain assumptions. The effective methods, based on analysing trading behaviour data, are not yet studied. This paper seeks to address that gap, and provides two data analytics based models. The first one, static model, detects manipulative behaviours through identifying abnormal patterns of trading activities. The activities are represented by transformed limit orders, in which the transformation method is proposed for partially reducing the non-stationarity nature of the financial data. The second one is hidden Markov model based dynamic model, which identifies the sequential and contextual changes in trading behaviours. Both models are evaluated using real stock tick data, which demonstrate their effectiveness on identifying a range of price manipulation scenarios, and outperforming the selected benchmarks. Thus, both models are shown to make a substantial contribution to the literature, and to offer a practical and effective approach to the identification of market manipulation

Performance Metrics for Network Intrusion Systems

Intrusion systems have been the subject of considerable research during the past 33 years, since the original work of Anderson. Much has been published attempting to improve their performance using advanced data processing techniques including neural nets, statistical pattern recognition and genetic algorithms. Whilst some significant improvements have been achieved they are often the result of assumptions that are difficult to justify and comparing performance between different research groups is difficult. The thesis develops a new approach to defining performance focussed on comparing intrusion systems and technologies. A new taxonomy is proposed in which the type of output and the data scale over which an intrusion system operates is used for classification. The inconsistencies and inadequacies of existing definitions of detection are examined and five new intrusion levels are proposed from analogy with other detection-based technologies. These levels are known as detection, recognition, identification, confirmation and prosecution, each representing an increase in the information output from, and functionality of, the intrusion system. These levels are contrasted over four physical data scales, from application/host through to enterprise networks, introducing and developing the concept of a footprint as a pictorial representation of the scope of an intrusion system. An intrusion is now defined as “an activity that leads to the violation of the security policy of a computer system”. Five different intrusion technologies are illustrated using the footprint with current challenges also shown to stimulate further research. Integrity in the presence of mixed trust data streams at the highest intrusion level is identified as particularly challenging. Two metrics new to intrusion systems are defined to quantify performance and further aid comparison. Sensitivity is introduced to define basic detectability of an attack in terms of a single parameter, rather than the usual four currently in use. Selectivity is used to describe the ability of an intrusion system to discriminate between attack types. These metrics are quantified experimentally for network intrusion using the DARPA 1999 dataset and SNORT. Only nine of the 58 attack types present were detected with sensitivities in excess of 12dB indicating that detection performance of the attack types present in this dataset remains a challenge. The measured selectivity was also poor indicting that only three of the attack types could be confidently distinguished. The highest value of selectivity was 3.52, significantly lower than the theoretical limit of 5.83 for the evaluated system. Options for improving selectivity and sensitivity through additional measurements are examined.Stochastic Systems Lt

On-line health monitoring of passive electronic components using digitally controlled power converter

This thesis presents System Identification based On-Line Health Monitoring to analyse the dynamic behaviour of the Switch-Mode Power Converter (SMPC), detect, and diagnose anomalies in passive electronic components. The anomaly detection in this research is determined by examining the change in passive component values due to degradation. Degradation, which is a long-term process, however, is characterised by inserting different component values in the power converter. The novel health-monitoring capability enables accurate detection of passive electronic components despite component variations and uncertainties and is valid for different topologies of the switch-mode power converter. The need for a novel on-line health-monitoring capability is driven by the need to improve unscheduled in-service, logistics, and engineering costs, including the requirement of Integrated Vehicle Health Management (IVHM) for electronic systems and components. The detection and diagnosis of degradations and failures within power converters is of great importance for aircraft electronic manufacturers, such as Thales, where component failures result in equipment downtime and large maintenance costs. The fact that existing techniques, including built-in-self test, use of dedicated sensors, physics-of-failure, and data-driven based health-monitoring, have yet to deliver extensive application in IVHM, provides the motivation for this research ... [cont.]

Response-based methods to measure road surface irregularity: a state-of-the-art review

"jats:sec" "jats:title"Purpose"/jats:title" "jats:p"With the development of smart technologies, Internet of Things and inexpensive onboard sensors, many response-based methods to evaluate road surface conditions have emerged in the recent decade. Various techniques and systems have been developed to measure road profiles and detect road anomalies for multiple purposes such as expedient maintenance of pavements and adaptive control of vehicle dynamics to improve ride comfort and ride handling. A holistic review of studies into modern response-based techniques for road pavement applications is found to be lacking. Herein, the focus of this article is threefold: to provide an overview of the state-of-the-art response-based methods, to highlight key differences between methods and thereby to propose key focus areas for future research."/jats:p" "/jats:sec" "jats:sec" "jats:title"Methods"/jats:title" "jats:p"Available articles regarding response-based methods to measure road surface condition were collected mainly from “Scopus” database and partially from “Google Scholar”. The search period is limited to the recent 15 years. Among the 130 reviewed documents, 37% are for road profile reconstruction, 39% for pothole detection and the remaining 24% for roughness index estimation."/jats:p" "/jats:sec" "jats:sec" "jats:title"Results"/jats:title" "jats:p"The results show that machine-learning techniques/data-driven methods have been used intensively with promising results but the disadvantages on data dependence have limited its application in some instances as compared to analytical/data processing methods. Recent algorithms to reconstruct/estimate road profiles are based mainly on passive suspension and quarter-vehicle-model, utilise fewer key parameters, being independent on speed variation and less computation for real-time/online applications. On the other hand, algorithms for pothole detection and road roughness index estimation are increasingly focusing on GPS accuracy, data aggregation and crowdsourcing platform for large-scale application. However, a novel and comprehensive system that is comparable to existing International Roughness Index and conventional Pavement Management System is still lacking."/jats:p" "/jats:sec Document type: Articl

Computational intelligent hybrid model for detecting disruptive trading activity

The term “disruptive trading behaviour” was first proposed by the U.S. Commodity Futures Trading Commission and is now widely used by US and EU regulation (MiFID II) to describe activities that create a misleading appearance of market liquidity or depth or an artificial price movement upward or downward according to their own purposes. Such activities, identified as a new form of financial fraud in EU regulations, damage the proper functioning and integrity of capital markets and are hence extremely harmful. While existing studies have explored this issue, they have, in most cases, either focused on empirical analysis of such cases or proposed detection models based on certain assumptions of the market. Effective methods that can analyse and detect such disruptive activities based on direct studies of trading behaviours have not been studied to date. There exists, accordingly, a knowledge gap in the literature. This paper seeks to address that gap and provides a hybrid model composed of two data-mining-based detection modules that effectively identify disruptive trading behaviours. The hybrid model is designed to work in an on-line scheme. The limit order stream is transformed, calculated and extracted as a feature stream. One detection module, “Single Order Detection,” detects disruptive behaviours by identifying abnormal patterns of every single trading order. Another module, “Order Sequence Detection,” approaches the problem by examining the contextual relationships of a sequence of trading orders using an extended hidden Markov model, which identifies whether sequential changes from the extracted features are manipulative activities (or not). Both models were evaluated using huge volumes of real tick data from the NASDAQ, which demonstrated that both are able to identify a range of disruptive trading behaviours and, furthermore, that they outperform the selected traditional benchmark models. Thus, this hybrid model is shown to make a substantial contribution to the literature on financial market surveillance and to offer a practical and effective approach for the identification of disruptive trading behaviour

Autonomous Abnormal Behaviour Detection Using Trajectory Analysis

Abnormal behaviour detection has attracted signification amount of attention in the past decade due to increased security concerns around the world. The amount of data from surveillance cameras have exceeded human capacity and there is a greater need for anomaly detection systems for crime monitoring. This paper proposes a solution to this problem in a reception area context by using trajectory extraction through Gaussian Mixture Models and Kalman Filter for data association. Here, trajectory analysis was performed on extracted trajectories to detect four different anomalies such as entering staff area, running, loitering and squatting down. The developed anomaly detection algorithms were tested on videos captured at Asia Pacific University’s reception area. These algorithms were able to achieve a promising detection accuracy of 89% and a false positive rate of 4.52%

Detecting Organizational Accounts from Twitter Based on Network and Behavioral Factors

abstract: With the rise of Online Social Networks (OSN) in the last decade, social network analysis has become a crucial research topic. The OSN graphs have unique properties that distinguish them from other types of graphs. In this thesis, five month Tweet corpus collected from Bangladesh - between June 2016 and October 2016 is analyzed, in order to detect accounts that belong to groups. These groups consist of official and non-official twitter handles of political organizations and NGOs in Bangladesh. A set of network, temporal, spatial and behavioral features are proposed to discriminate between accounts belonging to individual twitter users, news, groups and organization leaders. Finally, the experimental results are presented and a subset of relevant features is identified that lead to a generalizable model. Detection of tiny number of groups from large network is achieved with 0.8 precision, 0.75 recall and 0.77 F1 score. The domain independent network and behavioral features and models developed here are suitable for solving twitter account classification problem in any context.Dissertation/ThesisMasters Thesis Computer Science 201