Adapted K-Nearest Neighbors for Detecting Anomalies on Spatio–Temporal Traffic Flow

Outlier detection is an extensive research area, which has been intensively studied in several domains such as biological sciences, medical diagnosis, surveillance, and traffic anomaly detection. This paper explores advances in the outlier detection area by finding anomalies in spatio-temporal urban traffic flow. It proposes a new approach by considering the distribution of the flows in a given time interval. The flow distribution probability (FDP) databases are first constructed from the traffic flows by considering both spatial and temporal information. The outlier detection mechanism is then applied to the coming flow distribution probabilities, the inliers are stored to enrich the FDP databases, while the outliers are excluded from the FDP databases. Moreover, a k-nearest neighbor for distance-based outlier detection is investigated and adopted for FDP outlier detection. To validate the proposed framework, real data from Odense traffic flow case are evaluated at ten locations. The results reveal that the proposed framework is able to detect the real distribution of flow outliers. Another experiment has been carried out on Beijing data, the results show that our approach outperforms the baseline algorithms for high-urban traffic flow

A theoretical framework for network monitoring exploiting segment routing counters

Self-driving networks represent the next step of network management techniques in the close future. A fundamental point for such an evolution is the use of Machine Learning based solutions to extract information from data coming from network devices during their activity. In this work we focus on a new type of data, available thanks to the definition of the novel SRv6 paradigm, referred to as SRv6 Traffic Counters (SRTCs). SRTCs provide aggregated measurements related to forwarding operations performed by SRv6 routers. In this work a detailed description of different SRTCs types (SR.INT, PISD, PSID.TM and POL) is provided and their relationships is formalized. The theoretical framework deployed is used to identify, on the basis of network configuration parameters of both SRv6 and IGP protocols, the minimum set of independent SRTCs to characterize the Network Status: we show that about the 80% of counters can be neglected with no information loss. We also apply our framework to two use cases: i) Traffic Matrix (TM) Assessment and ii) Traffic Anomaly Detection. For the TM assessment, we show that in a partially deployed SRv6 scenario a specific type of SRTCs, i.e., PSID, is more reliable than other ones; on the contrary, in a fully deployed scenario POL and PSID.TM counters provide the full TM knowledge. For the Traffic Anomaly Detection case, we show that known solutions based on link load measurements can be improved when integrating SRTCs information

Enhancing structural health monitoring with vehicle identification and tracking

Traffic load monitoring and structural health monitoring (SHM) have been gaining increasing attention over the last decade. However, most of the current installations treat the two monitoring types as separated problems, thereby using dedicated installed sensors, such as smart cameras for traffic load or accelerometers for Structural Health Monitoring (SHM). This paper presents a new framework aimed at leveraging the data collected by a SHM system for a second use, namely, monitoring vehicles passing on the structure being monitored (a viaduct). Our framework first processes the raw three-axial acceleration signals through a series of transformations and extracts its energy. Then, an anomaly detection algorithm is used to detect peaks from 90 installed sensors, and a linear regression together with a simple threshold filters out false detection by estimating the speed of the vehicles. Initial results in conditions of moderate traffic load are promising, demonstrating the detection of vehicles and realistic characterization of their speed. Moreover, a k-means clustering analysis distinguishes two groups of peaks with statistically different features such as amplitude and damping duration that could be likely associated with heavy vehicles and cars, respectively

XFinder: Detecting Unknown Anomalies in Distributed Machine Learning Scenario

In recent years, the emergence of distributed machine learning has enabled deep learning models to ensure data security and privacy while training efficiently. Anomaly detection for network traffic in distributed machine learning scenarios is of great significance for network security. Although deep neural networks have made remarkable achievements in anomaly detection for network traffic, they mainly focus on closed sets, that is, assuming that all anomalies are known. However, in a real network environment, unknown abnormalities are fatal risks faced by the system because they have no labels and occur before the known anomalies. In this study, we design and implement XFinder, a dynamic unknown traffic anomaly detection framework in distributed machine learning. XFinder adopts an online mode to detect unknown anomalies in real-time. XFinder detects unknown anomalies by the unknowns detector, transfers the unknown anomalies to the prior knowledge base by the network updater, and adopts the online mode to report new anomalies in real-time. The experimental results show that the average accuracy of the unknown anomaly detection of our model is increased by 27% and the average F1-Score is improved by 20%. Compared with the offline mode, XFinder’s detection time is reduced by an average of approximately 33% on three datasets, and can better meet the network requirement