Discovering critical traffic anomalies from GPS trajectories for urban traffic dynamics understanding

Traffic anomaly (e.g., traffic jams) detection is essential for the development of intelligent transportation systems in smart cities. In particular, detecting critical traffic anomalies (e.g., rare traffic anomalies, sudden accidents) are far more meaningful than detecting general traffic anomalies and more helpful to understand urban traffic dynamics. For example, emerging traffic jams are more significant than regular traffic jams caused by common road bottlenecks like traffic lights or toll road entrances;  and discovering the original location of traffic chaos in an area is more important than finding roads that are just congested. However, using existing traffic indicators that represent traffic conditions, such as traffic flows and speeds, for critical traffic anomaly detection may be not accurate enough. That is, they usually miss some traffic anomalies while wrongly identifying a normal traffic status as an anomaly. Moreover, most existing detection methods only detect general traffic anomalies but not critical traffic anomalies. In this thesis, we provide two new indicators: frequency of jams (captured by stop-point clusters) and Visible Outlier Indexes (VOIs) (based on the Kolmogorov-Smirnov test of speed) to capture critical traffic anomalies more accurately. The advantage of our proposed indicators is that they help separate critical traffic anomalies from general traffic anomalies. The former can discover rare anomalies with low frequency, and the latter can find unexpected anomalies (i.e., when the difference between the predicted VOI and the real VOI is great). Based on these two indicators, we provide three novel methods for comprehensive traffic anomaly analysis, including traffic anomaly identification, prediction, and root cause discovery. First, we provide a novel analysis of spatial-temporal jam frequencies (ASTJF) method for identifying rare traffic anomalies. In the ASTJF method, spatially close stop-points in a time bin are grouped into stop-point clusters (SPCs) using Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm; an SPC is an instance of a spatiotemporal jam. Then, we develop a new adapted Hausdorff distance to measure the similarity of two SPCs and put SPCs which are relevant to the same spatiotemporal jam into a group. Finally, we calculate the number of SPCs in a group as the frequency of the corresponding traffic jams; traffic anomalies are classified as regular jams with high frequency and emerging jams with low frequency. The ASTJF method can correctly identify critical traffic anomalies (i.e., emerging jams). Second, we propose a novel prediction approach -  Visible Outlier Indices and Meshed Spatiotemporal Neighborhoods (VOIMSN) method. In this method, the trajectory data from the given region's geographic spatial neighbors and its time-series neighbors are both converted to the abnormal scores measured by VOIs and quantified by the matrix grid as the input of the prediction model to improve the accuracy. This method provides a comprehensive analysis using all relevant data for building a reliable prediction model. In particular, the proposed meshed spatiotemporal neighborhoods with arbitrary shape, which comprises all potential anomalies instead of just past anomalies, is theoretically more accurate than a fixed-size neighborhood for anomaly prediction. Third, we provide an innovative and integrated root cause analysis method using VOI as the probabilistic indicator of traffic anomalies. This method proposes automatically learns spatiotemporal causal relationships from historical data to build an uneven diffusion model for detecting the root cause of anomalies (i.e., the origin of traffic chaos). It is demonstrated to be better than the heat diffusion model. Experiments conducted on a real-world massive trajectory dataset demonstrate the accuracy and effectiveness of the proposed methods for discovering critical traffic anomalies for a better understanding of urban traffic dynamics