Application of big data in transportation safety analysis using statistical and deep learning methods

The emergence of new sensors and data sources provides large scale high-resolution big data from instantaneous vehicular movements, driver decision and states, surrounding environment, roadway characteristics, weather condition, etc. Such a big data can be served to expand our understanding regarding the current state of the transportation and help us to proactively evaluate and monitor the system performance. The key idea behind this dissertation is to identify the moments and locations where drivers are exhibiting different behavior comparing to the normal behavior. The concept of driving volatility is utilized which quantifies deviation from normal driving in terms of variations in speed, acceleration/deceleration, and vehicular jerk. This idea is utilized to explore the association of volatility in different hierarchies of transportation system, i.e.: 1) Instance level; 2) Event level; 3) Driver level; 4) Intersection level; and 5) Network level. In summary, the main contribution of this dissertation is exploring the association of variations in driving behavior in terms of driving volatility at different levels by harnessing big data generated from emerging data sources under real-world condition, which is applicable to the intelligent transportation systems and smart cities. By analyzing real-world crashes/near-crashes and predicting occurrence of extreme event, proactive warnings and feedback can be generated to warn drivers and adjacent vehicles regarding potential hazard. Furthermore, the results of this study help agencies to proactively monitor and evaluate safety performance of the network and identify locations where crashes are waiting to happen. The main objective of this dissertation is to integrate big data generated from emerging sources into safety analysis by considering different levels in the system. To this end, several data sources including Connected Vehicles data (with more than 2.2 billion seconds of observations), naturalistic driving data (with more than 2 million seconds of observations from vehicular kinematics and driver behavior), conventional data on roadway factors and crash data are integrated

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Multilane traffic density estimation with KDE and nonlinear LS and tracking with Scalar Kalman filtering

Tezin basılısı, İstanbul Şehir Üniversitesi Kütüphanesi'ndedir.With increasing population, the determination of traffic density becomes very critical in managing the urban city roads for safer driving and low carbon emission. In this study, Kernel Density Estimation is utilized in order to estimate the traffic density more accurately when the speeds of the vehicles are available for a given region. For the proposed approach, as a first step, the probability density function of the speed data is modeled by Kernel Density Estimation. Then, the speed centers from the density function are modeled as clusters. The cumulative distribution function of the speed data is then determined by Kolmogorov-Smirnov Test, whose complexity is less when compared to the other techniques and whose robustness is high when outliers exist. Then, the mean values of clusters are estimated from the smoothed density function of the distribution function, followed by a peak detection algorithm. The estimation of variance values and kernel weights, on the other hand, are found by a nonlinear Least Square approach. As the estimation problem has linear and non-linear components, the nonlinear Least Square with separation of parameters approach is adopted, instead of dealing with a high complexity nonlinear equation. Finally, the tracking of former and latter estimations of a road is calculated by using Scalar Kalman Filtering with scalar state - scalar observation generality level. Simulations are carried out in order to assess theperformanceoftheproposedapproach. Forallexampledatasets, theminimummean square error of kernel weights is found to be less than 0.002 while error of mean values is found to be less than 0.261. The proposed approach was also applied to real data from sample road traffic, and the speed center and the variance was accurately estimated. By using the proposed approach, accurate traffic density estimation is realized, providing extra information to the municipalities for better planning of their cities.Declaration of Authorship ii Abstract iii Öz iv Acknowledgments vi List of Figures ix List of Tables x Abbreviations xi 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Methods to Find Probability Density Function and Cumulative Distribution Function . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 1.3 Traffic Density Estimation with Kernel Density Estimation . . . . . . . . 4 1.4 The Approaches for Determination of Key Parameters of Traffic Density Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . .5 1.5 Tracking between Estimated Data and New Data . . . . . . . . . . . . . . 6 1.6 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Literature Review 7 2.1 Methodologies Used for Estimation of Traffic Density . . . . . . . . . . . . 7 2.2 An Example Study of Traffic Density Estimation with KDE and CvM . . 9 2.3 Three Complementary Studies for Traffic Density Estimation and Tracking 9 2.4 Comparison of Three Different Nonlinear Estimation Techniques on the Same Problem . . . . . . . . . . . . . . . . . . . . . . . . .10 2.4.1 A Maximum Likelihood Approach for Estimating DS-CDMA Multipath Fading Channels . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.2 Channel Estimation for the Uplink of a DS-CDMA System . . . . 12 2.4.3 A Robust Method for Estimating Multipath Channel Parameters in the Uplink of a DS-CDMA System. . . . . . . . . . . . . . .13 3 The Model 16 3.1 Finding Density Distribution with KDE . . . . . . . . . . . . . . . . . . . 16 3.2 Finding Empirical CDF with KS Test . . . . . . . . . . . . . . . . . . . . 18 3.3 Determination of Speed Centers via PDA . . . . . . . . . . . . . . . . . . 20 3.4 Estimation of Variance and Kernel Weights with Nonlinear LS Method . . 21 3.5 Tracking of Traffic Density Estimation with Scalar Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4 Numerical Calculations for Traffic Density Estimation 26 4.1 An Example Traffic Scenario with Five Speed Centers . . . . . . . . . . . 26 4.2 The Estimation of A Real Time Data . . . . . . . . . . . . . . . . . . . . . 29 4.3 Traffic Density Estimation with Different Kernel Numbers . . . . . . . . . 29 5 Examples to Test Tracking Part of the Model 31 5.1 Tracking with the Change only in Mean Values . . . . . . . . . . . . . . . 32 5.2 Tracking with the Change only in Kernel Weights . . . . . . . . . . . . . . 35 5.3 Tracking with the Change in All Three Parameters . . . . . . . . . . . . . 36 6 Assesment 38 7 Conclusion 41 A Derivation of Newton-Raphson Method for the Estimation of Variance Values and Kernel Weights 43 Bibliography 4