CERN Storage Systems for Large-Scale Wireless

The project aims at evaluating the use of CERN computing infrastructure for next generation sensor networks data analysis. The proposed system allows the simulation of a large-scale sensor array for traffic analysis, streaming data to CERN storage systems in an efficient way. The data are made available for offline and quasi-online analysis, enabling both long term planning and fast reaction on the environment

Big Data Approach for Secure Traffic Data Analytics using Hadoop

As the volume of traffic is increasing day by day, it gets difficult to store and process such huge sets of data using traditional software. A cluster of storage devices is needed to store such huge amounts of data and also a parallel computing model for analyzing those huge inputs of data. Hadoop is one such framework that provides reliable cluster of storage facility, which stores huge data in a distributed manner using a special file system, called Hadoop Distributed File System and provides efficient parallel processing feature through MapReduce framework. Using Map Reduce the filtered traffic data can be fetched easily, to provide end user with traffic analysis and giving useful predictions

A MapReduce-based nearest neighbor approach for big-data-driven traffic flow prediction

In big-data-driven traffic flow prediction systems, the robustness of prediction performance depends on accuracy and timeliness. This paper presents a new MapReduce-based nearest neighbor (NN) approach for traffic flow prediction using correlation analysis (TFPC) on a Hadoop platform. In particular, we develop a real-time prediction system including two key modules, i.e., offline distributed training (ODT) and online parallel prediction (OPP). Moreover, we build a parallel k-nearest neighbor optimization classifier, which incorporates correlation information among traffic flows into the classification process. Finally, we propose a novel prediction calculation method, combining the current data observed in OPP and the classification results obtained from large-scale historical data in ODT, to generate traffic flow prediction in real time. The empirical study on real-world traffic flow big data using the leave-one-out cross validation method shows that TFPC significantly outperforms four state-of-the-art prediction approaches, i.e., autoregressive integrated moving average, Naïve Bayes, multilayer perceptron neural networks, and NN regression, in terms of accuracy, which can be improved 90.07% in the best case, with an average mean absolute percent error of 5.53%. In addition, it displays excellent speedup, scaleup, and sizeup

From Social Data Mining to Forecasting Socio-Economic Crisis

Socio-economic data mining has a great potential in terms of gaining a better understanding of problems that our economy and society are facing, such as financial instability, shortages of resources, or conflicts. Without large-scale data mining, progress in these areas seems hard or impossible. Therefore, a suitable, distributed data mining infrastructure and research centers should be built in Europe. It also appears appropriate to build a network of Crisis Observatories. They can be imagined as laboratories devoted to the gathering and processing of enormous volumes of data on both natural systems such as the Earth and its ecosystem, as well as on human techno-socio-economic systems, so as to gain early warnings of impending events. Reality mining provides the chance to adapt more quickly and more accurately to changing situations. Further opportunities arise by individually customized services, which however should be provided in a privacy-respecting way. This requires the development of novel ICT (such as a self- organizing Web), but most likely new legal regulations and suitable institutions as well. As long as such regulations are lacking on a world-wide scale, it is in the public interest that scientists explore what can be done with the huge data available. Big data do have the potential to change or even threaten democratic societies. The same applies to sudden and large-scale failures of ICT systems. Therefore, dealing with data must be done with a large degree of responsibility and care. Self-interests of individuals, companies or institutions have limits, where the public interest is affected, and public interest is not a sufficient justification to violate human rights of individuals. Privacy is a high good, as confidentiality is, and damaging it would have serious side effects for society.Comment: 65 pages, 1 figure, Visioneer White Paper, see http://www.visioneer.ethz.c

Big data traffic management in vehicular ad-hoc network

Today, the world has experienced a new trend with regard to data system management, traditional database management tools have become outdated and they will no longer be able to process the mass of data generated by different systems, that's why big data is there to process this mass of data to bring out crucial information hidden in this data, and without big data technologies the treatment is very difficult to manage; among the domains that uses big data technologies is vehicular ad-hoc network to manage their voluminous data. In this article, we establish in the first step a method that allow to detect anomalies or accidents within the road and compute the time spent in each road section in real time, which permit us to obtain a database having the estimated time spent in all sections in real time, this will serve us to send to the vehicles the right estimated time of arrival all along their journey and the optimal route to attain their destination. This database is useful to utilize it like inputs for machine learning to predict the places and times where the probability of accidents is higher. The experimental results prove that our method permits us to avoid congestions and apportion the load of vehicles in all roads effectively, also it contributes to road safety

Big Data for Traffic Estimation and Prediction: A Survey of Data and Tools

Big data has been used widely in many areas including the transportation industry. Using various data sources, traffic states can be well estimated and further predicted for improving the overall operation efficiency. Combined with this trend, this study presents an up-to-date survey of open data and big data tools used for traffic estimation and prediction. Different data types are categorized and the off-the-shelf tools are introduced. To further promote the use of big data for traffic estimation and prediction tasks, challenges and future directions are given for future studies

An efficient MapReduce-based parallel clustering algorithm for distributed traffic subarea division

Traffic subarea division is vital for traffic system management and traffic network analysis in intelligent transportation systems (ITSs). Since existing methods may not be suitable for big traffic data processing, this paper presents a MapReduce-based Parallel Three-Phase K -Means (Par3PKM) algorithm for solving traffic subarea division problem on a widely adopted Hadoop distributed computing platform. Specifically, we first modify the distance metric and initialization strategy of K -Means and then employ a MapReduce paradigm to redesign the optimized K -Means algorithm for parallel clustering of large-scale taxi trajectories. Moreover, we propose a boundary identifying method to connect the borders of clustering results for each cluster. Finally, we divide traffic subarea of Beijing based on real-world trajectory data sets generated by 12,000 taxis in a period of one month using the proposed approach. Experimental evaluation results indicate that when compared with K -Means, Par2PK-Means, and ParCLARA, Par3PKM achieves higher efficiency, more accuracy, and better scalability and can effectively divide traffic subarea with big taxi trajectory data

Research on the Integration of Urban Traffic and Big Data

The powerful data processing ability of big data technology can allocate traffic resources more efficiently, and can deal with various sudden traffic problems flexibly. It is an unprecedented opportunity and challenge for urban transportation and smart cities to effectively collect and utilize traffic big data to meet the application requirements of high timeliness traffic administrative supervision, traffic enterprise management and traffic citizen service. This article expounds the concept and characteristics of big data, discusses the application research of big data in urban transportation at home and abroad in recent years, summarizes its application research scope and trend, points out that intelligent transportation is the focus of the application of big data in urban transportation, and finally looks forward to the future research direction