Big Data for Traffic Estimation and Prediction: A Survey of Data and Tools

Big data has been used widely in many areas including the transportation industry. Using various data sources, traffic states can be well estimated and further predicted for improving the overall operation efficiency. Combined with this trend, this study presents an up-to-date survey of open data and big data tools used for traffic estimation and prediction. Different data types are categorized and the off-the-shelf tools are introduced. To further promote the use of big data for traffic estimation and prediction tasks, challenges and future directions are given for future studies

A Map-matching Algorithm with Extraction of Multi-group Information for Low-frequency Data

The growing use of probe vehicles generates a huge number of GNSS data. Limited by the satellite positioning technology, further improving the accuracy of map-matching is challenging work, especially for low-frequency trajectories. When matching a trajectory, the ego vehicle's spatial-temporal information of the present trip is the most useful with the least amount of data. In addition, there are a large amount of other data, e.g., other vehicles' state and past prediction results, but it is hard to extract useful information for matching maps and inferring paths. Most map-matching studies only used the ego vehicle's data and ignored other vehicles' data. Based on it, this paper designs a new map-matching method to make full use of "Big data". We first sort all data into four groups according to their spatial and temporal distance from the present matching probe which allows us to sort for their usefulness. Then we design three different methods to extract valuable information (scores) from them: a score for speed and bearing, a score for historical usage, and a score for traffic state using the spectral graph Markov neutral network. Finally, we use a modified top-K shortest-path method to search the candidate paths within an ellipse region and then use the fused score to infer the path (projected location). We test the proposed method against baseline algorithms using a real-world dataset in China. The results show that all scoring methods can enhance map-matching accuracy. Furthermore, our method outperforms the others, especially when GNSS probing frequency is less than 0.01 Hz.Comment: 10 pages, 11 figures, 4 table

Hybrid group anomaly detection for sequence data: application to trajectory data analytics

Many research areas depend on group anomaly detection. The use of group anomaly detection can maintain and provide security and privacy to the data involved. This research attempts to solve the deficiency of the existing literature in outlier detection thus a novel hybrid framework to identify group anomaly detection from sequence data is proposed in this paper. It proposes two approaches for efficiently solving this problem: i) Hybrid Data Mining-based algorithm, consists of three main phases: first, the clustering algorithm is applied to derive the micro-clusters. Second, the kNN algorithm is applied to each micro-cluster to calculate the candidates of the group's outliers. Third, a pattern mining framework gets applied to the candidates of the group's outliers as a pruning strategy, to generate the groups of outliers, and ii) a GPU-based approach is presented, which benefits from the massively GPU computing to boost the runtime of the hybrid data mining-based algorithm. Extensive experiments were conducted to show the advantages of different sequence databases of our proposed model. Results clearly show the efficiency of a GPU direction when directly compared to a sequential approach by reaching a speedup of 451. In addition, both approaches outperform the baseline methods for group detection.acceptedVersio

State of the Art, Trends and Future of Bluetooth Low Energy, Near Field Communication and Visible Light Communication in the Development of Smart Cities

The current social impact of new technologies has produced major changes in all areas of society, creating the concept of a smart city supported by an electronic infrastructure, telecommunications and information technology. This paper presents a review of Bluetooth Low Energy (BLE), Near Field Communication (NFC) and Visible Light Communication (VLC) and their use and influence within different areas of the development of the smart city. The document also presents a review of Big Data Solutions for the management of information and the extraction of knowledge in an environment where things are connected by an “Internet of Things” (IoT) network. Lastly, we present how these technologies can be combined together to benefit the development of the smart city

Patterns of mobility in a smart city

Transportation data in smart cities is becoming increasingly available. This data allows building meaningful, intelligent solutions for city residents and city management authorities, the so-called Intelligent Transportation Systems. Our research focused on Lisbon mobility data, provided by Lisbon municipality. The main research objective was to address mobility problems, interdependence, and cascading effects solutions for the city of Lisbon. We developed a data-driven approach based on historical data with a strong focus on visualization methods and dashboard creation. Also, we applied a method based on time series to do prediction based on the traffic congestion data provided. A CRISP-DM approach was applied, integrating different data sources, using Python. Hence, understand traffic patterns, and help the city authorities in the decision-making process, namely more preparedness, adaptability, responsiveness to events.Os dados de transporte, no âmbito das cidades inteligentes, estão cada vez mais disponíveis. Estes dados permitem a construção de soluções inteligentes com impacto significativo na vida dos residentes e nos mecanismos das autoridades de gestão da cidade, os chamados Sistemas de Transporte Inteligentes. A nossa investigação incidiu sobre os dados de mobilidade urbana da cidade de Lisboa, disponibilizados pelo município. O principal objetivo da pesquisa foi abordar os problemas de mobilidade, interdependência e soluções de efeitos em cascata para a cidade de Lisboa. Para alcançar este objetivo foi desenvolvida uma metodologia baseada nos dados históricos do transito no centro urbano da cidade e principais acessos, com uma forte componente de visualização. Foi também aplicado um método baseado em series temporais para fazer a previsão das ocorrências de transito na cidade de Lisboa. Foi aplicada uma abordagem CRISP-DM, integrando diferentes fontes de dados, utilizando Python. Esta tese tem como objetivo identificar padrões de mobilidade urbana com análise e visualização de dados, de forma a auxiliar as autoridades municipais no processo de tomada de decisão, nomeadamente estar mais preparada, adaptada e responsiva

(So) Big Data and the transformation of the city

The exponential increase in the availability of large-scale mobility data has fueled the vision of smart cities that will transform our lives. The truth is that we have just scratched the surface of the research challenges that should be tackled in order to make this vision a reality. Consequently, there is an increasing interest among different research communities (ranging from civil engineering to computer science) and industrial stakeholders in building knowledge discovery pipelines over such data sources. At the same time, this widespread data availability also raises privacy issues that must be considered by both industrial and academic stakeholders. In this paper, we provide a wide perspective on the role that big data have in reshaping cities. The paper covers the main aspects of urban data analytics, focusing on privacy issues, algorithms, applications and services, and georeferenced data from social media. In discussing these aspects, we leverage, as concrete examples and case studies of urban data science tools, the results obtained in the “City of Citizens” thematic area of the Horizon 2020 SoBigData initiative, which includes a virtual research environment with mobility datasets and urban analytics methods developed by several institutions around Europe. We conclude the paper outlining the main research challenges that urban data science has yet to address in order to help make the smart city vision a reality