Crowdsourced Data Mining for Urban Activity: A Review of Data Sources, Applications and Methods

The penetration of devices integrated with location-based services and internet services has generated massive data about the everyday life of citizens and tracked their activities happening in cities. Crowdsourced data, such as social media data, POIs data and collaborative websites, generated by the crowd, has become fine-grained proxy data of urban activity and widely used in research in urban studies. However, due to the heterogeneity of data types of crowdsourced data and the limitation of previous studies mainly focusing on a specific application, a systematic review of crowdsourced data mining for urban activity is still lacking. In order to fill the gap, this paper conducts a literature search in the Web of Science database, selecting 226 highly related papers published between 2013 and 2019. Based on those papers, the review firstly conducts a bibliometric analysis identifying underpinning domains, pivot scholars and papers around this topic. The review also synthesises previous research into three parts: main applications of different data sources and data fusion; application of spatial analysis in mobility patterns, functional areas and event detection; application of socio-demographic and perception analysis in city attractiveness, demographic characteristics and sentiment analysis. The challenges of this type of data are also discussed in the end. This study provides a systematic and current review for both researchers and practitioners interested in the applications of crowdsourced data mining for urban activity.This research is funded by a scholarship from the China Scholarship Counci

Staging urban emergence through collective creativity: Devising an outdoor mobile augmented reality tool

The unpredictability of global geopolitical conflicts, economic trends, and impacts of climate change, coupled with an increasing urban population, necessitates a more profound commitment to resilience thinking in urban planning and design. In contrast to top-down planning and designing for sustainability, allowing for emergence to take place seems to contribute to a capacity to better deal with this complex unpredictability, by allowing incremental changes through bottom-up, self-organized adaptation made by diverse actors in the proximity of various social, economical and functional entities in the urban context.The present thesis looks into the processes of creating urban emergence from both theoretical and practical perspectives. The theoretical section of the thesis first looks into the relationship between the processes and the qualities of a compact city. The Japanese city of Tokyo is used as an example of a resilient compact city that continuously emerges through incremental micro-adaptations by individual actors guided by urban rules that ‘let it happen’ without much central control or top-down design of the individual outcomes. The thesis then connects such rule-based emergent processes and the qualities of a compact city to complex adaptive system’s (CAS) theory, emphasizing the value of incremental and individual multiple-stakeholder input. The latter part of the thesis focuses on how to create a platform that can combine the bottom-up, emergent, rule-based planning approaches, and collective creativity based on individual participation and input from the public. This section is dedicated to developing a tool for a collaborative urban design using outdoor mobile augmented reality (MAR) by research-through-design method.The thesis thus has three parts addressing the topics: 1. urban planning processes and resulting urban qualities concerning compact city – i.e., density and diversity; 2. the processes of urban emergence, which generates complexity that renders urban resilience from the urban planning theory perspective; 3. developing a tool for non-expert citizens and other stakeholders to design and visualize an urban neighborhood by simulating the rule-based urban emergence using outdoor MAR. The results include a proposal for a complementary hybrid planning approaches that might approximate the CAS in urban systems with qualities that contribute to urban resiliency. Thereafter, the results describe specifications and design criteria for a tool as a public collaborative design platform using outdoor MAR to promote public participation: Urban CoBuilder. The processes of developing and prototyping such a tool to test various urban concepts concerning identified adaptive urban planning approaches are also presented with an assessment of the MAR tool based on focus group user tests. Future studies need to better include the potential of crowdsourcing public creativity through mass participation using the collaborative design tool and actual integration of these participatory design results in urban policies

Combating User Misbehavior on Social Media

Social media encourages user participation and facilitates user’s self-expression like never before. While enriching user behavior in a spectrum of means, many social media platforms have become breeding grounds for user misbehavior. In this dissertation we focus on understanding and combating three specific threads of user misbehaviors that widely exist on social media — spamming, manipulation, and distortion. First, we address the challenge of detecting spam links. Rather than rely on traditional blacklist-based or content-based methods, we examine the behavioral factors of both who is posting the link and who is clicking on the link. The core intuition is that these behavioral signals may be more difficult to manipulate than traditional signals. We find that this purely behavioral approach can achieve good performance for robust behavior-based spam link detection. Next, we deal with uncovering manipulated behavior of link sharing. We propose a four-phase approach to model, identify, characterize, and classify organic and organized groups who engage in link sharing. The key motivating insight is that group-level behavioral signals can distinguish manipulated user groups. We find that levels of organized behavior vary by link type and that the proposed approach achieves good performance measured by commonly-used metrics. Finally, we investigate a particular distortion behavior: making bullshit (BS) statements on social media. We explore the factors impacting the perception of BS and what leads users to ultimately perceive and call a post BS. We begin by preparing a crowdsourced collection of real social media posts that have been called BS. We then build a classification model that can determine what posts are more likely to be called BS. Our experiments suggest our classifier has the potential of leveraging linguistic cues for detecting social media posts that are likely to be called BS. We complement these three studies with a cross-cutting investigation of learning user topical profiles, which can shed light into what subjects each user is associated with, which can benefit the understanding of the connection between user and misbehavior. Concretely, we propose a unified model for learning user topical profiles that simultaneously considers multiple footprints and we show how these footprints can be embedded in a generalized optimization framework. Through extensive experiments on millions of real social media posts, we find our proposed models can effectively combat user misbehavior on social media

Digital neighborhoods

With the advent of ‘big data’ there is an increased interest in using social media to describe city dynamics. This paper employs geo-located social media data to identify ‘digital neighborhoods’ – those areas in the city where social media is used more often. Starting with geo-located Twitter and Foursquare data for the New York City region in 2014, we applied spatial clustering techniques to detect significant groupings or ‘neighborhoods’ where social media use is high or low. The results show that beyond the business districts, digital neighborhoods occur in communities undergoing shifting socio-demographics. Neighborhoods that are not digitally oriented tend to have higher proportion of minorities and lower incomes, highlighting a social–economic divide in how social media is used in the city. Understanding the differences in these neighborhoods can help city planners interested in generating economic development proposals, civic engagement strategies, and urban design ideas that target these areas

A study on Analysis and Utilization of Crowd-sourced Spatio-temporal Contexts from Social Media

兵庫県立大学大学院201

Cell Towers as Urban Sensors: Understanding the Strengths and Limitations of Mobile Phone Location Data

Understanding urban dynamics and human mobility patterns not only benefits a wide range of real-world applications (e.g., business site selection, public transit planning), but also helps address many urgent issues caused by the rapid urbanization processes (e.g., population explosion, congestion, pollution). In the past few years, given the pervasive usage of mobile devices, call detail records collected by mobile network operators has been widely used in urban dynamics and human mobility studies. However, the derived knowledge might be strongly biased due to the uneven distribution of people’s phone communication activities in space and time. This dissertation research applies different analytical methods to better understand human activity and urban environment, as well as their interactions, mainly based on a new type of data source: actively tracked mobile phone location data. In particular, this dissertation research achieves three main research objectives. First, this research develops visualization and analysis approaches to uncover hidden urban dynamics patterns from actively tracked mobile phone location data. Second, this research designs quantitative methods to evaluate the representativeness issue of call detail record data. Third, this research develops an appropriate approach to evaluate the performance of different types of tracking data in urban dynamics research. The major contributions of this dissertation research include: 1) uncovering the dynamics of stay/move activities and distance decay effects, and the changing human mobility patterns based on several mobility indicators derived from actively tracked mobile phone location data; 2) taking the first step to evaluate the representativeness and effectiveness of call detail record and revealing its bias in human mobility research; and 3) extracting and comparing urban-level population movement patterns derived from three different types of tracking data as well as their pros and cons in urban population movement analysis

Mining large-scale human mobility data for long-term crime prediction

Traditional crime prediction models based on census data are limited, as they fail to capture the complexity and dynamics of human activity. With the rise of ubiquitous computing, there is the opportunity to improve such models with data that make for better proxies of human presence in cities. In this paper, we leverage large human mobility data to craft an extensive set of features for crime prediction, as informed by theories in criminology and urban studies. We employ averaging and boosting ensemble techniques from machine learning, to investigate their power in predicting yearly counts for different types of crimes occurring in New York City at census tract level. Our study shows that spatial and spatio-temporal features derived from Foursquare venues and checkins, subway rides, and taxi rides, improve the baseline models relying on census and POI data. The proposed models achieve absolute R^2 metrics of up to 65% (on a geographical out-of-sample test set) and up to 89% (on a temporal out-of-sample test set). This proves that, next to the residential population of an area, the ambient population there is strongly predictive of the area's crime levels. We deep-dive into the main crime categories, and find that the predictive gain of the human dynamics features varies across crime types: such features bring the biggest boost in case of grand larcenies, whereas assaults are already well predicted by the census features. Furthermore, we identify and discuss top predictive features for the main crime categories. These results offer valuable insights for those responsible for urban policy or law enforcement