Crime prediction and monitoring in Porto, Portugal, using machine learning, spatial and text analytics

Crimes are a common societal concern impacting quality of life and economic growth. Despite the global decrease in crime statistics, specific types of crime and feelings of insecurity, have often increased, leading safety and security agencies with the need to apply novel approaches and advanced systems to better predict and prevent occurrences. The use of geospatial technologies, combined with data mining and machine learning techniques allows for significant advances in the criminology of place. In this study, official police data from Porto, in Portugal, between 2016 and 2018, was georeferenced and treated using spatial analysis methods, which allowed the identification of spatial patterns and relevant hotspots. Then, machine learning processes were applied for space-time pattern mining. Using lasso regression analysis, significance for crime variables were found, with random forest and decision tree supporting the important variable selection. Lastly, tweets related to insecurity were collected and topic modeling and sentiment analysis was performed. Together, these methods assist interpretation of patterns, prediction and ultimately, performance of both police and planning professionals

Macro-micro approach for mining public sociopolitical opinion from social media

During the past decade, we have witnessed the emergence of social media, which has prominence as a means for the general public to exchange opinions towards a broad range of topics. Furthermore, its social and temporal dimensions make it a rich resource for policy makers and organisations to understand public opinion. In this thesis, we present our research in understanding public opinion on Twitter along three dimensions: sentiment, topics and summary. In the first line of our work, we study how to classify public sentiment on Twitter. We focus on the task of multi-target-specific sentiment recognition on Twitter, and propose an approach which utilises the syntactic information from parse-tree in conjunction with the left-right context of the target. We show the state-of-the-art performance on two datasets including a multi-target Twitter corpus on UK elections which we make public available for the research community. Additionally we also conduct two preliminary studies including cross-domain emotion classification on discourse around arts and cultural experiences, and social spam detection to improve the signal-to-noise ratio of our sentiment corpus. Our second line of work focuses on automatic topical clustering of tweets. Our aim is to group tweets into a number of clusters, with each cluster representing a meaningful topic, story, event or a reason behind a particular choice of sentiment. We explore various ways of tackling this challenge and propose a two-stage hierarchical topic modelling system that is efficient and effective in achieving our goal. Lastly, for our third line of work, we study the task of summarising tweets on common topics, with the goal to provide informative summaries for real-world events/stories or explanation underlying the sentiment expressed towards an issue/entity. As most existing tweet summarisation approaches rely on extractive methods, we propose to apply state-of-the-art neural abstractive summarisation model for tweets. We also tackle the challenge of cross-medium supervised summarisation with no target-medium training resources. To the best of our knowledge, there is no existing work on studying neural abstractive summarisation on tweets. In addition, we present a system for providing interactive visualisation of topic-entity sentiments and the corresponding summaries in chronological order. Throughout our work presented in this thesis, we conduct experiments to evaluate and verify the effectiveness of our proposed models, comparing to relevant baseline methods. Most of our evaluations are quantitative, however, we do perform qualitative analyses where it is appropriate. This thesis provides insights and findings that can be used for better understanding public opinion in social media

Analyzing fluctuation of topics and public sentiment through social media data

Over the past decade years, Internet users were expending rapidly in the world. They form various online social networks through such Internet platforms as Twitter, Facebook and Instagram. These platforms provide a fast way that helps their users receive and disseminate information and express personal opinions in virtual space. When dealing with massive and chaotic social media data, how to accurately determine what events or concepts users are discussing is an interesting and important problem. This dissertation work mainly consists of two parts. First, this research pays attention to mining the hidden topics and user interest trend by analyzing real-world social media activities. Topic modeling and sentiment analysis methods are proposed to classify the social media posts into different sentiment classes and then discover the trend of sentiment based on different topics over time. The presented case study focuses on COVID-19 pandemic that started in 2019. A large amount of Twitter data is collected and used to discover the vaccine-related topics during the pre- and post-vaccine emergency use period. By using the proposed framework, 11 vaccine-related trend topics are discovered. Ultimately the discovered topics can be used to improve the readability of confusing messages about vaccines on social media and provide effective results to support policymakers in making their policy their informed decisions about public health. Second, using conventional topic models cannot deal with the sparsity problem of short text. A novel topic model, named Topic Noise based-Biterm Topic Model with FastText embeddings (TN-BTMF), is proposed to deal with this problem. Word co-occurrence patterns (i.e. biterms) are dirctly generated in BTM. A scoring method based on word co-occurrence and semantic similarity is proposed to detect noise biterms. In th

Public perceptions of digital fashion: An analysis of sentiment and Latent Dirichlet Allocation topic modeling

Since digital technology has had a significant impact on the fashion industry, digital fashion has become a hot topic in today’s society. Currently, research on digital fashion is focused on the transformation of enterprise marketing strategies and the discussion of digital technology. Despite this, the current study does not include an analysis of the audience’s emotional and cognitive responses to digital fashion on social networking platforms. A comprehensive analysis and discussion of 52,891 posts about digital fashion and virtual fashion published on social networking sites was conducted using k-means clustering analysis, Latent Dirichlet Allocation (LDA) topic modeling, and sentiment analysis in this study. The study examines the public’s perception and hot topics about digital fashion, as well as the industry’s development situation and trends. According to the findings, both positive and neutral emotions accompany the public’s attitude toward digital fashion. There is a wide range of topics covered in the discussion. Innovations in digital technology have impacted the creation of jobs, talent demand, marketing strategies, profit forms, and industrial chain innovation of fashion-related businesses. Researchers in related fields will find this study useful not only as a reference for research methods and directions, but also as a source of references for research methodology. A case study and data reference will also be provided to industry practitioners

A Tutorial on Event Detection using Social Media Data Analysis: Applications, Challenges, and Open Problems

In recent years, social media has become one of the most popular platforms for communication. These platforms allow users to report real-world incidents that might swiftly and widely circulate throughout the whole social network. A social event is a real-world incident that is documented on social media. Social gatherings could contain vital documentation of crisis scenarios. Monitoring and analyzing this rich content can produce information that is extraordinarily valuable and help people and organizations learn how to take action. In this paper, a survey on the potential benefits and applications of event detection with social media data analysis will be presented. Moreover, the critical challenges and the fundamental tradeoffs in event detection will be methodically investigated by monitoring social media stream. Then, fundamental open questions and possible research directions will be introduced

On the Combination of Game-Theoretic Learning and Multi Model Adaptive Filters

This paper casts coordination of a team of robots within the framework of game theoretic learning algorithms. In particular a novel variant of fictitious play is proposed, by considering multi-model adaptive filters as a method to estimate other players’ strategies. The proposed algorithm can be used as a coordination mechanism between players when they should take decisions under uncertainty. Each player chooses an action after taking into account the actions of the other players and also the uncertainty. Uncertainty can occur either in terms of noisy observations or various types of other players. In addition, in contrast to other game-theoretic and heuristic algorithms for distributed optimisation, it is not necessary to find the optimal parameters a priori. Various parameter values can be used initially as inputs to different models. Therefore, the resulting decisions will be aggregate results of all the parameter values. Simulations are used to test the performance of the proposed methodology against other game-theoretic learning algorithms.</p

Assessing the social impacts of extreme weather events using social media

The frequency and severity of extreme weather events such as flooding, hurricanes/storms and heatwaves are increasing as a result of climate change. There is a need for information to better understand when, where and how these events are impacting people. However, there are currently limited sources of impact information beyond traditional meteorological observations. Social sensing, which is the use of unsolicited social media data to better understand real world events, is one method that may provide such information. Social sensing has successfully been used to detect earthquakes, floods, hurricanes, wildfires, heatwaves and other weather hazards. Here social sensing methods are adapted to explore potential for collecting impact information for meteorologists and decision makers concerned with extreme weather events. After a review of the literature, three experimental studies are presented. Social sensing is shown to be effective for detection of impacts of named storms in the UK and Ireland. Topics of discussion and sentiment are explored in the period before, during and after a storm event. Social sensing is also shown able to detect high-impact rainfall events worldwide, validating results against a manually curated database. Additional events which were not known to this database were found by social sensing. Finally, social sensing was applied to heatwaves in three European cities. Building on previous work on heatwaves in the UK, USA and Australia, the methods were extended to include impact phrases alongside hazard-related phrases, in three different languages (English, Dutch and Greek). Overall, social sensing is found to be a good source of impact information for organisations that need to better understand the impacts of extreme weather. The research described in this project has been commercialised for operational use by meteorological agencies in the UK, including the Met Office, Environment Agency and Natural Resources Wales.Engineering and Physical Sciences Research Council (EPSRC