Efficient Human Activity Recognition in Large Image and Video Databases

Vision-based human action recognition has attracted considerable interest in recent research for its applications to video surveillance, content-based search, healthcare, and interactive games. Most existing research deals with building informative feature descriptors, designing efficient and robust algorithms, proposing versatile and challenging datasets, and fusing multiple modalities. Often, these approaches build on certain conventions such as the use of motion cues to determine video descriptors, application of off-the-shelf classifiers, and single-factor classification of videos. In this thesis, we deal with important but overlooked issues such as efficiency, simplicity, and scalability of human activity recognition in different application scenarios: controlled video environment (e.g.~indoor surveillance), unconstrained videos (e.g.~YouTube), depth or skeletal data (e.g.~captured by Kinect), and person images (e.g.~Flicker). In particular, we are interested in answering questions like (a) is it possible to efficiently recognize human actions in controlled videos without temporal cues? (b) given that the large-scale unconstrained video data are often of high dimension low sample size (HDLSS) nature, how to efficiently recognize human actions in such data? (c) considering the rich 3D motion information available from depth or motion capture sensors, is it possible to recognize both the actions and the actors using only the motion dynamics of underlying activities? and (d) can motion information from monocular videos be used for automatically determining saliency regions for recognizing actions in still images

A Unified Recommendation Framework for Data-driven, People-centric Smart Home Applications

With the rapid growth in the number of things that can be connected to the internet, Recommendation Systems for the IoT (RSIoT) have become more significant in helping a variety of applications to meet user preferences, and such applications can be smart home, smart tourism, smart parking, m-health and so on. In this thesis, we propose a unified recommendation framework for data-driven, people-centric smart home applications. The framework involves three main stages: complex activity detection, constructing recommendations in timely manner, and insuring the data integrity. First, we review the latest state-of-the-art recommendations methods and development of applications for recommender system in the IoT so, as to form an overview of the current research progress. Challenges of using IoT for recommendation systems are introduced and explained. A reference framework to compare the existing studies and guide future research and practices is provided. In order to meet the requirements of complex activity detection that helps our system to understand what activity or activities our user is undertaking in relatively high level. We provide adequate resources to be fit for the recommender system. Furthermore, we consider two inherent challenges of RSIoT, that is, capturing dynamicity patterns of human activities and system update without a focus on user feedback. Based on these, we design a Reminder Care System (RCS) which harnesses the advantages of deep reinforcement learning (DQN) to further address these challenges. Then we utilize a contextual bandit approach for improving the quality of recommendations by considering the context as an input. We aim to address not only the two previous challenges of RSIoT but also to learn the best action in different scenarios and treat each state independently. Last but not least, we utilize a blockchain technology to ensure the safety of data storage in addition to decentralized feature. In the last part, we discuss a few open issues and provide some insights for future directions

Using spontaneously generated online patient experiences to improve healthcare : A case study using Modafinil

Background Acknowledged issues with the RCT focus of EBM and recognition of the value of patient input have created a need for new methods of knowledge generation that can give the depth of qualitative studies but on a much larger scale. Almost half of the global population uses social media regularly, with increasing numbers of people using online spaces as either a first- or second-line health information and exchange resource. Estimates suggest the volume of online health related data grew by 300% between 2017 and 2020. As a data source, this unstructured freeform textual data is a form of patient generated health data, containing a mass of patient centred, contextually grounded detail about the perceptions and health concerns of those who post online. Methods for analysing it are at an early stage of development, but it is seen as having potential to add to clinical understanding, either by augmenting existing knowledge, or in aiding understanding of real-world usage of healthcare interventions and services. Objectives To explore how large-scale analysis of SGOPE can help with understanding patient perspectives of their conditions, symptoms, and self-management behaviours, assess the effectiveness of interventions, contribute to the process of knowledge and evidence creation, and consequently help healthcare systems improve outcomes in the most efficient manner. A secondary aim is to contribute to the development of methods that can be generalised across other interventions or services. Methods Using Modafinil as a case study, a multistage approach was taken. First, an exploratory study, comparing both qualitative and basic NLP techniques was undertaken on a small sample of 260 posts to identify topics, evaluate effectiveness and identify perceived causal text. An umbrella scoping review was then undertaken exploring how and for what purposes SGOPE data is currently being used within healthcare research. Findings from both then guided the main study, which used a variety of unsupervised NLP tools to explore the main dataset of over 69k posts. Individual methods were compared against each other. Results from both studies were compared and for evaluation. Results In contrast to the existing inconclusive systematic review evidence for Modafinil for anything other than narcolepsy, both studies found that Modafinil is seen as by posters as effective in treating fatigue and cognition symptoms in a wide range of conditions. Both identified the topics mentioned in the data, although more work needs to be done to develop the NLP methods to achieve a greater depth of understanding. The first study identified eight themes within the posts: reason for taking, impact of symptoms, acquisition, dosage, side-effects, comparison with other interventions, effectiveness, and quality of life outcomes. Effectiveness of Modafinil was found to be 68% positive, 12% mixed and 18% negative. Expressions of causal belief were identified. In the main study, effectiveness was measured with sentiment analysis, with all methods showing strong positive sentiment. Topic modelling identified groups of themes. Linguistic techniques extracted phrases indicating causality. Various analysis methods were compared to develop a method that could be generalised across other health topics

Unlocking the Pragmatics of Emoji: Evaluation of the Integration of Pragmatic Markers for Sarcasm Detection

Emojis have become an integral element of online communications, serving as a powerful, under-utilised resource for enhancing pragmatic understanding in NLP. Previous works have highlighted their potential for improvement of more complex tasks such as the identification of figurative literary devices including sarcasm due to their role in conveying tone within text. However present state-of-the-art does not include the consideration of emoji or adequately address sarcastic markers such as sentiment incongruence. This work aims to integrate these concepts to generate more robust solutions for sarcasm detection leveraging enhanced pragmatic features from both emoji and text tokens. This was achieved by establishing methodologies for sentiment feature extraction from emojis and a depth statistical evaluation of the features which characterise sarcastic text on Twitter. Current convention for generation of training data which implements weak-labelling using hashtags or keywords was evaluated against a human-annotated baseline; postulated validity concerns were verified where statistical evaluation found the content features deviated significantly from the baseline, highlighting potential validity concerns for many prominent works on the topic to date. Organic labelled sarcastic tweets containing emojis were crowd sourced by means of a survey to ensure valid outcomes for the sarcasm detection model. Given an established importance of both semantic and sentiment information, a novel sentiment-aware attention mechanism was constructed to enhance pattern recognition, balancing core features of sarcastic text: sentiment incongruence and context. This work establishes a framework for emoji feature extraction; a key roadblock cited in literature for their use in NLP tasks. The proposed sarcasm detection pipeline successfully facilitates the task using a GRU neural network with sentiment-aware attention, at an accuracy of 73% and promising indications regarding model robustness as part of a framework which is easily scalable for the inclusion of any future emojis released. Both enhanced sentiment information to supplement context in addition to consideration of the emoji were found to improve outcomes for the task