Unlocking social media and user generated content as a data source for knowledge management

The pervasiveness of Social Media and user-generated content has triggered an exponential increase in global data volumes. However, due to collection and extraction challenges, data in many feeds, embedded comments, reviews and testimonials are inaccessible as a generic data source. This paper incorporates Knowledge Management framework as a paradigm for knowledge management and data value extraction. This framework embodies solutions to unlock the potential of UGC as a rich, real-time data source for analytical applications. The contributions described in this paper are threefold. Firstly, a method for automatically navigating pagination systems to expose UGC for collection is presented. This is evaluated using browser emulation integrated with dynamic data collection. Secondly, a new method for collecting social data without any a priori knowledge of the sites is introduced. Finally, a new testbed is developed to reflect the current state of internet sites and shared publicly to encourage future research. The discussion benchmarks the new algorithm alongside existing data extraction techniques and provides evidence of the increased amount of UGC data made accessible by the new algorithm

Towards distributed real-time physiological processing in mobile environments

Physiological monitoring has been used in a wide range of scenarios to assist in disease diagnosis, athlete monitoring and other activities. There are also many opportunities in analysing aggregate data from groups of people rather than individuals such as public event monitoring or athletic team performance optimisation. Numerous difficulties exist pertaining to this, particularly concerning how to process and transform the resulting physiological data in real-time when many devices are producing data. This paper proposes a system that is designed to monitor, analyse and report physiological data in real-time by leveraging mobile devices as distributed processors

Towards distributed real-time physiological processing in mobile environments

Physiological monitoring is the practice of using sensors to read, store, process and interpret physiological data from organic beings, including biofeedback signals associated with heart, brain, muscle and other organ activity. Physiological data retrieved from the body can be used for disease diagnose and other activities, such as monitoring physical and mental stress levels of participants in physical training exercises. In addition to monitoring individuals, physiological data can be ag- gregated to monitor groups. However, this kind of group-monitoring can present difficulties in mobile environments, particularly concerning how to process and transform the raw physiological data in real-time. Current techniques involve the use of either fixed processing resources (such as workstations or servers) or the use of Cloud computing, which requires a stable, uninterrupted mobile broadband communications network - neither of which are common in remote mobile environments. This dissertation proposes to improve existing methods of physiological monitoring. This technique aims to monitor, analyse and report physiological data in real-time by leveraging mobile devices as distributed processors. The viability of this approach is evaluated by testing the implementation of a system based on these principles in a number of real-world physiological processing examples

A scalable framework for integrated social data mining

Social Networking Sites (SNS) are ubiquitous within modern society, forming communications networks that span across cultural and geographical boundaries. The information posted to these sites provide useful insights into individuals, but can also provide a wealth of information that can be used for further analysis into the surrounding environment. Three main challenges limit the use of this information in applications: the quantity of data is often unmanageable, there is a significant amount of data unavailable for use due to a lack of generic interfaces for access, and there is difficulty in integrating multiple disparate social data sources. The overall aim of the research described in this thesis is to advance the field of data science and improve accessibility of social data in analytical applications, in both academic and commercial settings. This aim has been addressed with three primary contributions; new algorithms to efficiently locate and collect relevant social data, new methods of performing unsupervised data extraction from generic social sites, and the development and subsequent empirical evaluation of a framework to facilitate the collection, integration, storage and presentation of social data for use in applications. The first contribution was the presentation of a search query optimisation algorithm designed to reduce the amount of noise resulting from social data collection by learning from collected content and iteratively building new query keyword sets. The algorithm was empirically evaluated and the results indicated that it provides significantly more data than existing search tools while minimising signal-to-noise ratio. The second contribution aimed to improve access to social data available on Web 2.0 sites but without any existing interface access to the data. The algorithm is designed to extract social data from sites without any a priori knowledge of design or page layout. Its efficacy was empirically evaluated against a testbed consisting of popular news and current affairs websites. Results indicated that the algorithm was very effective at unsupervised retrieval of social data. The third major contribution presented a framework that integrated the previous two contributions into a framework designed to streamline use of social data in academic and commercial applications. The generic, component-based design was evaluated in real-world scenarios and determined to provide a full social collection and analytics workflow in an extensible and scalable manner. This research has theoretical and practical implications for the use of social data in analytical research and commercial use. It extends the data extraction field to include user-generated content, while providing new avenues for performing semi-intelligent social data sourcing, and significantly improves the accessibility of social data

Towards distributed real-time physiological processing in mobile environments

Physiological monitoring has been used in a wide range of scenarios to assist in disease diagnosis, athlete monitoring and other activities. There are also many opportunities in analysing aggregate data from groups of people rather than individuals such as public event monitoring or athletic team performance optimisation. Numerous difficulties exist pertaining to this, particularly concerning how to process and transform the resulting physiological data in real-time when many devices are producing data. This paper proposes a system that is designed to monitor, analyse and report physiological data in real-time by leveraging mobile devices as distributed processors

Energy aware software evolution for wireless sensor networks

Wireless Sensor Networks (WSNs) are subject to high levels of dynamism arising from changing environmental conditions and application requirements. Reconfiguration allows software functionality to be optimized for current environmental conditions and supports software evolution to meet variable application requirements. Contemporary software modularization approaches for WSNs allow for software evolution at various granularities; from monolithic re-flashing of OS and application functionality, through replacement of complete applications, to the reconfiguration of individual software components. As the nodes that compose a WSN must typically operate for long periods on a single battery charge, estimating the energy cost of software evolution is critical. This paper contributes a generic model for calculating the energy cost of the reconfiguration in WSN. We have embedded this model in the LooCI middleware, resulting in the first energy aware reconfigurable component model for sensor networks. We evaluate our approach using two real-world WSN applications and find that (i.) our model accurately predicts the energy cost of reconfiguration and (ii.) component-based reconfiguration has a high initial cost, but provides energy savings during software evolution.status: publishe