Active Learning for Text Classification

Text classification approaches are used extensively to solve real-world challenges. The success or failure of text classification systems hangs on the datasets used to train them, without a good dataset it is impossible to build a quality system. This thesis examines the applicability of active learning in text classification for the rapid and economical creation of labelled training data. Four main contributions are made in this thesis. First, we present two novel selection strategies to choose the most informative examples for manually labelling. One is an approach using an advanced aggregated confidence measurement instead of the direct output of classifiers to measure the confidence of the prediction and choose the examples with least confidence for querying. The other is a simple but effective exploration guided active learning selection strategy which uses only the notions of density and diversity, based on similarity, in its selection strategy. Second, we propose new methods of using deterministic clustering algorithms to help bootstrap the active learning process. We first illustrate the problems of using non-deterministic clustering for selecting initial training sets, showing how non-deterministic clustering methods can result in inconsistent behaviour in the active learning process. We then compare various deterministic clustering techniques and commonly used non-deterministic ones, and show that deterministic clustering algorithms are as good as non-deterministic clustering algorithms at selecting initial training examples for the active learning process. More importantly, we show that the use of deterministic approaches stabilises the active learning process. Our third direction is in the area of visualising the active learning process. We demonstrate the use of an existing visualisation technique in understanding active learning selection strategies to show that a better understanding of selection strategies can be achieved with the help of visualisation techniques. Finally, to evaluate the practicality and usefulness of active learning as a general dataset labelling methodology, it is desirable that actively labelled dataset can be reused more widely instead of being only limited to some particular classifier. We compare the reusability of popular active learning methods for text classification and identify the best classifiers to use in active learning for text classification. This thesis is concerned using active learning methods to label large unlabelled textual datasets. Our domain of interest is text classification, but most of the methods proposed are quite general and so are applicable to other domains having large collections of data with high dimensionality

Compactly supported basis functions as support vector kernels: Capturing feature interdependence in the embedding space

Ph.DDOCTOR OF PHILOSOPH

Exploiting general-purpose background knowledge for automated schema matching

The schema matching task is an integral part of the data integration process. It is usually the first step in integrating data. Schema matching is typically very complex and time-consuming. It is, therefore, to the largest part, carried out by humans. One reason for the low amount of automation is the fact that schemas are often defined with deep background knowledge that is not itself present within the schemas. Overcoming the problem of missing background knowledge is a core challenge in automating the data integration process. In this dissertation, the task of matching semantic models, so-called ontologies, with the help of external background knowledge is investigated in-depth in Part I. Throughout this thesis, the focus lies on large, general-purpose resources since domain-specific resources are rarely available for most domains. Besides new knowledge resources, this thesis also explores new strategies to exploit such resources. A technical base for the development and comparison of matching systems is presented in Part II. The framework introduced here allows for simple and modularized matcher development (with background knowledge sources) and for extensive evaluations of matching systems. One of the largest structured sources for general-purpose background knowledge are knowledge graphs which have grown significantly in size in recent years. However, exploiting such graphs is not trivial. In Part III, knowledge graph em- beddings are explored, analyzed, and compared. Multiple improvements to existing approaches are presented. In Part IV, numerous concrete matching systems which exploit general-purpose background knowledge are presented. Furthermore, exploitation strategies and resources are analyzed and compared. This dissertation closes with a perspective on real-world applications

Semantic Similarity of Spatial Scenes

The formalization of similarity in spatial information systems can unleash their functionality and contribute technology not only useful, but also desirable by broad groups of users. As a paradigm for information retrieval, similarity supersedes tedious querying techniques and unveils novel ways for user-system interaction by naturally supporting modalities such as speech and sketching. As a tool within the scope of a broader objective, it can facilitate such diverse tasks as data integration, landmark determination, and prediction making. This potential motivated the development of several similarity models within the geospatial and computer science communities. Despite the merit of these studies, their cognitive plausibility can be limited due to neglect of well-established psychological principles about properties and behaviors of similarity. Moreover, such approaches are typically guided by experience, intuition, and observation, thereby often relying on more narrow perspectives or restrictive assumptions that produce inflexible and incompatible measures. This thesis consolidates such fragmentary efforts and integrates them along with novel formalisms into a scalable, comprehensive, and cognitively-sensitive framework for similarity queries in spatial information systems. Three conceptually different similarity queries at the levels of attributes, objects, and scenes are distinguished. An analysis of the relationship between similarity and change provides a unifying basis for the approach and a theoretical foundation for measures satisfying important similarity properties such as asymmetry and context dependence. The classification of attributes into categories with common structural and cognitive characteristics drives the implementation of a small core of generic functions, able to perform any type of attribute value assessment. Appropriate techniques combine such atomic assessments to compute similarities at the object level and to handle more complex inquiries with multiple constraints. These techniques, along with a solid graph-theoretical methodology adapted to the particularities of the geospatial domain, provide the foundation for reasoning about scene similarity queries. Provisions are made so that all methods comply with major psychological findings about people’s perceptions of similarity. An experimental evaluation supplies the main result of this thesis, which separates psychological findings with a major impact on the results from those that can be safely incorporated into the framework through computationally simpler alternatives

Towards a Comprehensive Evidence-Based Approach For Information Security Value Assessment

This thesis is motivated by the goals of understanding in depth which information security value aspects are relevant in real-world business environments and contributing a value-prioritised information security investment decision model suitable for practitioners in the field. Pursuing this goal, we apply a mixed method research approach that combines the analysis of the relevant literature, expert interviews, practitioner survey data and structural equation modelling and multicriteria decision analysis. In the first step, we address the identified terminology gap to clarify the meaning of ‘cyber security’ by analysing authoritative definition sources in the literature and presenting an improved definition distinct from that of ‘information security’. We then investigate the influence of repeated information security breaches on an organisation’s stock market value to benchmark the wider economic impact of such events. We find abnormal returns following a breach event as well as weak statistical significance on abnormal returns for later breach events, confirming that data breaches have a negative impact on organisations. To understand how security practitioners view this topic, we conduct and analyse semi-structured interviews following a grounded theory approach. Our research identifies 15 principles aligned with a conceptual information security investment framework. The key components of this framework such as the business environment, drivers (threat landscape, legal and regulatory) and challenges (cost of security, uncertainty) are found to be a crucial part of value-prioritised information security investment decisions. We verify these findings through a structural model consisting of five latent variables representing key areas in value-focused information security investment decisions. The model shows that security capabilities have the largest direct effect on the value organisations gain from information security investment. In addition, the value outcome is strongly influenced by organisation-specific constructs such as the threat landscape and regulatory requirements, which must therefore be considered when creating security capabilities. By addressing one of the key uncertainty issues, we use a probabilistic topic modelling approach to identify latent security threat prediction topics from a large pool of security predictions publicised in the media. We further verify the prediction outcomes through a survey instrument. The results confirm the feasibility of forecasting notable threat developments in this context, implying that practitioners can use this approach to reduce uncertainty and improve security investment decisions. In the last part of the thesis, we present a multicriteria decision model that combines our results on value-prioritised information security investments in an organisational context. Based on predefined criteria and preferences and by utilising stochastic multicriteria acceptability analysis as the adopted methodology, our model can deal with substantial uncertainty while offering ease of use for practitioners