Representing semantic relatedness

University of Technology Sydney. Faculty of Engineering and Information Technology.To do text mining, the first question we must address is how to represent documents. The way a document is organised reflects certain explicit and implicit semantic and syntactical coupling relationships which are embedded in its contents. The effective capturing of such content couplings is thereby crucial for a genuine understanding of text representations. It has also led to the recent interest in document similarity analysis, including semantic relatedness, content coverage, word networking, and term-term couplings. Document similarity analysis has become increasingly relevant since roughly 80% of big data is unstructured. Accordingly, semantic relatedness has generated much interest owing to its ability to extract coupling relationships between terms (words or phrases). Existing work has focused more on explicit couplings and this is reflected in the models that have been built. In order to address the research limitations and challenges associated with document similarity analysis, this thesis proposes a semantic coupling similarity measure and the hierarchical tree learning model to fully enrich the semantics within terms and documents, and represent documents based on the comprehensive couplings of term pairs. In contrast to previous work, the models proposed can deal with unstructured data and terms that are coupled for various reasons, thereby addressing natural language ambiguity problems. Chapter 3 explores the semantic couplings of pairwise terms by involving three types of coupling relationships: (1) intra-term pair couplings, reflecting the explicit relatedness within term pairs that is represented by the relation strength over probabilistic distribution of terms across document collection; (2) the inter-term pair couplings, capturing the implicit relatedness between term pairs by considering the relation strength of their interactions with other term pairs on all possible paths via a graph-based representation of term couplings; and finally, (3) semantic coupling similarity, which effectively combine the intra- and inter-term couplings. The corresponding term semantic similarity measures are then defined to capture such couplings for the purposes of analysing term and document similarity. This approach effectively addresses both synonymy (many words per sense) and polysemy (many senses per word) in a graphical representation, two areas that have up until now been overlooked by previous models. Chapter 4 constructs a hierarchical tree-like structure to extract highly correlated terms in a layerwise fashion and to prune weak correlations in order to maintain efficiency. In keeping with the hierarchical tree-like structure, a hierarchical tree learning method is proposed. The main contributions of our work lie in three areas: (1) the hierarchical tree-like structure featuring hierarchical feature extraction and correlation computation procedures whereby highly correlated terms are merged into sets, and these are associated with more complete semantic information; (2) each layer is a maximal weighted spanning tree to prune weak feature correlations; (3) the hierarchical treelike structure can be applied to both supervised and unsupervised learning approaches. In this thesis, the tree is associated with Tree Augmented Naive Bayes (TAN) as the Hierarchical Tree Augmented Naive Bayes (HTAN). All of these models can be applied in the text mining tasks, including document clustering and text classification. The performance of the semantic coupling similarity measure is compared with typical document representation models on various benchmark data sets in terms of document clustering and classification evaluation metrics. These models provide insightful knowledge to organise and retrieve documents

Identification of Online Users' Social Status via Mining User-Generated Data

With the burst of available online user-generated data, identifying online users’ social status via mining user-generated data can play a significant role in many commercial applications, research and policy-making in many domains. Social status refers to the position of a person in relation to others within a society, which is an abstract concept. The actual definition of social status is specific in terms of specific measure indicator. For example, opinion leadership measures individual social status in terms of influence and expertise in an online society, while socioeconomic status characterizes personal real-life social status based on social and economic factors. Compared with traditional survey method which is time-consuming, expensive and sometimes difficult, some efforts have been made to identify specific social status of users based on specific user-generated data using classic machine learning methods. However, in fact, regarding specific social status identification based on specific user-generated data, the specific case has several specific challenges. However, classic machine learning methods in existing works fail to address these challenges, which lead to low identification accuracy. Given the importance of improving identification accuracy, this thesis studies three specific cases on identification of online and offline social status. For each work, this thesis proposes novel effective identification method to address the specific challenges for improving accuracy. The first work aims at identifying users’ online social status in terms of topic-sensitive influence and knowledge authority in social community question answering sites, namely identifying topical opinion leaders who are both influential and expert. Social community question answering (SCQA) site, an innovative community question answering platform, not only offers traditional question answering (QA) services but also integrates an online social network where users can follow each other. Identifying topical opinion leaders in SCQA has become an important research area due to the significant role of topical opinion leaders. However, most previous related work either focus on using knowledge expertise to find experts for improving the quality of answers, or aim at measuring user influence to identify influential ones. In order to identify the true topical opinion leaders, we propose a topical opinion leader identification framework called QALeaderRank which takes account of both topic-sensitive influence and topical knowledge expertise. In the proposed framework, to measure the topic-sensitive influence of each user, we design a novel influence measure algorithm that exploits both the social and QA features of SCQA, taking into account social network structure, topical similarity and knowledge authority. In addition, we propose three topic-relevant metrics to infer the topical expertise of each user. The extensive experiments along with an online user study show that the proposed QALeaderRank achieves significant improvement compared with the state-of-the-art methods. Furthermore, we analyze the topic interest change behaviors of users over time and examine the predictability of user topic interest through experiments. The second work focuses on predicting individual socioeconomic status from mobile phone data. Socioeconomic Status (SES) is an important social and economic aspect widely concerned. Assessing individual SES can assist related organizations in making a variety of policy decisions. Traditional approach suffers from the extremely high cost in collecting large-scale SES-related survey data. With the ubiquity of smart phones, mobile phone data has become a novel data source for predicting individual SES with low cost. However, the task of predicting individual SES on mobile phone data also proposes some new challenges, including sparse individual records, scarce explicit relationships and limited labeled samples, unconcerned in prior work restricted to regional or household-oriented SES prediction. To address these issues, we propose a semi-supervised Hypergraph based Factor Graph Model (HyperFGM) for individual SES prediction. HyperFGM is able to efficiently capture the associations between SES and individual mobile phone records to handle the individual record sparsity. For the scarce explicit relationships, HyperFGM models implicit high-order relationships among users on the hypergraph structure. Besides, HyperFGM explores the limited labeled data and unlabeled data in a semi-supervised way. Experimental results show that HyperFGM greatly outperforms the baseline methods on individual SES prediction with using a set of anonymized real mobile phone data. The third work is to predict social media users’ socioeconomic status based on their social media content, which is useful for related organizations and companies in a range of applications, such as economic and social policy-making. Previous work leverage manually defined textual features and platform-based user level attributes from social media content and feed them into a machine learning based classifier for SES prediction. However, they ignore some important information of social media content, containing the order and the hierarchical structure of social media text as well as the relationships among user level attributes. To this end, we propose a novel coupled social media content representation model for individual SES prediction, which not only utilizes a hierarchical neural network to incorporate the order and the hierarchical structure of social media text but also employs a coupled attribute representation method to take into account intra-coupled and inter-coupled interaction relationships among user level attributes. The experimental results show that the proposed model significantly outperforms other stat-of-the-art models on a real dataset, which validate the efficiency and robustness of the proposed model

Coupled clustering ensemble by exploring data interdependence

© 2018 ACM. Clustering ensembles combine multiple partitions of data into a single clustering solution. It is an effective technique for improving the quality of clustering results. Current clustering ensemble algorithms are usually built on the pairwise agreements between clusterings that focus on the similarity via consensus functions, between data objects that induce similarity measures from partitions and re-cluster objects, and between clusters that collapse groups of clusters into meta-clusters. In most of those models, there is a strong assumption on IIDness (i.e., independent and identical distribution), which states that base clusterings perform independently of one another and all objects are also independent. In the real world, however, objects are generally likely related to each other through features that are either explicit or even implicit. There is also latent but definite relationship among intermediate base clusterings because they are derived from the same set of data. All these demand a further investigation of clustering ensembles that explores the interdependence characteristics of data. To solve this problem, a new coupled clustering ensemble (CCE) framework that works on the interdependence nature of objects and intermediate base clusterings is proposed in this article. The main idea is to model the coupling relationship between objects by aggregating the similarity of base clusterings, and the interactive relationship among objects by addressing their neighborhood domains. Once these interdependence relationships are discovered, they will act as critical supplements to clustering ensembles. We verified our proposed framework by using three types of consensus function: clustering-based, object-based, and cluster-based. Substantial experiments on multiple synthetic and real-life benchmark datasets indicate that CCE can effectively capture the implicit interdependence relationships among base clusterings and among objects with higher clustering accuracy, stability, and robustness compared to 14 state-of-the-art techniques, supported by statistical analysis. In addition, we show that the final clustering quality is dependent on the data characteristics (e.g., quality and consistency) of base clusterings in terms of sensitivity analysis. Finally, the applications in document clustering, as well as on the datasets with much larger size and dimensionality, further demonstrate the effectiveness, efficiency, and scalability of our proposed models

Towards Open Information Management in Health Care

The utilization of information technology as tool in health care is increasing. The main benefits stem from the fact that information in electronic form can be transferred to different locations rapidly and from the possibility to automate certain information management tasks. The current technological approach for this automation relies on structured, formally coded representation of information. We discuss the limitations of the current technological approach and present a viewpoint, grounded on previous research and the authors’ own experiences, on how to progress. We present that a bottleneck in the automation of the management of constantly evolving clinical information is caused by the fact that the current technological approach requires the formal coding of information to be static in nature. This inherently hinders the expandability of the information case space to be managed. We present a new paradigm entitled open information management targeting unlimited case spaces. We also present a conceptual example from clinical medicine demonstrating open information management principles and mechanisms

Critically Examining the Claimed Value of Convolutions over User-Item Embedding Maps for Recommender Systems

In recent years, algorithm research in the area of recommender systems has shifted from matrix factorization techniques and their latent factor models to neural approaches. However, given the proven power of latent factor models, some newer neural approaches incorporate them within more complex network architectures. One specific idea, recently put forward by several researchers, is to consider potential correlations between the latent factors, i.e., embeddings, by applying convolutions over the user-item interaction map. However, contrary to what is claimed in these articles, such interaction maps do not share the properties of images where Convolutional Neural Networks (CNNs) are particularly useful. In this work, we show through analytical considerations and empirical evaluations that the claimed gains reported in the literature cannot be attributed to the ability of CNNs to model embedding correlations, as argued in the original papers. Moreover, additional performance evaluations show that all of the examined recent CNN-based models are outperformed by existing non-neural machine learning techniques or traditional nearest-neighbor approaches. On a more general level, our work points to major methodological issues in recommender systems research.Comment: Source code available here: https://github.com/MaurizioFD/RecSys2019_DeepLearning_Evaluatio

Consciosusness in Cognitive Architectures. A Principled Analysis of RCS, Soar and ACT-R

This report analyses the aplicability of the principles of consciousness developed in the ASys project to three of the most relevant cognitive architectures. This is done in relation to their aplicability to build integrated control systems and studying their support for general mechanisms of real-time consciousness.\ud To analyse these architectures the ASys Framework is employed. This is a conceptual framework based on an extension for cognitive autonomous systems of the General Systems Theory (GST).\ud A general qualitative evaluation criteria for cognitive architectures is established based upon: a) requirements for a cognitive architecture, b) the theoretical framework based on the GST and c) core design principles for integrated cognitive conscious control systems

Visualizing Evaluative Language in Relation to Constructing Identity in English Editorials and Op-Eds

This thesis is concerned with the problem of managing complexity in Systemic Functional Linguistic (SFL) analyses of language, particularly at the discourse semantics level. To deal with this complexity, the thesis develops AppAnn, a suite of linguistic visualization techniques that are specifically designed to provide both synoptic and dynamic views on discourse semantic patterns in text and corpus. Moreover, AppAnn visualizations are illustrated in a series of explorations of identity in a corpus of editorials and op-eds about the bin Laden killing. The findings suggest that the intriguing intricacies of discourse semantic meanings can be successfully discerned and more readily understood through linguistic visualization. The findings also provide insightful implications for discourse analysis by contributing to our understanding of a number of underdeveloped concepts of SFL, including coupling, commitment, instantiation, affiliation and individuation

Non-IID recommender systems : a machine learning approach

University of Technology Sydney. Faculty of Engineering and Information Technology.A recommender system (RS) comprises the core software, tools, and techniques that effectively and efficiently cope with information overload as well as locate information that is genuinely required. As one of the most widely used artificial intelligence (AI) systems, RSs have been integrated into daily life over the past two decades. In recent decade, the machine learning approach has dominated AI research in almost all areas. Therefore, modeling advanced RSs using the machine learning approach forms the basic methodology of this thesis. Current RSs suffer from many problems, such as data sparsity and cold start, because they fail to consider the non-IIDness in data, which includes the heterogeneities and coupled relations within and between users and items, as well as their interactions. Thus, we propose non-IID recommender systems by modeling the non-IIDness in recommendation data with the machine learning approach. Specifically, we study non-IID RS modeling techniques from three perspectives: users, items, and interactions. This research not only promotes the design of new machine learning models and algorithms in theory, but also extensively influences the evolution of technology and society. To construct the non-IID RS from a user perspective, we jointly model two aspects: (1) the heterogeneities of users and (2) the coupling between users. Specifically, we study the non-IID user modeling in two representative RSs: (1) a group-based RS (GBRS) and (2) a social network-based RS (SNRS). First, we perform an in-depth analysis of existing GBRSs and demonstrate their deficiencies in modeling the heterogeneity and coupling between group members for making group decisions. A deep neural network is designed to learn a group preference representation, which jointly considers all members’ heterogeneous preferences. Second, we model an SNRS by modeling the influential contexts that embed the influence of relevant users and items, because a user’s selection is largely influenced by other users with social relationships. To construct the non-IID RS from an item perspective, we target two modeling aspects: (1) the heterogeneities of items and (2) the coupling between items. Specifically, we study the non-IID item modeling in two representative RSs: (1) a cross-domain RS (CDRS) and (2) a session-based RS (SBRS). First, existing CDRSs may fail to conduct cross-domain transfer because of domain heterogeneity; thus, we propose an irregular tensor factorization model, which can more effectively capture the coupling between heterogeneous domains with learning the domain factors for each domain. Second, we construct an effective and efficient personalized SBRS to more effectively capture the couplings between items by modeling intra- and inter-session contexts. To construct the non-IID RS from an interaction perspective, we target two modeling aspects: (1) the heterogeneities of interactions and (2) the coupling between interactions. Specifically, we study the non-IID interaction modeling in two representative RSs: (1) a multi-objective RS (MORS) and (2) an attraction-based RS (ABRS). First, we study an MORS to tackle the challenges of recommendation for users and items in the long tail. Subsequently, a coupled regularization model is proposed to jointly optimize two objectives: the credibility and specialty. Existing content-based RSs can recommend new content according to similarity; however, they are not capable of interpreting the attraction points in user-item interactions. Therefore, to construct an interpretable content-based RS, we propose attraction modeling to learn and track user attractiveness. In the last section, we summarize the contributions of our work and present the future directions that can improve and extend the non-IID RS