40 research outputs found
Cancer Subtyping Detection using Biomarker Discovery in Multi-Omics Tensor Datasets
This thesis begins with a thorough review of research trends from 2015 to 2022, examining the challenges and issues related to biomarker discovery in multi-omics datasets. The review covers areas of application, proposed methodologies, evaluation criteria used to assess performance, as well as limitations and drawbacks that require further investigation and improvement. This comprehensive overview serves to provide a deeper understanding of the current state of research in this field and the opportunities for future research. It will be particularly useful for those who are interested in this area of study and seeking to expand their knowledge.
In the second part of this thesis, a novel methodology is proposed for the identification of significant biomarkers in a multi-omics colon cancer dataset. The integration of clinical features with biomarker discovery has the potential to facilitate the early identification of mortality risk and the development of personalized therapies for a range of diseases, including cancer and stroke. Recent advancements in รขโฌลomicsรขโฌ๏ฟฝ technologies have opened up new avenues for researchers to identify disease biomarkers through system-level analysis. Machine learning methods, particularly those based on tensor decomposition techniques, have gained popularity due to the challenges associated with integrative analysis of multi-omics data owing to the complexity of biological systems. Despite extensive efforts towards discovering disease-associated biomolecules by analyzing data from various รขโฌลomicsรขโฌ๏ฟฝ experiments, such as genomics, transcriptomics, and metabolomics, the poor integration of diverse forms of 'omics' data has made the integrative analysis of multi-omics data a daunting task.
Our research includes ANOVA simultaneous component analysis (ASCA) and Tucker3 modeling to analyze a multivariate dataset with an underlying experimental design. By comparing the spaces spanned by different model components we showed how the two methods can be used for confirmatory analysis and provide complementary information. we demonstrated the novel use of ASCA to analyze the residuals of Tucker3 models to find the optimum one. Increasing the model complexity to more factors removed the last remaining ASCA detectable structure in the residuals. Bootstrap analysis of the core matrix values of the Tucker3 models used to check that additional triads of eigenvectors were needed to describe the remaining structure in the residuals. Also, we developed a new simple, novel strategy for aligning Tucker3 bootstrap models with the Tucker3 model of the original data so that eigenvectors of the three modes, the order of the values in the core matrix, and their algebraic signs match the original Tucker3 model without the need for complicated bookkeeping strategies or performing rotational transformations. Additionally, to avoid getting an overparameterized Tucker3 model, we used the bootstrap method to determine 95% confidence intervals of the loadings and core values. Also, important variables for classification were identified by inspection of loading confidence intervals. The experimental results obtained using the colon cancer dataset demonstrate that our proposed methodology is effective in improving the performance of biomarker discovery in a multi-omics cancer dataset. Overall, our study highlights the potential of integrating multi-omics data with machine learning methods to gain deeper insights into the complex biological mechanisms underlying cancer and other diseases. The experimental results using NIH colon cancer dataset demonstrate that the successful application of our proposed methodology in cancer subtype classification provides a foundation for further investigation into its utility in other disease areas
Graph Mining for Cybersecurity: A Survey
The explosive growth of cyber attacks nowadays, such as malware, spam, and
intrusions, caused severe consequences on society. Securing cyberspace has
become an utmost concern for organizations and governments. Traditional Machine
Learning (ML) based methods are extensively used in detecting cyber threats,
but they hardly model the correlations between real-world cyber entities. In
recent years, with the proliferation of graph mining techniques, many
researchers investigated these techniques for capturing correlations between
cyber entities and achieving high performance. It is imperative to summarize
existing graph-based cybersecurity solutions to provide a guide for future
studies. Therefore, as a key contribution of this paper, we provide a
comprehensive review of graph mining for cybersecurity, including an overview
of cybersecurity tasks, the typical graph mining techniques, and the general
process of applying them to cybersecurity, as well as various solutions for
different cybersecurity tasks. For each task, we probe into relevant methods
and highlight the graph types, graph approaches, and task levels in their
modeling. Furthermore, we collect open datasets and toolkits for graph-based
cybersecurity. Finally, we outlook the potential directions of this field for
future research
Self-Supervised Learning for Recommender Systems: A Survey
In recent years, neural architecture-based recommender systems have achieved
tremendous success, but they still fall short of expectation when dealing with
highly sparse data. Self-supervised learning (SSL), as an emerging technique
for learning from unlabeled data, has attracted considerable attention as a
potential solution to this issue. This survey paper presents a systematic and
timely review of research efforts on self-supervised recommendation (SSR).
Specifically, we propose an exclusive definition of SSR, on top of which we
develop a comprehensive taxonomy to divide existing SSR methods into four
categories: contrastive, generative, predictive, and hybrid. For each category,
we elucidate its concept and formulation, the involved methods, as well as its
pros and cons. Furthermore, to facilitate empirical comparison, we release an
open-source library SELFRec (https://github.com/Coder-Yu/SELFRec), which
incorporates a wide range of SSR models and benchmark datasets. Through
rigorous experiments using this library, we derive and report some significant
findings regarding the selection of self-supervised signals for enhancing
recommendation. Finally, we shed light on the limitations in the current
research and outline the future research directions.Comment: 20 pages. Accepted by TKD
Learning Logical Rules from Knowledge Graphs
Ph.D. (Integrated) ThesisExpressing and extracting regularities in multi-relational data, where data points are interrelated
and heterogeneous, requires well-designed knowledge representation. Knowledge Graphs (KGs),
as a graph-based representation of multi-relational data, have seen a rapidly growing presence in
industry and academia, where many real-world applications and academic research are either
enabled or augmented through the incorporation of KGs. However, due to the way KGs are
constructed, they are inherently noisy and incomplete. In this thesis, we focus on developing
logic-based graph reasoning systems that utilize logical rules to infer missing facts for the
completion of KGs. Unlike most rule learners that primarily mine abstract rules that contain
no constants, we are particularly interested in learning instantiated rules that contain constants
due to their ability to represent meaningful patterns and correlations that can not be expressed
by abstract rules. The inclusion of instantiated rules often leads to exponential growth in the
search space. Therefore, it is necessary to develop optimization strategies to balance between
scalability and expressivity. To such an end, we propose GPFL, a probabilistic rule learning
system optimized to mine instantiated rules through the implementation of a novel two-stage
rule generation mechanism. Through experiments, we demonstrate that GPFL not only performs
competitively on knowledge graph completion but is also much more efficient then existing
methods at mining instantiated rules. With GPFL, we also reveal overfitting instantiated rules
and provide detailed analyses about their impact on system performance. Then, we propose RHF,
a generic framework for constructing rule hierarchies from a given set of rules. We demonstrate
through experiments that with RHF and the hierarchical pruning techniques enabled by it,
significant reductions in runtime and rule size are observed due to the pruning of unpromising
rules. Eventually, to test the practicability of rule learning systems, we develop Ranta, a novel
drug repurposing system that relies on logical rules as features to make interpretable inferences.
Ranta outperforms existing methods by a large margin in predictive performance and can make
reasonable repurposing suggestions with interpretable evidence
Algoritmos bio-inspirados para la detecciรณn de comunidades dinรกmicas en redes complejas
Tesis Doctoral inรฉdita leรญda en la Universidad Autรณnoma de Madrid, Escuela Politรฉcnica Superior, Departamento de Ingenierรญa Informรกtica. Fecha de Lectura: 22-07-202
Head-Driven Phrase Structure Grammar
Head-Driven Phrase Structure Grammar (HPSG) is a constraint-based or declarative approach to linguistic knowledge, which analyses all descriptive levels (phonology, morphology, syntax, semantics, pragmatics) with feature value pairs, structure sharing, and relational constraints. In syntax it assumes that expressions have a single relatively simple constituent structure. This volume provides a state-of-the-art introduction to the framework. Various chapters discuss basic assumptions and formal foundations, describe the evolution of the framework, and go into the details of the main syntactic phenomena. Further chapters are devoted to non-syntactic levels of description. The book also considers related fields and research areas (gesture, sign languages, computational linguistics) and includes chapters comparing HPSG with other frameworks (Lexical Functional Grammar, Categorial Grammar, Construction Grammar, Dependency Grammar, and Minimalism)
Network-driven strategies to integrate and exploit biomedical data
[eng] In the quest for understanding complex biological systems, the scientific community has been delving into protein, chemical and disease biology, populating biomedical databases with a wealth of data and knowledge. Currently, the field of biomedicine has entered a Big Data era, in which computational-driven research can largely benefit from existing knowledge to better understand and characterize biological and chemical entities. And yet, the heterogeneity and complexity of biomedical data trigger the need for a proper integration and representation of this knowledge, so that it can be effectively and efficiently exploited.
In this thesis, we aim at developing new strategies to leverage the current biomedical knowledge, so that meaningful information can be extracted and fused into downstream applications. To this goal, we have capitalized on network analysis algorithms to integrate and exploit biomedical data in a wide variety of scenarios, providing a better understanding of pharmacoomics experiments while helping accelerate the drug discovery process. More specifically, we have (i) devised an approach to identify functional gene sets associated with drug response mechanisms of action, (ii) created a resource of biomedical descriptors able to anticipate cellular drug response and identify new drug repurposing opportunities, (iii) designed a tool to annotate biomedical support for a given set of experimental observations, and (iv) reviewed different chemical and biological descriptors relevant for drug discovery, illustrating how they can be used to provide solutions to current challenges in biomedicine.[cat] En la cerca dโuna millor comprensiรณ dels sistemes biolรฒgics complexos, la comunitat cientรญfica ha estat aprofundint en la biologia de les proteรฏnes, fร rmacs i malalties, poblant les bases de dades biomรจdiques amb un gran volum de dades i coneixement. En lโactualitat, el camp de la biomedicina es troba en una era de โdades massivesโ (Big Data), on la investigaciรณ duta a terme per ordinadors seโn pot beneficiar per entendre i caracteritzar millor les entitats quรญmiques i biolรฒgiques. No obstant, la heterogeneรฏtat i complexitat de les dades biomรจdiques requereix que aquestes sโintegrin i es representin dโuna manera idรฒnia, permetent aixรญ explotar aquesta informaciรณ dโuna manera efectiva i eficient.
Lโobjectiu dโaquesta tesis doctoral รฉs desenvolupar noves estratรจgies que permetin explotar el coneixement biomรจdic actual i aixรญ extreure informaciรณ rellevant per aplicacions biomรจdiques futures. Per aquesta finalitat, em fet servir algoritmes de xarxes per tal dโintegrar i explotar el coneixement biomรจdic en diferents tasques, proporcionant un millor enteniment dels experiments farmacoรฒmics per tal dโajudar accelerar el procรฉs de descobriment de nous fร rmacs. Com a resultat, en aquesta tesi hem (i) dissenyat una estratรจgia per identificar grups funcionals de gens associats a la resposta de lรญnies celยทlulars als fร rmacs, (ii) creat una colยทlecciรณ de descriptors biomรจdics capaรงos, entre altres coses, dโanticipar com les cรจlยทlules responen als fร rmacs o trobar nous usos per fร rmacs existents, (iii) desenvolupat una eina per descobrir quins contextos biolรฒgics corresponen a una associaciรณ biolรฒgica observada experimentalment i, finalment, (iv) hem explorat diferents descriptors quรญmics i biolรฒgics rellevants pel procรฉs de descobriment de nous fร rmacs, mostrant com aquests poden ser utilitzats per trobar solucions a reptes actuals dins el camp de la biomedicina
Hypergraph product codes: a bridge to scalable quantum computers
A physical machine for storage and manipulation of information, being physical, will always be subject to noise and failure. For this reason, the design of fault-tolerant architectures is of prime importance for building a working quantum computer. Quantum error correction codes offer a possible elegant framework for fault-tolerance when provided with methods to operate qubits without corrupting the information stored therein. This work specialises in hypergraph product (HGP) codes and seeks to lay the groundwork for a quantum computer architecture based on them.
The leading approach to fault-tolerant quantum computation is, today, based on the planar code. A planar-code-based quantum computer, however, would require dramatic qubit overhead and we believe that good low-density parity-check (LDPC) codes are necessary to attain the full potential of quantum computing. The HGP codes, of which the planar code is an instance, are not, strictly speaking, good LDPC codes. Still, they are an efficient alternative. On the one hand, the best HGP codes improve upon the planar code as they can store multiple logical qubits. On the other, they are not considered good because their noise robustness is sub-optimal. Nonetheless, we see the design of a HGP-based quantum computer as a bridge between the currently-favoured planar code design and the gold standard of good LDPC codes. A HGP-based architecture would inform our knowledge on how to design fault-tolerant protocols when a code stores multiple logical qubits, which is, to a large extent, still an open question.
Our first original contribution is a decoding algorithm for all families of two-fold HGP codes. Second, we exhibit a constructive method to implement some logical encoded operations, given HGP codes with particular symmetries. Last, we propose the concept of confinement as an essential characteristic for a code family to be robust against syndrome measurement errors. Importantly, we show that both expander and three-dimensional HGP codes have the desired confinement property
๋์ข , ์ด์ข , ๊ทธ๋ฆฌ๊ณ ๋๋ฌด ํํ์ ๊ทธ๋ํ๋ฅผ ์ํ ๋น์ง๋ ํํ ํ์ต
ํ์๋
ผ๋ฌธ(๋ฐ์ฌ) -- ์์ธ๋ํ๊ต๋ํ์ : ๊ณต๊ณผ๋ํ ์ ๊ธฐยท์ ๋ณด๊ณตํ๋ถ, 2022. 8. ์ต์ง์.๊ทธ๋ํ ๋ฐ์ดํฐ์ ๋ํ ๋น์ง๋ ํํ ํ์ต์ ๋ชฉ์ ์ ๊ทธ๋ํ์ ๊ตฌ์กฐ์ ๋
ธ๋์ ์์ฑ์ ์ ๋ฐ์ํ๋ ์ ์ฉํ ๋
ธ๋ ๋จ์ ํน์ ๊ทธ๋ํ ๋จ์์ ๋ฒกํฐ ํํ ํํ์ ํ์ตํ๋ ๊ฒ์ด๋ค. ์ต๊ทผ, ๊ทธ๋ํ ๋ฐ์ดํฐ์ ๋ํด ๊ฐ๋ ฅํ ํํ ํ์ต ๋ฅ๋ ฅ์ ๊ฐ์ถ ๊ทธ๋ํ ์ ๊ฒฝ๋ง์ ํ์ฉํ ๋น์ง๋ ๊ทธ๋ํ ํํ ํ์ต ๋ชจ๋ธ์ ์ค๊ณ๊ฐ ์ฃผ๋ชฉ์ ๋ฐ๊ณ ์๋ค. ๋ง์ ๋ฐฉ๋ฒ๋ค์ ํ ์ข
๋ฅ์ ์ฃ์ง์ ํ ์ข
๋ฅ์ ๋
ธ๋๊ฐ ์กด์ฌํ๋ ๋์ข
๊ทธ๋ํ์ ๋ํ ํ์ต์ ์ง์ค์ ํ๋ค. ํ์ง๋ง ์ด ์ธ์์ ์๋ง์ ์ข
๋ฅ์ ๊ด๊ณ๊ฐ ์กด์ฌํ๊ธฐ ๋๋ฌธ์, ๊ทธ๋ํ ๋ํ ๊ตฌ์กฐ์ , ์๋ฏธ๋ก ์ ์์ฑ์ ํตํด ๋ค์ํ ์ข
๋ฅ๋ก ๋ถ๋ฅํ ์ ์๋ค. ๊ทธ๋์, ๊ทธ๋ํ๋ก๋ถํฐ ์ ์ฉํ ํํ์ ํ์ตํ๊ธฐ ์ํด์๋ ๋น์ง๋ ํ์ต ํ๋ ์์ํฌ๋ ์
๋ ฅ ๊ทธ๋ํ์ ํน์ง์ ์ ๋๋ก ๊ณ ๋ คํด์ผ๋ง ํ๋ค. ๋ณธ ํ์๋
ผ๋ฌธ์์ ์ฐ๋ฆฌ๋ ๋๋ฆฌ ์ ํ ์ ์๋ ์ธ๊ฐ์ง ๊ทธ๋ํ ๊ตฌ์กฐ์ธ ๋์ข
๊ทธ๋ํ, ํธ๋ฆฌ ํํ์ ๊ทธ๋ํ, ๊ทธ๋ฆฌ๊ณ ์ด์ข
๊ทธ๋ํ์ ๋ํ ๊ทธ๋ํ ์ ๊ฒฝ๋ง์ ํ์ฉํ๋ ๋น์ง๋ ํ์ต ๋ชจ๋ธ๋ค์ ์ ์ํ๋ค.
์ฒ์์ผ๋ก, ์ฐ๋ฆฌ๋ ๋์ข
๊ทธ๋ํ์ ๋
ธ๋์ ๋ํ์ฌ ์ ์ฐจ์ ํํ์ ํ์ตํ๋ ๊ทธ๋ํ ์ปจ๋ณผ๋ฃจ์
์คํ ์ธ์ฝ๋ ๋ชจ๋ธ์ ์ ์ํ๋ค. ๊ธฐ์กด์ ๊ทธ๋ํ ์คํ ์ธ์ฝ๋๋ ๊ตฌ์กฐ์ ์ ์ฒด๊ฐ ํ์ต์ด ๋ถ๊ฐ๋ฅํด์ ์ ํ์ ์ธ ํํ ํ์ต ๋ฅ๋ ฅ์ ๊ฐ์ง ์ ์๋ ๋ฐ๋ฉด์, ์ ์ํ๋ ์คํ ์ธ์ฝ๋๋ ๋
ธ๋์ ํผ์ณ๋ฅผ ๋ณต์ํ๋ฉฐ,๊ตฌ์กฐ์ ์ ์ฒด๊ฐ ํ์ต์ด ๊ฐ๋ฅํ๋ค. ๋
ธ๋์ ํผ์ณ๋ฅผ ๋ณต์ํ๊ธฐ ์ํด์, ์ฐ๋ฆฌ๋ ์ธ์ฝ๋ ๋ถ๋ถ์ ์ญํ ์ด ์ด์ํ ๋
ธ๋๋ผ๋ฆฌ ์ ์ฌํ ํํ์ ๊ฐ์ง๊ฒ ํ๋ ๋ผํ๋ผ์์ ์ค๋ฌด๋ฉ์ด๋ผ๋ ๊ฒ์ ์ฃผ๋ชฉํ์ฌ ๋์ฝ๋ ๋ถ๋ถ์์๋ ์ด์ ๋
ธ๋์ ํํ๊ณผ ๋ฉ์ด์ง๊ฒ ํ๋ ๋ผํ๋ผ์์ ์คํ๋์ ํ๋๋ก ์ค๊ณํ์๋ค. ๋ํ ๋ผํ๋ผ์์ ์คํ๋์ ๊ทธ๋๋ก ์ ์ฉํ๋ฉด ๋ถ์์ ์ฑ์ ์ ๋ฐํ ์ ์๊ธฐ ๋๋ฌธ์, ์ฃ์ง์ ๊ฐ์ค์น ๊ฐ์ ์์ ๊ฐ์ ์ค ์ ์๋ ๋ถํธํ ๊ทธ๋ํ๋ฅผ ํ์ฉํ์ฌ ์์ ์ ์ธ ๋ผํ๋ผ์์ ์คํ๋์ ํํ๋ฅผ ์ ์ํ์๋ค. ๋์ข
๊ทธ๋ํ์ ๋ํ ๋
ธ๋ ํด๋ฌ์คํฐ๋ง๊ณผ ๋งํฌ ์์ธก ์คํ์ ํตํ์ฌ ์ ์ํ๋ ๋ฐฉ๋ฒ์ด ์์ ์ ์ผ๋ก ์ฐ์ํ ์ฑ๋ฅ์ ๋ณด์์ ํ์ธํ์๋ค.
๋์งธ๋ก, ์ฐ๋ฆฌ๋ ํธ๋ฆฌ์ ํํ๋ฅผ ๊ฐ์ง๋ ๊ณ์ธต์ ์ธ ๊ด๊ณ๋ฅผ ๊ฐ์ง๊ณ ์๋ ๊ทธ๋ํ์ ๋
ธ๋ ํํ์ ์ ํํ๊ฒ ํ์ตํ๊ธฐ ์ํ์ฌ ์๊ณก์ ๊ณต๊ฐ์์ ๋์ํ๋ ์คํ ์ธ์ฝ๋ ๋ชจ๋ธ์ ์ ์ํ๋ค. ์ ํด๋ฆฌ๋์ธ ๊ณต๊ฐ์ ํธ๋ฆฌ๋ฅผ ์ฌ์ํ๊ธฐ์ ๋ถ์ ์ ํ๋ค๋ ์ต๊ทผ์ ๋ถ์์ ํตํ์ฌ, ์๊ณก์ ๊ณต๊ฐ์์ ๊ทธ๋ํ ์ ๊ฒฝ๋ง์ ๋ ์ด์ด๋ฅผ ํ์ฉํ์ฌ ๋
ธ๋์ ์ ์ฐจ์ ํํ์ ํ์ตํ๊ฒ ๋๋ค. ์ด ๋, ๊ทธ๋ํ ์ ๊ฒฝ๋ง์ด ์๊ณก์ ๊ธฐํํ์์ ๊ณ์ธต ์ ๋ณด๋ฅผ ๋ด๊ณ ์๋ ๊ฑฐ๋ฆฌ์ ๊ฐ์ ํ์ฉํ์ฌ ๋
ธ๋์ ์ด์์ฌ์ด์ ์ค์๋๋ฅผ ํ์ฉํ๋๋ก ์ค๊ณํ์๋ค. ์ฐ๋ฆฌ๋ ๋
ผ๋ฌธ ์ธ์ฉ ๊ด๊ณ ๋คํธ์ํฌ, ๊ณํต๋, ์ด๋ฏธ์ง ์ฌ์ด์ ๋คํธ์ํฌ๋ฑ์ ๋ํด ์ ์ํ ๋ชจ๋ธ์ ์ ์ฉํ์ฌ ๋
ธ๋ ํด๋ฌ์คํฐ๋ง๊ณผ ๋งํฌ ์์ธก ์คํ์ ํ์์ผ๋ฉฐ, ํธ๋ฆฌ์ ํํ๋ฅผ ๊ฐ์ง๋ ๊ทธ๋ํ์ ๋ํด์ ์ ์ํ ๋ชจ๋ธ์ด ์ ํด๋ฆฌ๋์ธ ๊ณต๊ฐ์์ ์ํํ๋ ๋ชจ๋ธ์ ๋นํด ํฅ์๋ ์ฑ๋ฅ์ ๋ณด์๋ค๋ ๊ฒ์ ํ์ธํ์๋ค.
๋ง์ง๋ง์ผ๋ก, ์ฐ๋ฆฌ๋ ์ฌ๋ฌ ์ข
๋ฅ์ ๋
ธ๋์ ์ฃ์ง๋ฅผ ๊ฐ์ง๋ ์ด์ข
๊ทธ๋ํ์ ๋ํ ๋์กฐ ํ์ต ๋ชจ๋ธ์ ์ ์ํ๋ค. ์ฐ๋ฆฌ๋ ๊ธฐ์กด์ ๋ฐฉ๋ฒ๋ค์ด ํ์ตํ๊ธฐ ์ด์ ์ ์ถฉ๋ถํ ๋๋ฉ์ธ ์ง์์ ์ฌ์ฉํ์ฌ ์ค๊ณํ ๋ฉํํจ์ค๋ ๋ฉํ๊ทธ๋ํ์ ์์กดํ๋ค๋ ๋จ์ ๊ณผ ๋ง์ ์ด์ข
๊ทธ๋ํ์ ์ฃ์ง๊ฐ ๋ค๋ฅธ ๋
ธ๋ ์ข
๋ฅ์ฌ์ด์ ๊ด๊ณ์ ์ง์คํ๊ณ ์๋ค๋ ์ ์ ์ฃผ๋ชฉํ์๋ค. ์ด๋ฅผ ํตํด ์ฐ๋ฆฌ๋ ์ฌ์ ๊ณผ์ ์ด ํ์์์ผ๋ฉฐ ๋ค๋ฅธ ์ข
๋ฅ ์ฌ์ด์ ๊ด๊ณ์ ๋ํ์ฌ ๊ฐ์ ์ข
๋ฅ ์ฌ์ด์ ๊ด๊ณ๋ ๋์์ ํจ์จ์ ์ผ๋ก ํ์ตํ๊ฒ ํ๋ ๋ฉํ๋
ธ๋๋ผ๋ ๊ฐ๋
์ ์ ์ํ์๋ค. ๋ํ ๋ฉํ๋
ธ๋๋ฅผ ๊ธฐ๋ฐ์ผ๋กํ๋ ๊ทธ๋ํ ์ ๊ฒฝ๋ง๊ณผ ๋์กฐ ํ์ต ๋ชจ๋ธ์ ์ ์ํ์๋ค. ์ฐ๋ฆฌ๋ ์ ์ํ ๋ชจ๋ธ์ ๋ฉํํจ์ค๋ฅผ ์ฌ์ฉํ๋ ์ด์ข
๊ทธ๋ํ ํ์ต ๋ชจ๋ธ๊ณผ ๋
ธ๋ ํด๋ฌ์คํฐ๋ง ๋ฑ์ ์คํ ์ฑ๋ฅ์ผ๋ก ๋น๊ตํด๋ณด์์ ๋, ๋น๋ฑํ๊ฑฐ๋ ๋์ ์ฑ๋ฅ์ ๋ณด์์์ ํ์ธํ์๋ค.The goal of unsupervised graph representation learning is extracting useful node-wise or graph-wise vector representation that is aware of the intrinsic structures of the graph and its attributes. These days, designing methodology of unsupervised graph representation learning based on graph neural networks has growing attention due to their powerful representation ability. Many methods are focused on a homogeneous graph that is a network with a single type of node and a single type of edge. However, as many types of relationships exist in this world, graphs can also be classified into various types by structural and semantic properties. For this reason, to learn useful representations from graphs, the unsupervised learning framework must consider the characteristics of the input graph. In this dissertation, we focus on designing unsupervised learning models using graph neural networks for three graph structures that are widely available: homogeneous graphs, tree-like graphs, and heterogeneous graphs.
First, we propose a symmetric graph convolutional autoencoder which produces a low-dimensional latent representation from a homogeneous graph. In contrast to the existing graph autoencoders with asymmetric decoder parts, the proposed autoencoder has a newly designed decoder which builds a completely symmetric autoencoder form. For the reconstruction of node features, the decoder is designed based on Laplacian sharpening as the counterpart of Laplacian smoothing of the encoder, which allows utilizing the graph structure in the whole processes of the proposed autoencoder architecture. In order to prevent the numerical instability of the network caused by the Laplacian sharpening introduction, we further propose a new numerically stable form of the Laplacian sharpening by incorporating the signed graphs. The experimental results of clustering, link prediction and visualization tasks on homogeneous graphs strongly support that the proposed model is stable and outperforms various state-of-the-art algorithms.
Second, we analyze how unsupervised tasks can benefit from learned representations in hyperbolic space. To explore how well the hierarchical structure of unlabeled data can be represented in hyperbolic spaces, we design a novel hyperbolic message passing autoencoder whose overall auto-encoding is performed in hyperbolic space. The proposed model conducts auto-encoding the networks via fully utilizing hyperbolic geometry in message passing. Through extensive quantitative and qualitative analyses, we validate the properties and benefits of the unsupervised hyperbolic representations of tree-like graphs.
Third, we propose the novel concept of metanode for message passing to learn both heterogeneous and homogeneous relationships between any two nodes without meta-paths and meta-graphs. Unlike conventional methods, metanodes do not require a predetermined step to manipulate the given relations between different types to enrich relational information. Going one step further, we propose a metanode-based message passing layer and a contrastive learning model using the proposed layer. In our experiments, we show the competitive performance of the proposed metanode-based message passing method on node clustering and node classification tasks, when compared to state-of-the-art methods for message passing networks for heterogeneous graphs.1 Introduction 1
2 Representation Learning on Graph-Structured Data 4
2.1 Basic Introduction 4
2.1.1 Notations 5
2.2 Traditional Approaches 5
2.2.1 Graph Statistic 5
2.2.2 Neighborhood Overlap 7
2.2.3 Graph Kernel 9
2.2.4 Spectral Approaches 10
2.3 Node Embeddings I: Factorization and Random Walks 15
2.3.1 Factorization-based Methods 15
2.3.2 Random Walk-based Methods 16
2.4 Node Embeddings II: Graph Neural Networks 17
2.4.1 Overview of Framework 17
2.4.2 Representative Models 18
2.5 Learning in Unsupervised Environments 21
2.5.1 Predictive Coding 21
2.5.2 Contrastive Coding 22
2.6 Applications 24
2.6.1 Classifications 24
2.6.2 Link Prediction 26
3 Autoencoder Architecture for Homogeneous Graphs 27
3.1 Overview 27
3.2 Preliminaries 30
3.2.1 Spectral Convolution on Graphs 30
3.2.2 Laplacian Smoothing 32
3.3 Methodology 33
3.3.1 Laplacian Sharpening 33
3.3.2 Numerically Stable Laplacian Sharpening 34
3.3.3 Subspace Clustering Cost for Image Clustering 37
3.3.4 Training 39
3.4 Experiments 40
3.4.1 Datasets 40
3.4.2 Experimental Settings 42
3.4.3 Comparing Methods 42
3.4.4 Node Clustering 43
3.4.5 Image Clustering 45
3.4.6 Ablation Studies 46
3.4.7 Link Prediction 47
3.4.8 Visualization 47
3.5 Summary 49
4 Autoencoder Architecture for Tree-like Graphs 50
4.1 Overview 50
4.2 Preliminaries 52
4.2.1 Hyperbolic Embeddings 52
4.2.2 Hyperbolic Geometry 53
4.3 Methodology 55
4.3.1 Geometry-Aware Message Passing 56
4.3.2 Nonlinear Activation 57
4.3.3 Loss Function 58
4.4 Experiments 58
4.4.1 Datasets 59
4.4.2 Compared Methods 61
4.4.3 Experimental Details 62
4.4.4 Node Clustering and Link Prediction 64
4.4.5 Image Clustering 66
4.4.6 Structure-Aware Unsupervised Embeddings 68
4.4.7 Hyperbolic Distance to Filter Training Samples 71
4.4.8 Ablation Studies 74
4.5 Further Discussions 75
4.5.1 Connection to Contrastive Learning 75
4.5.2 Failure Cases of Hyperbolic Embedding Spaces 75
4.6 Summary 77
5 Contrastive Learning for Heterogeneous Graphs 78
5.1 Overview 78
5.2 Preliminaries 82
5.2.1 Meta-path 82
5.2.2 Representation Learning on Heterogeneous Graphs 82
5.2.3 Contrastive methods for Heterogeneous Graphs 83
5.3 Methodology 84
5.3.1 Definitions 84
5.3.2 Metanode-based Message Passing Layer 86
5.3.3 Contrastive Learning Framework 88
5.4 Experiments 89
5.4.1 Experimental Details 90
5.4.2 Node Classification 94
5.4.3 Node Clustering 96
5.4.4 Visualization 96
5.4.5 Effectiveness of Metanodes 97
5.5 Summary 99
6 Conclusions 101๋ฐ
Sparsity-aware neural user behavior modeling in online interaction platforms
Modern online platforms offer users an opportunity to participate in a variety of content-creation, social networking, and shopping activities. With the rapid proliferation of such online services, learning data-driven user behavior models is indispensable to enable personalized user experiences. Recently, representation learning has emerged as an effective strategy for user modeling, powered by neural networks trained over large volumes of interaction data. Despite their enormous potential, we encounter the unique challenge of data sparsity for a vast majority of entities, e.g., sparsity in ground-truth labels for entities and in entity-level interactions (cold-start users, items in the long-tail, and ephemeral groups).
In this dissertation, we develop generalizable neural representation learning frameworks for user behavior modeling designed to address different sparsity challenges across applications. Our problem settings span transductive and inductive learning scenarios, where transductive learning models entities seen during training and inductive learning targets entities that are only observed during inference. We leverage different facets of information reflecting user behavior (e.g., interconnectivity in social networks, temporal and attributed interaction information) to enable personalized inference at scale. Our proposed models are complementary to concurrent advances in neural architectural choices and are adaptive to the rapid addition of new applications in online platforms.
First, we examine two transductive learning settings: inference and recommendation in graph-structured and bipartite user-item interactions. In chapter 3, we formulate user profiling in social platforms as semi-supervised learning over graphs given sparse ground-truth labels for node attributes. We present a graph neural network framework that exploits higher-order connectivity structures (network motifs) to learn attributed structural roles of nodes that identify structurally similar nodes with co-varying local attributes. In chapter 4, we design neural collaborative filtering models for few-shot recommendations over user-item interactions. To address item interaction sparsity due to heavy-tailed distributions, our proposed meta-learning framework learns-to-recommend few-shot items by knowledge transfer from arbitrary base recommenders. We show that our framework consistently outperforms state-of-art approaches on overall recommendation (by 5% Recall) while achieving significant gains (of 60-80% Recall) for tail items with fewer than 20 interactions.
Next, we explored three inductive learning settings: modeling spread of user-generated content in social networks; item recommendations for ephemeral groups; and friend ranking in large-scale social platforms. In chapter 5, we focus on diffusion prediction in social networks where a vast population of users rarely post content. We introduce a deep generative modeling framework that models users as probability distributions in the latent space with variational priors parameterized by graph neural networks. Our approach enables massive performance gains (over 150% recall) for users with sparse activities while being faster than state-of-the-art neural models by an order of magnitude. In chapter 6, we examine item recommendations for ephemeral groups with limited or no historical interactions together. To overcome group interaction sparsity, we present self-supervised learning strategies that exploit the preference co-variance in observed group memberships for group recommender training. Our framework achieves significant performance gains (over 30% NDCG) over prior state-of-the-art group recommendation models. In chapter 7, we introduce multi-modal inference with graph neural networks that captures knowledge from multiple feature modalities and user interactions for multi-faceted friend ranking. Our approach achieves notable higher performance gains for critical populations of less-active and low degree users