30 research outputs found
Integrating Natural Language Processing (NLP) and Language Resources Using Linked Data
This thesis is a compendium of scientific works and engineering
specifications that have been contributed to a large community of
stakeholders to be copied, adapted, mixed, built upon and exploited in
any way possible to achieve a common goal: Integrating Natural Language
Processing (NLP) and Language Resources Using Linked Data
The explosion of information technology in the last two decades has led
to a substantial growth in quantity, diversity and complexity of
web-accessible linguistic data. These resources become even more useful
when linked with each other and the last few years have seen the
emergence of numerous approaches in various disciplines concerned with
linguistic resources and NLP tools. It is the challenge of our time to
store, interlink and exploit this wealth of data accumulated in more
than half a century of computational linguistics, of empirical,
corpus-based study of language, and of computational lexicography in all
its heterogeneity.
The vision of the Giant Global Graph (GGG) was conceived by Tim
Berners-Lee aiming at connecting all data on the Web and allowing to
discover new relations between this openly-accessible data. This vision
has been pursued by the Linked Open Data (LOD) community, where the
cloud of published datasets comprises 295 data repositories and more
than 30 billion RDF triples (as of September 2011).
RDF is based on globally unique and accessible URIs and it was
specifically designed to establish links between such URIs (or
resources). This is captured in the Linked Data paradigm that postulates
four rules: (1) Referred entities should be designated by URIs, (2)
these URIs should be resolvable over HTTP, (3) data should be
represented by means of standards such as RDF, (4) and a resource should
include links to other resources.
Although it is difficult to precisely identify the reasons for the
success of the LOD effort, advocates generally argue that open licenses
as well as open access are key enablers for the growth of such a network
as they provide a strong incentive for collaboration and contribution by
third parties. In his keynote at BNCOD 2011, Chris Bizer argued that
with RDF the overall data integration effort can be “split between data
publishers, third parties, and the data consumer”, a claim that can be
substantiated by observing the evolution of many large data sets
constituting the LOD cloud.
As written in the acknowledgement section, parts of this thesis has
received numerous feedback from other scientists, practitioners and
industry in many different ways. The main contributions of this thesis
are summarized here:
Part I – Introduction and Background.
During his keynote at the Language Resource and Evaluation Conference in
2012, Sören Auer stressed the decentralized, collaborative, interlinked
and interoperable nature of the Web of Data. The keynote provides strong
evidence that Semantic Web technologies such as Linked Data are on its
way to become main stream for the representation of language resources.
The jointly written companion publication for the keynote was later
extended as a book chapter in The People’s Web Meets NLP and serves as
the basis for “Introduction” and “Background”, outlining some stages of
the Linked Data publication and refinement chain. Both chapters stress
the importance of open licenses and open access as an enabler for
collaboration, the ability to interlink data on the Web as a key feature
of RDF as well as provide a discussion about scalability issues and
decentralization. Furthermore, we elaborate on how conceptual
interoperability can be achieved by (1) re-using vocabularies, (2) agile
ontology development, (3) meetings to refine and adapt ontologies and
(4) tool support to enrich ontologies and match schemata.
Part II - Language Resources as Linked Data.
“Linked Data in Linguistics” and “NLP & DBpedia, an Upward Knowledge
Acquisition Spiral” summarize the results of the Linked Data in
Linguistics (LDL) Workshop in 2012 and the NLP & DBpedia Workshop in
2013 and give a preview of the MLOD special issue. In total, five
proceedings – three published at CEUR (OKCon 2011, WoLE 2012, NLP &
DBpedia 2013), one Springer book (Linked Data in Linguistics, LDL 2012)
and one journal special issue (Multilingual Linked Open Data, MLOD to
appear) – have been (co-)edited to create incentives for scientists to
convert and publish Linked Data and thus to contribute open and/or
linguistic data to the LOD cloud. Based on the disseminated call for
papers, 152 authors contributed one or more accepted submissions to our
venues and 120 reviewers were involved in peer-reviewing.
“DBpedia as a Multilingual Language Resource” and “Leveraging the
Crowdsourcing of Lexical Resources for Bootstrapping a Linguistic Linked
Data Cloud” contain this thesis’ contribution to the DBpedia Project in
order to further increase the size and inter-linkage of the LOD Cloud
with lexical-semantic resources. Our contribution comprises extracted
data from Wiktionary (an online, collaborative dictionary similar to
Wikipedia) in more than four languages (now six) as well as
language-specific versions of DBpedia, including a quality assessment of
inter-language links between Wikipedia editions and internationalized
content negotiation rules for Linked Data. In particular the work
described in created the foundation for a DBpedia Internationalisation
Committee with members from over 15 different languages with the common
goal to push DBpedia as a free and open multilingual language resource.
Part III - The NLP Interchange Format (NIF).
“NIF 2.0 Core Specification”, “NIF 2.0 Resources and Architecture” and
“Evaluation and Related Work” constitute one of the main contribution of
this thesis. The NLP Interchange Format (NIF) is an RDF/OWL-based format
that aims to achieve interoperability between Natural Language
Processing (NLP) tools, language resources and annotations. The core
specification is included in and describes which URI schemes and RDF
vocabularies must be used for (parts of) natural language texts and
annotations in order to create an RDF/OWL-based interoperability layer
with NIF built upon Unicode Code Points in Normal Form C. In , classes
and properties of the NIF Core Ontology are described to formally define
the relations between text, substrings and their URI schemes. contains
the evaluation of NIF.
In a questionnaire, we asked questions to 13 developers using NIF. UIMA,
GATE and Stanbol are extensible NLP frameworks and NIF was not yet able
to provide off-the-shelf NLP domain ontologies for all possible domains,
but only for the plugins used in this study. After inspecting the
software, the developers agreed however that NIF is adequate enough to
provide a generic RDF output based on NIF using literal objects for
annotations. All developers were able to map the internal data structure
to NIF URIs to serialize RDF output (Adequacy). The development effort
in hours (ranging between 3 and 40 hours) as well as the number of code
lines (ranging between 110 and 445) suggest, that the implementation of
NIF wrappers is easy and fast for an average developer. Furthermore the
evaluation contains a comparison to other formats and an evaluation of
the available URI schemes for web annotation.
In order to collect input from the wide group of stakeholders, a total
of 16 presentations were given with extensive discussions and feedback,
which has lead to a constant improvement of NIF from 2010 until 2013.
After the release of NIF (Version 1.0) in November 2011, a total of 32
vocabulary employments and implementations for different NLP tools and
converters were reported (8 by the (co-)authors, including Wiki-link
corpus, 13 by people participating in our survey and 11 more, of
which we have heard). Several roll-out meetings and tutorials were held
(e.g. in Leipzig and Prague in 2013) and are planned (e.g. at LREC
2014).
Part IV - The NLP Interchange Format in Use.
“Use Cases and Applications for NIF” and “Publication of Corpora using
NIF” describe 8 concrete instances where NIF has been successfully used.
One major contribution in is the usage of NIF as the recommended RDF
mapping in the Internationalization Tag Set (ITS) 2.0 W3C standard
and the conversion algorithms from ITS to NIF and back. One outcome
of the discussions in the standardization meetings and telephone
conferences for ITS 2.0 resulted in the conclusion there was no
alternative RDF format or vocabulary other than NIF with the required
features to fulfill the working group charter. Five further uses of NIF
are described for the Ontology of Linguistic Annotations (OLiA), the
RDFaCE tool, the Tiger Corpus Navigator, the OntosFeeder and
visualisations of NIF using the RelFinder tool. These 8 instances
provide an implemented proof-of-concept of the features of NIF.
starts with describing the conversion and hosting of the huge Google
Wikilinks corpus with 40 million annotations for 3 million web sites.
The resulting RDF dump contains 477 million triples in a 5.6 GB
compressed dump file in turtle syntax. describes how NIF can be used to
publish extracted facts from news feeds in the RDFLiveNews tool as
Linked Data.
Part V - Conclusions.
provides lessons learned for NIF, conclusions and an outlook on future
work. Most of the contributions are already summarized above. One
particular aspect worth mentioning is the increasing number of
NIF-formated corpora for Named Entity Recognition (NER) that have come
into existence after the publication of the main NIF paper Integrating
NLP using Linked Data at ISWC 2013. These include the corpora converted
by Steinmetz, Knuth and Sack for the NLP & DBpedia workshop and an
OpenNLP-based CoNLL converter by BrĂĽmmer. Furthermore, we are aware of
three LREC 2014 submissions that leverage NIF: NIF4OGGD - NLP
Interchange Format for Open German Governmental Data, N^3 – A Collection
of Datasets for Named Entity Recognition and Disambiguation in the NLP
Interchange Format and Global Intelligent Content: Active Curation of
Language Resources using Linked Data as well as an early implementation
of a GATE-based NER/NEL evaluation framework by Dojchinovski and Kliegr.
Further funding for the maintenance, interlinking and publication of
Linguistic Linked Data as well as support and improvements of NIF is
available via the expiring LOD2 EU project, as well as the CSA EU
project called LIDER, which started in November 2013. Based on the
evidence of successful adoption presented in this thesis, we can expect
a decent to high chance of reaching critical mass of Linked Data
technology as well as the NIF standard in the field of Natural Language
Processing and Language Resources.:CONTENTS
i introduction and background 1
1 introduction 3
1.1 Natural Language Processing . . . . . . . . . . . . . . . 3
1.2 Open licenses, open access and collaboration . . . . . . 5
1.3 Linked Data in Linguistics . . . . . . . . . . . . . . . . . 6
1.4 NLP for and by the Semantic Web – the NLP Inter-
change Format (NIF) . . . . . . . . . . . . . . . . . . . . 8
1.5 Requirements for NLP Integration . . . . . . . . . . . . 10
1.6 Overview and Contributions . . . . . . . . . . . . . . . 11
2 background 15
2.1 The Working Group on Open Data in Linguistics (OWLG) 15
2.1.1 The Open Knowledge Foundation . . . . . . . . 15
2.1.2 Goals of the Open Linguistics Working Group . 16
2.1.3 Open linguistics resources, problems and chal-
lenges . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.4 Recent activities and on-going developments . . 18
2.2 Technological Background . . . . . . . . . . . . . . . . . 18
2.3 RDF as a data model . . . . . . . . . . . . . . . . . . . . 21
2.4 Performance and scalability . . . . . . . . . . . . . . . . 22
2.5 Conceptual interoperability . . . . . . . . . . . . . . . . 22
ii language resources as linked data 25
3 linked data in linguistics 27
3.1 Lexical Resources . . . . . . . . . . . . . . . . . . . . . . 29
3.2 Linguistic Corpora . . . . . . . . . . . . . . . . . . . . . 30
3.3 Linguistic Knowledgebases . . . . . . . . . . . . . . . . 31
3.4 Towards a Linguistic Linked Open Data Cloud . . . . . 32
3.5 State of the Linguistic Linked Open Data Cloud in 2012 33
3.6 Querying linked resources in the LLOD . . . . . . . . . 36
3.6.1 Enriching metadata repositories with linguistic
features (Glottolog → OLiA) . . . . . . . . . . . 36
3.6.2 Enriching lexical-semantic resources with lin-
guistic information (DBpedia (→ POWLA) →
OLiA) . . . . . . . . . . . . . . . . . . . . . . . . 38
4 DBpedia as a multilingual language resource:
the case of the greek dbpedia edition. 39
4.1 Current state of the internationalization effort . . . . . 40
4.2 Language-specific design of DBpedia resource identifiers 41
4.3 Inter-DBpedia linking . . . . . . . . . . . . . . . . . . . 42
4.4 Outlook on DBpedia Internationalization . . . . . . . . 44
5 leveraging the crowdsourcing of lexical resources
for bootstrapping a linguistic linked data cloud 47
5.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . 48
5.2 Problem Description . . . . . . . . . . . . . . . . . . . . 50
5.2.1 Processing Wiki Syntax . . . . . . . . . . . . . . 50
5.2.2 Wiktionary . . . . . . . . . . . . . . . . . . . . . . 52
5.2.3 Wiki-scale Data Extraction . . . . . . . . . . . . . 53
5.3 Design and Implementation . . . . . . . . . . . . . . . . 54
5.3.1 Extraction Templates . . . . . . . . . . . . . . . . 56
5.3.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . 56
5.3.3 Language Mapping . . . . . . . . . . . . . . . . . 58
5.3.4 Schema Mediation by Annotation with lemon . 58
5.4 Resulting Data . . . . . . . . . . . . . . . . . . . . . . . . 58
5.5 Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . 60
5.6 Discussion and Future Work . . . . . . . . . . . . . . . 60
5.6.1 Next Steps . . . . . . . . . . . . . . . . . . . . . . 61
5.6.2 Open Research Questions . . . . . . . . . . . . . 61
6 nlp & dbpedia, an upward knowledge acquisition
spiral 63
6.1 Knowledge acquisition and structuring . . . . . . . . . 64
6.2 Representation of knowledge . . . . . . . . . . . . . . . 65
6.3 NLP tasks and applications . . . . . . . . . . . . . . . . 65
6.3.1 Named Entity Recognition . . . . . . . . . . . . 66
6.3.2 Relation extraction . . . . . . . . . . . . . . . . . 67
6.3.3 Question Answering over Linked Data . . . . . 67
6.4 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.4.1 Gold and silver standards . . . . . . . . . . . . . 69
6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
iii the nlp interchange format (nif) 73
7 nif 2.0 core specification 75
7.1 Conformance checklist . . . . . . . . . . . . . . . . . . . 75
7.2 Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7.2.1 Definition of Strings . . . . . . . . . . . . . . . . 78
7.2.2 Representation of Document Content with the
nif:Context Class . . . . . . . . . . . . . . . . . . 80
7.3 Extension of NIF . . . . . . . . . . . . . . . . . . . . . . 82
7.3.1 Part of Speech Tagging with OLiA . . . . . . . . 83
7.3.2 Named Entity Recognition with ITS 2.0, DBpe-
dia and NERD . . . . . . . . . . . . . . . . . . . 84
7.3.3 lemon and Wiktionary2RDF . . . . . . . . . . . 86
8 nif 2.0 resources and architecture 89
8.1 NIF Core Ontology . . . . . . . . . . . . . . . . . . . . . 89
8.1.1 Logical Modules . . . . . . . . . . . . . . . . . . 90
8.2 Workflows . . . . . . . . . . . . . . . . . . . . . . . . . . 91
8.2.1 Access via REST Services . . . . . . . . . . . . . 92
8.2.2 NIF Combinator Demo . . . . . . . . . . . . . .
92
8.3 Granularity Profiles . . . . . . . . . . . . . . . . . . . . .
93
8.4 Further URI Schemes for NIF . . . . . . . . . . . . . . .
95
8.4.1 Context-Hash-based URIs . . . . . . . . . . . . .
99
9 evaluation and related work 101
9.1 Questionnaire and Developers Study for NIF 1.0 . . . . 101
9.2 Qualitative Comparison with other Frameworks and
Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
9.3 URI Stability Evaluation . . . . . . . . . . . . . . . . . . 103
9.4 Related URI Schemes . . . . . . . . . . . . . . . . . . . . 104
iv the nlp interchange format in use 109
10 use cases and applications for nif 111
10.1 Internationalization Tag Set 2.0 . . . . . . . . . . . . . . 111
10.1.1 ITS2NIF and NIF2ITS conversion . . . . . . . . . 112
10.2 OLiA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
10.3 RDFaCE . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
10.4 Tiger Corpus Navigator . . . . . . . . . . . . . . . . . . 121
10.4.1 Tools and Resources . . . . . . . . . . . . . . . . 122
10.4.2 NLP2RDF in 2010 . . . . . . . . . . . . . . . . . . 123
10.4.3 Linguistic Ontologies . . . . . . . . . . . . . . . . 124
10.4.4 Implementation . . . . . . . . . . . . . . . . . . . 125
10.4.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . 126
10.4.6 Related Work and Outlook . . . . . . . . . . . . 129
10.5 OntosFeeder – a Versatile Semantic Context Provider
for Web Content Authoring . . . . . . . . . . . . . . . . 131
10.5.1 Feature Description and User Interface Walk-
through . . . . . . . . . . . . . . . . . . . . . . . 132
10.5.2 Architecture . . . . . . . . . . . . . . . . . . . . . 134
10.5.3 Embedding Metadata . . . . . . . . . . . . . . . 135
10.5.4 Related Work and Summary . . . . . . . . . . . 135
10.6 RelFinder: Revealing Relationships in RDF Knowledge
Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
10.6.1 Implementation . . . . . . . . . . . . . . . . . . . 137
10.6.2 Disambiguation . . . . . . . . . . . . . . . . . . . 138
10.6.3 Searching for Relationships . . . . . . . . . . . . 139
10.6.4 Graph Visualization . . . . . . . . . . . . . . . . 140
10.6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . 141
11 publication of corpora using nif 143
11.1 Wikilinks Corpus . . . . . . . . . . . . . . . . . . . . . . 143
11.1.1 Description of the corpus . . . . . . . . . . . . . 143
11.1.2 Quantitative Analysis with Google Wikilinks Cor-
pus . . . . . . . . . . . . . . . . . . . . . . . . . . 144
11.2 RDFLiveNews . . . . . . . . . . . . . . . . . . . . . . . . 144
11.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . 145
11.2.2 Mapping to RDF and Publication on the Web of
Data . . . . . . . . . . . . . . . . . . . . . . . . . 146
v conclusions 149
12 lessons learned, conclusions and future work 151
12.1 Lessons Learned for NIF . . . . . . . . . . . . . . . . . . 151
12.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 151
12.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 15
User Interfaces to the Web of Data based on Natural Language Generation
We explore how Virtual Research Environments based on Semantic Web technologies support research interactions with RDF data in various stages of corpus-based analysis, analyze the Web of Data in terms of human readability, derive labels from variables in SPARQL queries, apply Natural Language Generation to improve user interfaces to the Web of Data by verbalizing SPARQL queries and RDF graphs, and present a method to automatically induce RDF graph verbalization templates via distant supervision
Linked Data Supported Information Retrieval
Um Inhalte im World Wide Web ausfindig zu machen, sind Suchmaschienen nicht mehr wegzudenken. Semantic Web und Linked Data Technologien ermöglichen ein detaillierteres und eindeutiges Strukturieren der Inhalte und erlauben vollkommen neue Herangehensweisen an die Lösung von Information Retrieval Problemen. Diese Arbeit befasst sich mit den Möglichkeiten, wie Information Retrieval Anwendungen von der Einbeziehung von Linked Data profitieren können. Neue Methoden der computer-gestützten semantischen Textanalyse, semantischen Suche, Informationspriorisierung und -visualisierung werden vorgestellt und umfassend evaluiert. Dabei werden Linked Data Ressourcen und ihre Beziehungen in die Verfahren integriert, um eine Steigerung der Effektivität der Verfahren bzw. ihrer Benutzerfreundlichkeit zu erzielen. Zunächst wird eine Einführung in die Grundlagen des Information Retrieval und Linked Data gegeben. Anschließend werden neue manuelle und automatisierte Verfahren zum semantischen Annotieren von Dokumenten durch deren Verknüpfung mit Linked Data Ressourcen vorgestellt (Entity Linking). Eine umfassende Evaluation der Verfahren wird durchgeführt und das zu Grunde liegende Evaluationssystem umfangreich verbessert. Aufbauend auf den Annotationsverfahren werden zwei neue Retrievalmodelle zur semantischen Suche vorgestellt und evaluiert. Die Verfahren basieren auf dem generalisierten Vektorraummodell und beziehen die semantische Ähnlichkeit anhand von taxonomie-basierten Beziehungen der Linked Data Ressourcen in Dokumenten und Suchanfragen in die Berechnung der Suchergebnisrangfolge ein. Mit dem Ziel die Berechnung von semantischer Ähnlichkeit weiter zu verfeinern, wird ein Verfahren zur Priorisierung von Linked Data Ressourcen vorgestellt und evaluiert. Darauf aufbauend werden Visualisierungstechniken aufgezeigt mit dem Ziel, die Explorierbarkeit und Navigierbarkeit innerhalb eines semantisch annotierten Dokumentenkorpus zu verbessern. Hierfür werden zwei Anwendungen präsentiert. Zum einen eine Linked Data basierte explorative Erweiterung als Ergänzung zu einer traditionellen schlüsselwort-basierten Suchmaschine, zum anderen ein Linked Data basiertes Empfehlungssystem
Proceedings of the First Workshop on Computing News Storylines (CNewsStory 2015)
This volume contains the proceedings of the 1st Workshop on Computing News Storylines (CNewsStory
2015) held in conjunction with the 53rd Annual Meeting of the Association for Computational
Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP
2015) at the China National Convention Center in Beijing, on July 31st 2015.
Narratives are at the heart of information sharing. Ever since people began to share their experiences,
they have connected them to form narratives. The study od storytelling and the field of literary theory
called narratology have developed complex frameworks and models related to various aspects of
narrative such as plots structures, narrative embeddings, characters’ perspectives, reader response, point
of view, narrative voice, narrative goals, and many others. These notions from narratology have been
applied mainly in Artificial Intelligence and to model formal semantic approaches to narratives (e.g.
Plot Units developed by Lehnert (1981)). In recent years, computational narratology has qualified as an
autonomous field of study and research. Narrative has been the focus of a number of workshops and
conferences (AAAI Symposia, Interactive Storytelling Conference (ICIDS), Computational Models of
Narrative). Furthermore, reference annotation schemes for narratives have been proposed (NarrativeML
by Mani (2013)).
The workshop aimed at bringing together researchers from different communities working on
representing and extracting narrative structures in news, a text genre which is highly used in NLP
but which has received little attention with respect to narrative structure, representation and analysis.
Currently, advances in NLP technology have made it feasible to look beyond scenario-driven, atomic
extraction of events from single documents and work towards extracting story structures from multiple
documents, while these documents are published over time as news streams. Policy makers, NGOs,
information specialists (such as journalists and librarians) and others are increasingly in need of tools
that support them in finding salient stories in large amounts of information to more effectively implement
policies, monitor actions of “big players” in the society and check facts. Their tasks often revolve around
reconstructing cases either with respect to specific entities (e.g. person or organizations) or events (e.g.
hurricane Katrina). Storylines represent explanatory schemas that enable us to make better selections
of relevant information but also projections to the future. They form a valuable potential for exploiting
news data in an innovative way.JRC.G.2-Global security and crisis managemen
Entity-Oriented Search
This open access book covers all facets of entity-oriented search—where “search” can be interpreted in the broadest sense of information access—from a unified point of view, and provides a coherent and comprehensive overview of the state of the art. It represents the first synthesis of research in this broad and rapidly developing area. Selected topics are discussed in-depth, the goal being to establish fundamental techniques and methods as a basis for future research and development. Additional topics are treated at a survey level only, containing numerous pointers to the relevant literature. A roadmap for future research, based on open issues and challenges identified along the way, rounds out the book. The book is divided into three main parts, sandwiched between introductory and concluding chapters. The first two chapters introduce readers to the basic concepts, provide an overview of entity-oriented search tasks, and present the various types and sources of data that will be used throughout the book. Part I deals with the core task of entity ranking: given a textual query, possibly enriched with additional elements or structural hints, return a ranked list of entities. This core task is examined in a number of different variants, using both structured and unstructured data collections, and numerous query formulations. In turn, Part II is devoted to the role of entities in bridging unstructured and structured data. Part III explores how entities can enable search engines to understand the concepts, meaning, and intent behind the query that the user enters into the search box, and how they can provide rich and focused responses (as opposed to merely a list of documents)—a process known as semantic search. The final chapter concludes the book by discussing the limitations of current approaches, and suggesting directions for future research. Researchers and graduate students are the primary target audience of this book. A general background in information retrieval is sufficient to follow the material, including an understanding of basic probability and statistics concepts as well as a basic knowledge of machine learning concepts and supervised learning algorithms
Visual Text Analysis in Digital Humanities
In 2005, Franco Moretti introduced Distant Reading to analyse entire literary text collections. This was a rather revolutionary idea compared to the traditional Close Reading, which focuses on the thorough interpretation of an individual work. Both reading techniques are the prior means of Visual Text Analysis. We present an overview of the research conducted since 2005 on supporting text analysis tasks with close and distant reading visualizations in the digital humanities. Therefore, we classify the observed papers according to a taxonomy of text analysis tasks, categorize applied close and distant reading techniques to support the investigation of these tasks and illustrate approaches that combine both reading techniques in order to provide a multi-faceted view of the textual data. In addition, we take a look at the used text sources and at the typical data transformation steps required for the proposed visualizations. Finally, we summarize collaboration experiences when developing visualizations for close and distant reading, and we give an outlook on future challenges in that research area
Domain-sensitive topic management in a modular conversational agent framework
Flexible nontask-oriented conversational agents require content for generating responses and mechanisms that serve them for choosing appropriate topics to drive interactions with users. Structured knowledge resources such as ontologies are a useful mechanism to represent conversational topics. In order to develop the topic-management mechanism, we addressed a number of research issues related to the development of the required infrastructure. First, we address the issue of heavy human involvement in the construction of knowledge resources by proposing a four-stage automatic process for building domain-specific ontologies. These ontologies are comprised of a set of subtaxonomies obtained from WordNet, an electronic dictionary that arranges concepts in a hierarchical structure. The roots of these subtaxonomies are obtained from Wikipedia’s article links or wikilinks; this under the hypothesis that wikilinks provide a sense of relatedness from the article consulted to their destinations. With the knowledge structures defined, we explore the possibility of using semantic relatedness over these domain-specific ontologies as a mean to propose conversational topics in a coherent manner. For this, we examine different automatic measures of semantic relatedness to determine which correlates with human judgements obtained from an automatically constructed dataset. We then examine the question of whether domain information influences the human perception of semantic relatedness in a way that automatic measures do not replicate. This study requires us to design and implement a process to build datasets with pairs of concepts as those used in the literature to evaluate automatic measures of semantic relatedness, but with domain information associated. This study shows, to statistical significance, that existing measures of semantic relatedness do not take domain into consideration, and that including domain as a factor in this calculation can enhance the agreement of automatic measures with human assessments. Finally, this artificially constructed measure is integrated into the Toy’s dialogue manager, in order to help in the real-time selection of conversational topics. This supplements our result that the use of semantic relatedness seems to produce more coherent and interesting topic transitions than existing mechanisms