94 research outputs found
Similarity Reasoning over Semantic Context-Graphs
Similarity is a central cognitive mechanism for humans which enables a broad range of perceptual and abstraction processes, including recognizing and categorizing objects, drawing parallelism, and predicting outcomes. It has been studied computationally through models designed to replicate human judgment. The work presented in this dissertation leverages general purpose semantic networks to derive similarity measures in a problem-independent manner. We model both general and relational similarity using connectivity between concepts within semantic networks. Our first contribution is to model general similarity using concept connectivity, which we use to partition vocabularies into topics without the need of document corpora. We apply this model to derive topics from unstructured dialog, specifically enabling an early literacy primer application to support parents in having better conversations with their young children, as they are using the primer together. Second, we model relational similarity in proportional analogies. To do so, we derive relational parallelism by searching in semantic networks for similar path pairs that connect either side of this analogy statement. We then derive human readable explanations from the resulting similar path pair. We show that our model can answer broad-vocabulary analogy questions designed for human test takers with high confidence. The third contribution is to enable symbolic plan repair in robot planning through object substitution. When a failure occurs due to unforeseen changes in the environment, such as missing objects, we enable the planning domain to be extended with a number of alternative objects such that the plan can be repaired and execution to continue. To evaluate this type of similarity, we use both general and relational similarity. We demonstrate that the task context is essential in establishing which objects are interchangeable
Semantic interpretation of events in lifelogging
The topic of this thesis is lifelogging, the automatic, passive recording of a personâs daily activities and in particular, on performing a semantic analysis and enrichment of lifelogged data. Our work centers on visual lifelogged data, such as taken from wearable cameras. Such wearable cameras generate an archive of a personâs day taken from a first-person viewpoint but one of the problems with this is the sheer volume of information that can be generated. In order to make this potentially very large volume of information more manageable, our analysis of this data is based on segmenting each dayâs lifelog data into discrete and non-overlapping events corresponding to activities in the wearerâs day. To manage lifelog data at an event level, we define a set of concepts using an ontology which is appropriate to the wearer, applying automatic detection of concepts to these events and then semantically enriching each of the detected lifelog events making them an index into the events. Once this enrichment is complete we can use the lifelog to support semantic search for everyday media management, as a memory aid, or as part of medical analysis on the activities of daily living (ADL), and so on. In the thesis, we address the problem of how to select the concepts to be used for indexing events and we propose a semantic, density- based algorithm to cope with concept selection issues for lifelogging. We then apply activity detection to classify everyday activities by employing the selected concepts as high-level semantic features. Finally, the activity is modeled by multi-context representations and enriched by Semantic Web technologies. The thesis includes an experimental evaluation using real data from users and shows the performance of our algorithms in capturing the semantics of everyday concepts and their efficacy in activity recognition and semantic enrichment
Knowledge management framework based on brain models and human physiology
The life of humans and most living beings depend on sensation and perception for the best assessment of the surrounding world. Sensorial organs acquire a variety of stimuli that are interpreted and
integrated in our brain for immediate use or stored in memory for later recall. Among the reasoning aspects, a person has to decide what to do with available information. Emotions are classifiers of collected information, assigning a personal meaning to objects, events and individuals, making part of our own identity. Emotions play a decisive role in cognitive processes as reasoning, decision and memory by assigning relevance to collected information.
The access to pervasive computing devices, empowered by the ability to sense and perceive the world, provides new forms of acquiring and integrating information. But prior to data assessment on its usefulness, systems must capture and ensure that data is properly managed for diverse possible goals.
Portable and wearable devices are now able to gather and store information, from the environment and from our body, using cloud based services and Internet connections.
Systems limitations in handling sensorial data, compared with our sensorial capabilities constitute an identified problem. Another problem is the lack of interoperability between humans and devices, as they do not properly understand humanâs emotional states and human needs. Addressing those problems is a motivation for the present research work.
The mission hereby assumed is to include sensorial and physiological data into a Framework that will be able to manage collected data towards human cognitive functions, supported by a new data model.
By learning from selected human functional and behavioural models and reasoning over collected data, the Framework aims at providing evaluation on a personâs emotional state, for empowering human centric applications, along with the capability of storing episodic information on a personâs life with physiologic indicators on emotional states to be used by new generation applications
Towards Multi-modal Interpretation and Explanation
Multimodal task processes on different modalities simultaneously. Visual Question Answering, as a type of multimodal task, aims to answer the natural question answering based on the given image. To understand and process the image, many models to solve the visual question answering task encode the object regions through the convolutional neural network based backbones. Such an image processing method captures the visual features of the object regions in the image. However, the relations between objects are also important information to comprehensively understand the image for answering the complex question, and whether such relational information is captured by the visual features of the object regions remains opaque. To explicitly extract such relational information in images for visual question answering tasks, this research explores an interpretable and structural graph representation to encode the relations between objects. This research works on the three variants of Visual Question Answering tasks with different types of images, including photo-realistic images, daily scene pictures and document pages. Different task-specific relational graphs have been used and proposed to explicitly capture and encode the relations to be used by the proposed models. Such a relational graph provides an interpretable representation of the model inputs and proves its effectiveness in improving the model performance in output prediction. In addition, to improve the interpretation of the modelâs prediction, this research also explores the suitable local interpretation method to be applied to the VQA model
Linked Data Supported Information Retrieval
Um Inhalte im World Wide Web ausfindig zu machen, sind Suchmaschienen nicht mehr wegzudenken. Semantic Web und Linked Data Technologien ermöglichen ein detaillierteres und eindeutiges Strukturieren der Inhalte und erlauben vollkommen neue Herangehensweisen an die Lösung von Information Retrieval Problemen. Diese Arbeit befasst sich mit den Möglichkeiten, wie Information Retrieval Anwendungen von der Einbeziehung von Linked Data profitieren können. Neue Methoden der computer-gestĂŒtzten semantischen Textanalyse, semantischen Suche, Informationspriorisierung und -visualisierung werden vorgestellt und umfassend evaluiert. Dabei werden Linked Data Ressourcen und ihre Beziehungen in die Verfahren integriert, um eine Steigerung der EffektivitĂ€t der Verfahren bzw. ihrer Benutzerfreundlichkeit zu erzielen. ZunĂ€chst wird eine EinfĂŒhrung in die Grundlagen des Information Retrieval und Linked Data gegeben. AnschlieĂend werden neue manuelle und automatisierte Verfahren zum semantischen Annotieren von Dokumenten durch deren VerknĂŒpfung mit Linked Data Ressourcen vorgestellt (Entity Linking). Eine umfassende Evaluation der Verfahren wird durchgefĂŒhrt und das zu Grunde liegende Evaluationssystem umfangreich verbessert. Aufbauend auf den Annotationsverfahren werden zwei neue Retrievalmodelle zur semantischen Suche vorgestellt und evaluiert. Die Verfahren basieren auf dem generalisierten Vektorraummodell und beziehen die semantische Ăhnlichkeit anhand von taxonomie-basierten Beziehungen der Linked Data Ressourcen in Dokumenten und Suchanfragen in die Berechnung der Suchergebnisrangfolge ein. Mit dem Ziel die Berechnung von semantischer Ăhnlichkeit weiter zu verfeinern, wird ein Verfahren zur Priorisierung von Linked Data Ressourcen vorgestellt und evaluiert. Darauf aufbauend werden Visualisierungstechniken aufgezeigt mit dem Ziel, die Explorierbarkeit und Navigierbarkeit innerhalb eines semantisch annotierten Dokumentenkorpus zu verbessern. HierfĂŒr werden zwei Anwendungen prĂ€sentiert. Zum einen eine Linked Data basierte explorative Erweiterung als ErgĂ€nzung zu einer traditionellen schlĂŒsselwort-basierten Suchmaschine, zum anderen ein Linked Data basiertes Empfehlungssystem
Surveillance Graphs: Vulgarity and Cloud Orthodoxy in Linked Data Infrastructures
Information is power, and that power has been largely enclosed by a handful of information conglomerates. The logic of the surveillance-driven information economy demands systems for handling mass quantities of heterogeneous data, increasingly in the form of knowledge graphs. An archaeology of knowledge graphs and their mutation from the liberatory aspirations of the semantic web gives us an underexplored lens to understand contemporary information systems. I explore how the ideology of cloud systems steers two projects from the NIH and NSF intended to build information infrastructures for the public good to inevitable corporate capture, facilitating the development of a new kind of multilayered public/private surveillance system in the process. I argue that understanding technologies like large language models as interfaces to knowledge graphs is critical to understand their role in a larger project of informational enclosure and concentration of power. I draw from multiple histories of liberatory information technologies to develop Vulgar Linked Data as an alternative to the Cloud Orthodoxy, resisting the colonial urge for universality in favor of vernacular expression in peer to peer systems
Semantically en enhanced information retrieval: an ontology-based aprroach
Tesis doctoral inédita. Universidad Autónoma de Madrid, Escuela Politécnica Superior, enero de 2009Bibliogr.: [227]-240 p
Bayesian models of category acquisition and meaning development
The ability to organize concepts (e.g., dog, chair) into efficient mental representations,
i.e., categories (e.g., animal, furniture) is a fundamental mechanism which allows humans
to perceive, organize, and adapt to their world. Much research has been dedicated
to the questions of how categories emerge and how they are represented. Experimental
evidence suggests that (i) concepts and categories are represented through sets of
features (e.g., dogs bark, chairs are made of wood) which are structured into different
types (e.g, behavior, material); (ii) categories and their featural representations are
learnt jointly and incrementally; and (iii) categories are dynamic and their representations
adapt to changing environments.
This thesis investigates the mechanisms underlying the incremental and dynamic formation
of categories and their featural representations through cognitively motivated
Bayesian computational models. Models of category acquisition have been extensively
studied in cognitive science and primarily tested on perceptual abstractions or artificial
stimuli. In this thesis, we focus on categories acquired from natural language stimuli,
using nouns as a stand-in for their reference concepts, and their linguistic contexts as
a representation of the conceptsâ features. The use of text corpora allows us to (i) develop
large-scale unsupervised models thus simulating human learning, and (ii) model
child category acquisition, leveraging the linguistic input available to children in the
form of transcribed child-directed language.
In the first part of this thesis we investigate the incremental process of category acquisition.
We present a Bayesian model and an incremental learning algorithm which
sequentially integrates newly observed data. We evaluate our model output against
gold standard categories (elicited experimentally from human participants), and show
that high-quality categories are learnt both from child-directed data and from large,
thematically unrestricted text corpora. We find that the model performs well even under
constrained memory resources, resembling human cognitive limitations. While
lists of representative features for categories emerge from this model, they are neither
structured nor jointly optimized with the categories.
We address these shortcomings in the second part of the thesis, and present a Bayesian
model which jointly learns categories and structured featural representations. We
present both batch and incremental learning algorithms, and demonstrate the modelâs
effectiveness on both encyclopedic and child-directed data. We show that high-quality
categories and features emerge in the joint learning process, and that the structured
features are intuitively interpretable through human plausibility judgment evaluation.
In the third part of the thesis we turn to the dynamic nature of meaning: categories and
their featural representations change over time, e.g., children distinguish some types
of features (such as size and shade) less clearly than adults, and word meanings adapt
to our ever changing environment and its structure. We present a dynamic Bayesian
model of meaning change, which infers time-specific concept representations as a set
of feature types and their prevalence, and captures their development as a smooth process.
We analyze the development of concept representations in their complexity over
time from child-directed data, and show that our model captures established patterns of
child concept learning. We also apply our model to diachronic change of word meaning,
modeling how word senses change internally and in prevalence over centuries.
The contributions of this thesis are threefold. Firstly, we show that a variety of experimental
results on the acquisition and representation of categories can be captured
with computational models within the framework of Bayesian modeling. Secondly,
we show that natural language text is an appropriate source of information for modeling
categorization-related phenomena suggesting that the environmental structure that
drives category formation is encoded in this data. Thirdly, we show that the experimental
findings hold on a larger scale. Our models are trained and tested on a larger
set of concepts and categories than is common in behavioral experiments and the categories
and featural representations they can learn from linguistic text are in principle
unrestricted
- âŠ