132,771 research outputs found
Adaptive image retrieval using a graph model for semantic feature integration
The variety of features available to represent multimedia data constitutes a rich pool of information. However, the plethora of data poses a challenge in terms of feature selection and integration for effective retrieval. Moreover, to further improve effectiveness, the
retrieval model should ideally incorporate context-dependent feature representations to allow for retrieval on a higher semantic level. In this paper we present a retrieval model and learning framework for the purpose of interactive information retrieval. We describe
how semantic relations between multimedia objects based on user interaction can be learnt and then integrated with visual and textual features into a unified framework. The framework models both feature similarities and semantic relations in a single graph. Querying in this model is implemented using the theory of random walks. In addition, we present ideas to implement short-term learning from relevance feedback. Systematic experimental results validate the effectiveness of the proposed approach for image retrieval. However, the model is not restricted to the image domain and could easily be employed for retrieving multimedia data (and even a combination of different domains, eg images, audio and text documents)
Boosting Text-to-Image Diffusion Models with Fine-Grained Semantic Rewards
Recent advances in text-to-image diffusion models have achieved remarkable
success in generating high-quality, realistic images from given text prompts.
However, previous methods fail to perform accurate modality alignment between
text concepts and generated images due to the lack of fine-level semantic
guidance that successfully diagnoses the modality discrepancy. In this paper,
we propose FineRewards to improve the alignment between text and images in
text-to-image diffusion models by introducing two new fine-grained semantic
rewards: the caption reward and the Semantic Segment Anything (SAM) reward.
From the global semantic view, the caption reward generates a corresponding
detailed caption that depicts all important contents in the synthetic image via
a BLIP-2 model and then calculates the reward score by measuring the similarity
between the generated caption and the given prompt. From the local semantic
view, the SAM reward segments the generated images into local parts with
category labels, and scores the segmented parts by measuring the likelihood of
each category appearing in the prompted scene via a large language model, i.e.,
Vicuna-7B. Additionally, we adopt an assemble reward-ranked learning strategy
to enable the integration of multiple reward functions to jointly guide the
model training. Adapting results of text-to-image models on the MS-COCO
benchmark show that the proposed semantic reward outperforms other baseline
reward functions with a considerable margin on both visual quality and semantic
similarity with the input prompt. Moreover, by adopting the assemble
reward-ranked learning strategy, we further demonstrate that model performance
is further improved when adapting under the unifying of the proposed semantic
reward with the current image rewards
Performing PowerPoint lectures: examining the extent of slide-text integration into lecturersâ spoken expositions
The PowerPoint assisted lecture (slide-lecture) is a common lecturing approach in Higher Education, in spite of much criticism of its use. Its popularity is facilitated by its affordances for multimodal instructional design, e.g. text with images and speech. Little is known about the integration of different semiotic modalities within the instructional communication practices of slide-lectures, nor the learning conditions that they create. Given that text bulletpoints are ubiquitous in slide-lectures, and may impose linearity into instructional communications (Kinchin et al., 2008), this study explores the extent to which lecturing speech is systematically coordinated with slide-text. Eleven slide-lectures given in psychology departments across the UK were recorded and transcribed. Patterns of semantic matches between speech and slide-text were analysed to produce similarity scores for each lecturer. Lectures were scored using an integration scoring system of 0-1, with 1 indicating a perfect match of speech and slide-text. There was significant departure from a systematic voicing of the slide text (i.e. reading off the slides). Two characteristic speech-slide relationship styles were identified. The âreferentâ style is one in which the slide is an object of reference for the lecturer to comment on, and the âscaffoldingâ style is one in which the slide text is blended into the spoken narrative. Consequences of the lecturerâs coordination with presentational slides are discussed in terms of the learning environment it might produce. It is suggested that whichever relationship a lecturer has with their slide-text, students might benefit from the integration being consistent
Protein interaction sentence detection using multiple semantic kernels
<p>Abstract</p> <p>Background</p> <p>Detection of sentences that describe protein-protein interactions (PPIs) in biomedical publications is a challenging and unresolved pattern recognition problem. Many state-of-the-art approaches for this task employ kernel classification methods, in particular support vector machines (SVMs). In this work we propose a novel data integration approach that utilises semantic kernels and a kernel classification method that is a probabilistic analogue to SVMs. Semantic kernels are created from statistical information gathered from large amounts of unlabelled text using lexical semantic models. Several semantic kernels are then fused into an overall composite classification space. In this initial study, we use simple features in order to examine whether the use of combinations of kernels constructed using word-based semantic models can improve PPI sentence detection.</p> <p>Results</p> <p>We show that combinations of semantic kernels lead to statistically significant improvements in recognition rates and receiver operating characteristic (ROC) scores over the plain Gaussian kernel, when applied to a well-known labelled collection of abstracts. The proposed kernel composition method also allows us to automatically infer the most discriminative kernels.</p> <p>Conclusions</p> <p>The results from this paper indicate that using semantic information from unlabelled text, and combinations of such information, can be valuable for classification of short texts such as PPI sentences. This study, however, is only a first step in evaluation of semantic kernels and probabilistic multiple kernel learning in the context of PPI detection. The method described herein is modular, and can be applied with a variety of feature types, kernels, and semantic models, in order to facilitate full extraction of interacting proteins.</p
Boosting the Coverage of a Semantic Lexicon by Automatically Extracted Event Nominalizations
International audienceAn important trend in recent works on lexical semantics has been the development of learning methods capable of extracting semantic information from text corpora. The majority of these methods are based on the distributional hypothesis of meaning and acquire semantic information by identifying distributional patterns in texts. In this article, we present a distributional analysis method for extracting nominalization relations from monolingual corpora. The acquisition method makes use of distributional and morphological information to select nominalization candidates. We explain how the learning is performed on a dependency annotated corpus and describe the nominalization results. Furthermore, we show how these results served to enrich an existing lexical resource, the WOLF (Wordnet Libre du Français). We present the techniques that we developed in order to integrate the new information into WOLF, based on both its structure and content. Finally, we evaluate the validity of the automatically obtained information and the correctness of its integration into the semantic resource. The method proved to be useful for boosting the coverage of WOLF and presents the advantage of filling verbal synsets, which are particularly difficult to handle due to the high level of verbal polysemy
Recommended from our members
A linked data-driven & service-oriented architecture for sharing educational resources
The two fundamental aims of managing educational resources are to enable resources to be reusable and interoperable and to enable Web-scale sharing of resources across learning communities. Currently, a variety of approaches have been proposed to expose and manage educational resources and their metadata on the Web. These are usually based on heterogeneous metadata standards and schemas, such as IEEE LOM or ADL SCORM, and diverse repository interfaces such as OAI-PMH or SQI. Also, there is still a lack of usage of controlled vocabularies and available data sets that could replace the widespread use of unstructured text for describing resources. On the other hand, the Linked Data approach has proven that it offers a set of successful principles that have the potential to alleviate the aforementioned issues. In this paper, we introduce an architecture and prototype which is fundamentally based on (a) Linked Data principles and (b) Service-orientation to resolve the integration issues for sharing educational resources
Political Text Scaling Meets Computational Semantics
During the last fifteen years, automatic text scaling has become one of the
key tools of the Text as Data community in political science. Prominent text
scaling algorithms, however, rely on the assumption that latent positions can
be captured just by leveraging the information about word frequencies in
documents under study. We challenge this traditional view and present a new,
semantically aware text scaling algorithm, SemScale, which combines recent
developments in the area of computational linguistics with unsupervised
graph-based clustering. We conduct an extensive quantitative analysis over a
collection of speeches from the European Parliament in five different languages
and from two different legislative terms, and show that a scaling approach
relying on semantic document representations is often better at capturing known
underlying political dimensions than the established frequency-based (i.e.,
symbolic) scaling method. We further validate our findings through a series of
experiments focused on text preprocessing and feature selection, document
representation, scaling of party manifestos, and a supervised extension of our
algorithm. To catalyze further research on this new branch of text scaling
methods, we release a Python implementation of SemScale with all included data
sets and evaluation procedures.Comment: Updated version - accepted for Transactions on Data Science (TDS
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning
Over the past years, distributed semantic representations have proved to be
effective and flexible keepers of prior knowledge to be integrated into
downstream applications. This survey focuses on the representation of meaning.
We start from the theoretical background behind word vector space models and
highlight one of their major limitations: the meaning conflation deficiency,
which arises from representing a word with all its possible meanings as a
single vector. Then, we explain how this deficiency can be addressed through a
transition from the word level to the more fine-grained level of word senses
(in its broader acceptation) as a method for modelling unambiguous lexical
meaning. We present a comprehensive overview of the wide range of techniques in
the two main branches of sense representation, i.e., unsupervised and
knowledge-based. Finally, this survey covers the main evaluation procedures and
applications for this type of representation, and provides an analysis of four
of its important aspects: interpretability, sense granularity, adaptability to
different domains and compositionality.Comment: 46 pages, 8 figures. Published in Journal of Artificial Intelligence
Researc
Linked education: interlinking educational resources and the web of data
Research on interoperability of technology-enhanced learning (TEL) repositories throughout the last decade has led to a fragmented landscape of competing approaches, such as metadata schemas and interface mechanisms. However, so far Web-scale integration of resources is not facilitated, mainly due to the lack of take-up of shared principles, datasets and schemas. On the other hand, the Linked Data approach has emerged as the de-facto standard for sharing data on the Web and offers a large potential to solve interoperability issues in the field of TEL. In this paper, we describe a general approach to exploit the wealth of already existing TEL data on the Web by allowing its exposure as Linked Data and by taking into account automated enrichment and interlinking techniques to provide rich and well-interlinked data for the educational domain. This approach has been implemented in the context of the mEducator project where data from a number of open TEL data repositories has been integrated, exposed and enriched by following Linked Data principles
- âŠ