Harvesting Information from Captions for Weakly Supervised Semantic Segmentation

Since acquiring pixel-wise annotations for training convolutional neural networks for semantic image segmentation is time-consuming, weakly supervised approaches that only require class tags have been proposed. In this work, we propose another form of supervision, namely image captions as they can be found on the Internet. These captions have two advantages. They do not require additional curation as it is the case for the clean class tags used by current weakly supervised approaches and they provide textual context for the classes present in an image. To leverage such textual context, we deploy a multi-modal network that learns a joint embedding of the visual representation of the image and the textual representation of the caption. The network estimates text activation maps (TAMs) for class names as well as compound concepts, i.e. combinations of nouns and their attributes. The TAMs of compound concepts describing classes of interest substantially improve the quality of the estimated class activation maps which are then used to train a network for semantic segmentation. We evaluate our method on the COCO dataset where it achieves state of the art results for weakly supervised image segmentation

DBLPLink: An Entity Linker for the DBLP Scholarly Knowledge Graph

In this work, we present a web application named DBLPLink, which performs entity linking over the DBLP scholarly knowledge graph. DBLPLink uses text-to-text pre-trained language models, such as T5, to produce entity label spans from an input text question. Entity candidates are fetched from a database based on the labels, and an entity re-ranker sorts them based on entity embeddings, such as TransE, DistMult and ComplEx. The results are displayed so that users may compare and contrast the results between T5-small, T5-base and the different KG embeddings used. The demo can be accessed at https://ltdemos.informatik.uni-hamburg.de/dblplink/.Comment: Accepted at International Semantic Web Conference (ISWC) 2023 Posters & Demo Trac

DBLP-QuAD: A Question Answering Dataset over the DBLP Scholarly Knowledge Graph

In this work we create a question answering dataset over the DBLP scholarly knowledge graph (KG). DBLP is an on-line reference for bibliographic information on major computer science publications that indexes over 4.4 million publications published by more than 2.2 million authors. Our dataset consists of 10,000 question answer pairs with the corresponding SPARQL queries which can be executed over the DBLP KG to fetch the correct answer. DBLP-QuAD is the largest scholarly question answering dataset.Comment: 12 pages ceur-ws 1 column accepted at International Bibliometric Information Retrieval Workshp @ ECIR 202

GETT-QA: Graph Embedding based T2T Transformer for Knowledge Graph Question Answering

In this work, we present an end-to-end Knowledge Graph Question Answering (KGQA) system named GETT-QA. GETT-QA uses T5, a popular text-to-text pre-trained language model. The model takes a question in natural language as input and produces a simpler form of the intended SPARQL query. In the simpler form, the model does not directly produce entity and relation IDs. Instead, it produces corresponding entity and relation labels. The labels are grounded to KG entity and relation IDs in a subsequent step. To further improve the results, we instruct the model to produce a truncated version of the KG embedding for each entity. The truncated KG embedding enables a finer search for disambiguation purposes. We find that T5 is able to learn the truncated KG embeddings without any change of loss function, improving KGQA performance. As a result, we report strong results for LC-QuAD 2.0 and SimpleQuestions-Wikidata datasets on end-to-end KGQA over Wikidata.Comment: 16 pages single column format accepted at ESWC 2023 research trac

The Role of Output Vocabulary in T2T LMs for SPARQL Semantic Parsing

In this work, we analyse the role of output vocabulary for text-to-text (T2T) models on the task of SPARQL semantic parsing. We perform experiments within the the context of knowledge graph question answering (KGQA), where the task is to convert questions in natural language to the SPARQL query language. We observe that the query vocabulary is distinct from human vocabulary. Language Models (LMs) are pre-dominantly trained for human language tasks, and hence, if the query vocabulary is replaced with a vocabulary more attuned to the LM tokenizer, the performance of models may improve. We carry out carefully selected vocabulary substitutions on the queries and find absolute gains in the range of 17% on the GrailQA dataset.Comment: Accepted as a short paper to ACL 2023 finding

Side-Chain Polarity Modulates the Intrinsic Conformational Landscape of Model Dipeptides.

The intrinsic conformational preferences of small peptides may provide additional insight into the thermodynamics and kinetics of protein folding. In this study, we explore the underlying energy landscapes of two model peptides, namely, Ac-Ala-NH2 and Ac-Ser-NH2, using geometry-optimization-based tools developed within the context of energy landscape theory. We analyze not only how side-chain polarity influences the structural preferences of the dipeptides, but also other emergent properties of the landscape, including heat capacity profiles, and kinetics of conformational rearrangements. The contrasting topographies of the free energy landscape agree with recent results from Fourier transform microwave spectroscopy experiments, where Ac-Ala-NH2 was found to exist as a mixture of two conformers, while Ac-Ser-NH2 remained structurally locked, despite exhibiting an apparently rich conformational landscape.epsr

Exploring the Influence of Climate Change on Earthen Embankments with Expansive Soil

Climate change is known to cause alterations in weather patterns and disturb the natural equilibrium. Changes in climatic conditions lead to increased environmental stress on embankments, which can result in slope failures. Due to wetting–drying cycles, expansive clayey soil often swells and shrinks, and matric suction is a major factor that controls the behavior. Increased temperature accelerates soil evaporation and drying, which can cause desiccation cracks, while precipitation can rapidly reduce soil shear strength. Desiccated slopes on embankments built with such soils can cause surficial slope failures after intense precipitation. This study used slope stability analysis to quantify how climate-change-induced extreme weather affects embankments. Historic extreme climatic events were used as a baseline to estimate future extremes. CMIP6 provided historical and future climatic data for the study area. An embankment was numerically modeled to evaluate the effect on slope stability due to the precipitation change induced by climate change. Coupled hydro-mechanical finite element analyses used a two-dimensional transient unsaturated seepage model and a limit equilibrium slope stability model. The study found that extreme climatic interactions like precipitation and temperature due to climate change may reduce embankment slope safety. The reduction in the stability of the embankment due to increased precipitation resulting from different greenhouse gas emission scenarios was investigated. The use of unsaturated soil strength and variation of permeability with suction, along with the phase transition of these earthen embankments from near-dry to near-saturated, shows how unsaturated soil mechanics and the hydro-mechanical model can identify climate change issues on critical geotechnical infrastructure