Automatic quantitative morphological analysis of interacting galaxies

The large number of galaxies imaged by digital sky surveys reinforces the need for computational methods for analyzing galaxy morphology. While the morphology of most galaxies can be associated with a stage on the Hubble sequence, morphology of galaxy mergers is far more complex due to the combination of two or more galaxies with different morphologies and the interaction between them. Here we propose a computational method based on unsupervised machine learning that can quantitatively analyze morphologies of galaxy mergers and associate galaxies by their morphology. The method works by first generating multiple synthetic galaxy models for each galaxy merger, and then extracting a large set of numerical image content descriptors for each galaxy model. These numbers are weighted using Fisher discriminant scores, and then the similarities between the galaxy mergers are deduced using a variation of Weighted Nearest Neighbor analysis such that the Fisher scores are used as weights. The similarities between the galaxy mergers are visualized using phylogenies to provide a graph that reflects the morphological similarities between the different galaxy mergers, and thus quantitatively profile the morphology of galaxy mergers.Comment: Astronomy & Computing, accepte

Visual Question Answering: A Survey of Methods and Datasets

Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities. Given an image and a question in natural language, it requires reasoning over visual elements of the image and general knowledge to infer the correct answer. In the first part of this survey, we examine the state of the art by comparing modern approaches to the problem. We classify methods by their mechanism to connect the visual and textual modalities. In particular, we examine the common approach of combining convolutional and recurrent neural networks to map images and questions to a common feature space. We also discuss memory-augmented and modular architectures that interface with structured knowledge bases. In the second part of this survey, we review the datasets available for training and evaluating VQA systems. The various datatsets contain questions at different levels of complexity, which require different capabilities and types of reasoning. We examine in depth the question/answer pairs from the Visual Genome project, and evaluate the relevance of the structured annotations of images with scene graphs for VQA. Finally, we discuss promising future directions for the field, in particular the connection to structured knowledge bases and the use of natural language processing models.Comment: 25 page

Logical models for automated semantics-directed program analysis

[EN] In Computer Science and Software Engineering, even at this time, when huge hardware and software resources are available, the problem of checking correctness of an specific piece of software is a very complicated one. Since the manual inspection of software is a difficult and error prone task, we propose as the main objective of this thesis the development of a tool which is able to generate logical models which can be used as a basis for semantics directed program analysis. To develop this tool, we rely on Order-Sorted First-Order Logic, which is the logic we use to define our programs and properties to be analyzed. We use this logic because it is sufficiently expressive to be used in the semantic description of most programming languages. Also, we use Convex Domain Interpretations as a flexible and computationally suitable basis for our derived models. We will also use polynomial interpretations and we will deal with conditional polynomial constraints, which are amenable for automating tests of properties written in order-sorted first-order logic. We have developed an automatic test tool, named AGES, which applies the aforementioned theoretical framework to implement the automated generation of models using convex domains and conditional polynomial constraints. The tool accepts Order-Sorted First-Order theories written in MAUDE, transforms them into a set of polynomial constraints, and then solves those constraints using an external SMT solver tool. The outcome of the constraint solver is used to assemble a logical model for the initial theory, which often can be used later to test other properties (termination, correctness, etc). The tool is written in Haskell to mutually exploit synergies between the functionalities provided by AGES and the functionalities provided by the mu-term tool.Reinoso Mendoza, EP. (2015). Logical models for automated semantics-directed program analysis. http://hdl.handle.net/10251/75034Archivo delegad

Exploiting Query Structure and Document Structure to Improve Document Retrieval Effectiveness

In this paper we present a systematic analysis of document retrieval using unstructured and structured queries within the score region algebra (SRA) structured retrieval framework. The behavior of di®erent retrieval models, namely Boolean, tf.idf, GPX, language models, and Okapi, is tested using the transparent SRA framework in our three-level structured retrieval system called TIJAH. The retrieval models are implemented along four elementary retrieval aspects: element and term selection, element score computation, score combination, and score propagation. The analysis is performed on a numerous experiments evaluated on TREC and CLEF collections, using manually generated unstructured and structured queries. Unstructured queries range from the short title queries to long title + description + narrative queries. For generating structured queries we exploit the knowledge of the document structure and the content used to semantically describe or classify documents. We show that such structured information can be utilized in retrieval engines to give more precise answers to user queries then when using unstructured queries

A Joint Speaker-Listener-Reinforcer Model for Referring Expressions

Referring expressions are natural language constructions used to identify particular objects within a scene. In this paper, we propose a unified framework for the tasks of referring expression comprehension and generation. Our model is composed of three modules: speaker, listener, and reinforcer. The speaker generates referring expressions, the listener comprehends referring expressions, and the reinforcer introduces a reward function to guide sampling of more discriminative expressions. The listener-speaker modules are trained jointly in an end-to-end learning framework, allowing the modules to be aware of one another during learning while also benefiting from the discriminative reinforcer's feedback. We demonstrate that this unified framework and training achieves state-of-the-art results for both comprehension and generation on three referring expression datasets. Project and demo page: https://vision.cs.unc.edu/referComment: Some typo fixed; comprehension results on refcocog updated; more human evaluation results adde