Visual Question Answering: A Survey of Methods and Datasets

Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities. Given an image and a question in natural language, it requires reasoning over visual elements of the image and general knowledge to infer the correct answer. In the first part of this survey, we examine the state of the art by comparing modern approaches to the problem. We classify methods by their mechanism to connect the visual and textual modalities. In particular, we examine the common approach of combining convolutional and recurrent neural networks to map images and questions to a common feature space. We also discuss memory-augmented and modular architectures that interface with structured knowledge bases. In the second part of this survey, we review the datasets available for training and evaluating VQA systems. The various datatsets contain questions at different levels of complexity, which require different capabilities and types of reasoning. We examine in depth the question/answer pairs from the Visual Genome project, and evaluate the relevance of the structured annotations of images with scene graphs for VQA. Finally, we discuss promising future directions for the field, in particular the connection to structured knowledge bases and the use of natural language processing models.Comment: 25 page

Predicting cell types and genetic variations contributing to disease by combining GWAS and epigenetic data

Genome-wide association studies (GWASs) identify single nucleotide polymorphisms (SNPs) that are enriched in individuals suffering from a given disease. Most disease-associated SNPs fall into non-coding regions, so that it is not straightforward to infer phenotype or function; moreover, many SNPs are in tight genetic linkage, so that a SNP identified as associated with a particular disease may not itself be causal, but rather signify the presence of a linked SNP that is functionally relevant to disease pathogenesis. Here, we present an analysis method that takes advantage of the recent rapid accumulation of epigenomics data to address these problems for some SNPs. Using asthma as a prototypic example; we show that non-coding disease-associated SNPs are enriched in genomic regions that function as regulators of transcription, such as enhancers and promoters. Identifying enhancers based on the presence of the histone modification marks such as H3K4me1 in different cell types, we show that the location of enhancers is highly cell-type specific. We use these findings to predict which SNPs are likely to be directly contributing to disease based on their presence in regulatory regions, and in which cell types their effect is expected to be detectable. Moreover, we can also predict which cell types contribute to a disease based on overlap of the disease-associated SNPs with the locations of enhancers present in a given cell type. Finally, we suggest that it will be possible to re-analyze GWAS studies with much higher power by limiting the SNPs considered to those in coding or regulatory regions of cell types relevant to a given disease

Learning, Arts, and the Brain: The Dana Consortium Report on Arts and Cognition

Reports findings from multiple neuroscientific studies on the impact of arts training on the enhancement of other cognitive capacities, such as reading acquisition, sequence learning, geometrical reasoning, and memory

The Machine Conception of the Organism in Development and Evolution: A Critical Analysis

This article critically examines one of the most prevalent metaphors in modern biology, namely the machine conception of the organism (MCO). Although the fundamental differences between organisms and machines make the MCO an inadequate metaphor for conceptualizing living systems, many biologists and philosophers continue to draw upon the MCO or tacitly accept it as the standard model of the organism. This paper analyses the specific difficulties that arise when the MCO is invoked in the study of development and evolution. In developmental biology the MCO underlies a logically incoherent model of ontogeny, the genetic program, which serves to legitimate three problematic theses about development: genetic animism, neo-preformationism, and developmental computability. In evolutionary biology the MCO is responsible for grounding unwarranted theoretical appeals to the concept of design as well as to the interpretation of natural selection as an engineer, which promote a distorted understanding of the process and products of evolutionary change. Overall, it is argued that, despite its heuristic value, the MCO today is impeding rather than enabling further progress in our comprehension of living systems

Ontology-based knowledge representation of experiment metadata in biological data mining

According to the PubMed resource from the U.S. National Library of Medicine, over 750,000 scientific articles have been published in the ~5000 biomedical journals worldwide in the year 2007 alone. The vast majority of these publications include results from hypothesis-driven experimentation in overlapping biomedical research domains. Unfortunately, the sheer volume of information being generated by the biomedical research enterprise has made it virtually impossible for investigators to stay aware of the latest findings in their domain of interest, let alone to be able to assimilate and mine data from related investigations for purposes of meta-analysis. While computers have the potential for assisting investigators in the extraction, management and analysis of these data, information contained in the traditional journal publication is still largely unstructured, free-text descriptions of study design, experimental application and results interpretation, making it difficult for computers to gain access to the content of what is being conveyed without significant manual intervention. In order to circumvent these roadblocks and make the most of the output from the biomedical research enterprise, a variety of related standards in knowledge representation are being developed, proposed and adopted in the biomedical community. In this chapter, we will explore the current status of efforts to develop minimum information standards for the representation of a biomedical experiment, ontologies composed of shared vocabularies assembled into subsumption hierarchical structures, and extensible relational data models that link the information components together in a machine-readable and human-useable framework for data mining purposes

Computational modeling to elucidate molecular mechanisms of epigenetic memory

How do mammalian cells that share the same genome exist in notably distinct phenotypes, exhibiting differences in morphology, gene expression patterns, and epigenetic chromatin statuses? Furthermore how do cells of different phenotypes differentiate reproducibly from a single fertilized egg? These are fundamental problems in developmental biology. Epigenetic histone modifications play an important role in the maintenance of different cell phenotypes. The exact molecular mechanism for inheritance of the modification patterns over cell generations remains elusive. The complexity comes partly from the number of molecular species and the broad time scales involved. In recent years mathematical modeling has made significant contributions on elucidating the molecular mechanisms of DNA methylation and histone covalent modification inheritance. We will pedagogically introduce the typical procedure and some technical details of performing a mathematical modeling study, and discuss future developments.Comment: 36 pages, 4 figures, 2 tables, book chapte

Genome-wide screening for DNA variants associated with reading and language traits

This research was funded by: Max Planck Society, the University of St Andrews - Grant Number: 018696, US National Institutes of Health - Grant Number: P50 HD027802, Wellcome Trust - Grant Number: 090532/Z/09/Z, and Medical Research Council Hub Grant Grant Number: G0900747 91070Reading and language abilities are heritable traits that are likely to share some genetic influences with each other. To identify pleiotropic genetic variants affecting these traits, we first performed a genome‐wide association scan (GWAS) meta‐analysis using three richly characterized datasets comprising individuals with histories of reading or language problems, and their siblings. GWAS was performed in a total of 1862 participants using the first principal component computed from several quantitative measures of reading‐ and language‐related abilities, both before and after adjustment for performance IQ. We identified novel suggestive associations at the SNPs rs59197085 and rs5995177 (uncorrected P ≈ 10–7 for each SNP), located respectively at the CCDC136/FLNC and RBFOX2 genes. Each of these SNPs then showed evidence for effects across multiple reading and language traits in univariate association testing against the individual traits. FLNC encodes a structural protein involved in cytoskeleton remodelling, while RBFOX2 is an important regulator of alternative splicing in neurons. The CCDC136/FLNC locus showed association with a comparable reading/language measure in an independent sample of 6434 participants from the general population, although involving distinct alleles of the associated SNP. Our datasets will form an important part of on‐going international efforts to identify genes contributing to reading and language skills.Publisher PDFPeer reviewe