Visual arguments and meta-arguments

Visual arguments—arguments that appeal to visual elements essentially—are legitimate arguments. To show this, I first consider what I call (perfect) fit arguments—arguments in which the recognition that items fit together suggests that they were once conjoined, perhaps originally. This form of argumentation is a type of abduction or inference to the best explanation (IBE). I then consider mathematical visual meta-arguments—arguments in which the validity or soundness of a mathematical argument is confirmed or refuted by appeal to diagrams

A Formal Approach for Modeling Interactive Visual Interfaces

We provide a mathematical model that supports the formal description of visual interfaces’ behaviour. The formalism is based on type-inference notation, in which each variable is defined in the domain of the interface basic widgets, and each transition from a given state of the interface to the following one is represented by the application of an inference rule. When a sequence of actions is made by the user, the behaviour of the corresponding interface is totally defined by the set of inference rules. This formalism allows the designer to formally verify the properties of the interface

Characterization of response properties and connectivity in mouse visual thalamus and cortex

How neuronal activity is shaped by circuit connectivity between neuronal populations is a central question in visual neuroscience. Combined with experimental data, computational models allow causal investigation and prediction of both how connectivity influences activity and how activity constrains connectivity. In order to develop and refine these computational models of the visual system, thorough characterization of neuronal response patterns is required. In this thesis, I first present an approach to infer connectivity from in vivo stimulus responses in mouse visual cortex, revealing underlying principles of connectivity between excitatory and inhibitory neurons. Second, I investigate suppressed-by-contrast neurons, which, while known since the 1960s, still remain to be included in standard models of visual function. I present a characterization of intrinsic firing properties and stimulus responses that expands the knowledge about this obscure neuron type. Inferring the neuronal connectome from neural activity is a major objective of computational connectomics. Complementary to direct experimental investigation of connectivity, inference approaches combine simultaneous activity data of individual neurons with methods ranging from statistical considerations of similarity to large-scale simulations of neuronal networks. However, due to the mathematically ill-defined nature of inferring connectivity from in vivo activity, most approaches have to constrain the inference procedure using experimental findings that are not part of the neural activity data set at hand. Combining the stabilized-supralinear network model with response data from the visual thalamus and cortex of mice, my collaborators and I have found a way to infer connectivity from in vivo data alone. Leveraging a property of neural responses known as contrast-invariance of orientation tuning, our inference approach reveals a consistent order of connection strengths between cortical neuron populations as well as tuning differences between thalamic inputs and cortex. Throughout the history of visual neuroscience, neurons that respond to a visual stimulus with an increase in firing have been at the center of attention. A different response type that decreases its activity in response to visual stimuli, however, has been only sparsely investigated. Consequently, these suppressed-by-contrast neurons, while recently receiving renewed attention from researchers, have not been characterized in depth. Together with my collaborators, I have conducted a survey of SbC properties covering firing reliability, cortical location, and tuning to stimulus orientation. We find SbC neurons to fire less regularly than expected, be located in the lower parts of cortex, and show significant tuning to oriented gratings

Scene Graph Generation by Iterative Message Passing

Understanding a visual scene goes beyond recognizing individual objects in isolation. Relationships between objects also constitute rich semantic information about the scene. In this work, we explicitly model the objects and their relationships using scene graphs, a visually-grounded graphical structure of an image. We propose a novel end-to-end model that generates such structured scene representation from an input image. The model solves the scene graph inference problem using standard RNNs and learns to iteratively improves its predictions via message passing. Our joint inference model can take advantage of contextual cues to make better predictions on objects and their relationships. The experiments show that our model significantly outperforms previous methods for generating scene graphs using Visual Genome dataset and inferring support relations with NYU Depth v2 dataset.Comment: CVPR 201