Visualizing Natural Image Statistics

Natural image statistics is an important area of research in cognitive sciences and computer vision. Visualization of statistical results can help identify clusters and anomalies as well as analyze deviation, distribution and correlation. Furthermore, they can provide visual abstractions and symbolism for categorized data. In this paper, we begin our study of visualization of image statistics by considering visual representations of power spectra, which are commonly used to visualize different categories of images. We show that they convey a limited amount of statistical information about image categories and their support for analytical tasks is ineffective. We then introduce several new visual representations, which convey different or more information about image statistics. We apply ANOVA to the image statistics to help select statistically more meaningful measurements in our design process. A task-based user evaluation was carried out to compare the new visual representations with the conventional power spectra plots. Based on the results of the evaluation, we made further improvement of visualizations by introducing composite visual representations of image statistics

The statistics of contour fragments in natural scenes

Summary: Recent electrophysiology recordings in macaque V4/IT suggest that single neuron response to synthetic closed contours can be largely captured by models which only consider a small number of contour fragments (Brincat and Connor 2004). Motivated by this experimental work, we sought firstly to characterize the statistics of contour fragments in natural scenes, and secondly to generate synthetic images which reflect the measured contour-fragment statistics.

To detect contour fragments, we defined a set of feature detectors which respond only in the presence of two edges co-occurring at a fixed relative angle – implemented as a logical ‘AND’ of two Gabor-like, laplacian-of-gaussian linear filters. We then determined the pairwise correlations of these contour fragments in a natural image ensemble. If efficient coding extends to higher cortical centers and processing in the ventral visual stream can be modeled as a sequence of logical operations on linear shape features, then the pairwise statistics we measure should be informative about neural shape coding. 

Using these statistics directly, it is possible to produce a generative model of simple images which contain the measured statistics. We implemented a modified Ising model and solved the inverse problem of determining the optimal model parameters which satisfy the measured correlations. The resulting Ising-like model of the pairwise statistics can generate the probability of any arrangement of contour fragments as measured in the natural image ensemble. 

As a complementary approach to producing images with naturalistic contour fragment statistics, it is possible to start with a natural scene and isolate the target features. This is achieved by applying our contour fragment detection processing to the single scene and then separately visualizing the fragments detected. This second procedure lends itself to parametric randomization of the generated image.

Narrative Elaboration: The central question guiding our study is how shapes are represented in inferotemporal cortex. To that end, we have investigated natural images in order to motivate experiments capable of targeting the extent to which neural processing of shapes involves representing shapes as combinations of key contour features. To simplify, we are focusing on black-and-white images and prioritizing contour features. This project suggests it is possible to generate synthetic images containing only a select set of contour statistics. Our subsequent goals include conducting collaborative macaque electrophysiology experiments with our generated images as visual stimuli

Superheat: An R package for creating beautiful and extendable heatmaps for visualizing complex data

The technological advancements of the modern era have enabled the collection of huge amounts of data in science and beyond. Extracting useful information from such massive datasets is an ongoing challenge as traditional data visualization tools typically do not scale well in high-dimensional settings. An existing visualization technique that is particularly well suited to visualizing large datasets is the heatmap. Although heatmaps are extremely popular in fields such as bioinformatics for visualizing large gene expression datasets, they remain a severely underutilized visualization tool in modern data analysis. In this paper we introduce superheat, a new R package that provides an extremely flexible and customizable platform for visualizing large datasets using extendable heatmaps. Superheat enhances the traditional heatmap by providing a platform to visualize a wide range of data types simultaneously, adding to the heatmap a response variable as a scatterplot, model results as boxplots, correlation information as barplots, text information, and more. Superheat allows the user to explore their data to greater depths and to take advantage of the heterogeneity present in the data to inform analysis decisions. The goal of this paper is two-fold: (1) to demonstrate the potential of the heatmap as a default visualization method for a wide range of data types using reproducible examples, and (2) to highlight the customizability and ease of implementation of the superheat package in R for creating beautiful and extendable heatmaps. The capabilities and fundamental applicability of the superheat package will be explored via three case studies, each based on publicly available data sources and accompanied by a file outlining the step-by-step analytic pipeline (with code).Comment: 26 pages, 10 figure