Cross-modal correspondences in non-human mammal communication

For both humans and other animals, the ability to combine information obtained through different senses is fundamental to the perception of the environment. It is well established that humans form systematic cross-modal correspondences between stimulus features that can facilitate the accurate combination of sensory percepts. However, the evolutionary origins of the perceptual and cognitive mechanisms involved in these cross-modal associations remain surprisingly underexplored. In this review we outline recent comparative studies investigating how non-human mammals naturally combine information encoded in different sensory modalities during communication. The results of these behavioural studies demonstrate that various mammalian species are able to combine signals from different sensory channels when they are perceived to share the same basic features, either be- cause they can be redundantly sensed and/or because they are processed in the same way. Moreover, evidence that a wide range of mammals form complex cognitive representations about signallers, both within and across species, suggests that animals also learn to associate different sensory features which regularly co-occur. Further research is now necessary to determine how multisensory representations are formed in individual animals, including the relative importance of low level feature-related correspondences. Such investigations will generate important insights into how animals perceive and categorise their environment, as well as provide an essential basis for understanding the evolution of multisensory perception in humans

Visuospatial coding as ubiquitous scaffolding for human cognition

For more than 100 years we have known that the visual field is mapped onto the surface of visual cortex, imposing an inherently spatial reference frame on visual information processing. Recent studies highlight visuospatial coding not only throughout visual cortex, but also brain areas not typically considered visual. Such widespread access to visuospatial coding raises important questions about its role in wider cognitive functioning. Here, we synthesise these recent developments and propose that visuospatial coding scaffolds human cognition by providing a reference frame through which neural computations interface with environmental statistics and task demands via perception–action loops

Statistics of gradient directions in natural images.

Interest in finding statistical regularities in natural images has been growing since the advent of information theory and the advancement of the efficient coding hypothesis that the human visual system is optimised to encode natural visual stimuli. In this thesis, a statistical analysis of gradient directions in an ensemble of natural images is reported. Information-theoretic measures have been used to compute the amount of dependency which exists between triples of gradient directions at separate image locations. Control experiments are performed on other image classes: phase randomized natural images, whitened natural images, and Gaussian noise images. The main results show that for an ensemble of natural images the average amount of de pendency between two and three gradient directions is the same as for an ensemble of phase randomized natural images. This result does not extend to i) the amount dependency between gradient magnitudes, ii) gradient directions at high gradient magnitude locations, or iii) individual natural images. Furthermore, no significant synergetic dependencies are found between triples of gradient directions in an ensemble natural images a synergetic dependency is an increase in dependency between a pair of gradient directions given the interaction of a third gradient direction. Additional experiments are performed to establish both the generality and specificity of the main results by studying the gradient direction dependencies of ensembles of noise (random phases) images with varying power law power spectra. The results of the additional experiments indicate that, for ensembles of images with varying power law power spectra, the amount of dependency between two and three gradient directions is determined by the ensemble's mean power spectrum rather than the phase spectrum. A framework is also presented for future work and preliminary results are provided for the dependency between second order derivative measurements (shape index) for up to 9-point configurations

Understanding The Implications Of Neural Population Activity On Behavior

Learning how neural activity in the brain leads to the behavior we exhibit is one of the fundamental questions in Neuroscience. In this dissertation, several lines of work are presented to that use principles of neural coding to understand behavior. In one line of work, we formulate the efficient coding hypothesis in a non-traditional manner in order to test human perceptual sensitivity to complex visual textures. We find a striking agreement between how variable a particular texture signal is and how sensitive humans are to its presence. This reveals that the efficient coding hypothesis is still a guiding principle for neural organization beyond the sensory periphery, and that the nature of cortical constraints differs from the peripheral counterpart. In another line of work, we relate frequency discrimination acuity to neural responses from auditory cortex in mice. It has been previously observed that optogenetic manipulation of auditory cortex, in addition to changing neural responses, evokes changes in behavioral frequency discrimination. We are able to account for changes in frequency discrimination acuity on an individual basis by examining the Fisher information from the neural population with and without optogenetic manipulation. In the third line of work, we address the question of what a neural population should encode given that its inputs are responses from another group of neurons. Drawing inspiration from techniques in machine learning, we train Deep Belief Networks on fake retinal data and show the emergence of Garbor-like filters, reminiscent of responses in primary visual cortex. In the last line of work, we model the state of a cortical excitatory-inhibitory network during complex adaptive stimuli. Using a rate model with Wilson-Cowan dynamics, we demonstrate that simple non-linearities in the signal transferred from inhibitory to excitatory neurons can account for real neural recordings taken from auditory cortex. This work establishes and tests a variety of hypotheses that will be useful in helping to understand the relationship between neural activity and behavior as recorded neural populations continue to grow

A human visual system based image coder

Over the years, society has changed considerably due to technological changes, and digital images have become part and parcel of our everyday lives. Irrespective of applications (i.e., digital camera) and services (information sharing, e.g., Youtube, archive / storage), there is the need for high image quality with high compression ratios. Hence, considerable efforts have been invested in the area of image compression. The traditional image compression systems take into account of statistical redundancies inherent in the image data. However, the development and adaptation of vision models, which take into account the properties of the human visual system (HVS), into picture coders have since shown promising results. The objective of the thesis is to propose the implementation of a vision model in two different manners in the JPEG2000 coding system: (a) a Perceptual Colour Distortion Measure (PCDM) for colour images in the encoding stage, and (b) a Perceptual Post Filtering (PPF) algorithm for colour images in the decoding stage. Both implementations are embedded into the JPEG2000 coder. The vision model here exploits the contrast sensitivity, the inter-orientation masking and intra-band masking visual properties of the HVS. Extensive calibration work has been undertaken to fine-tune the 42 model parameters of the PCDM and Just-Noticeable-Difference thresholds of the PPF for colour images. Evaluation with subjective assessments of PCDM based coder has shown perceived quality improvement over the JPEG2000 benchmark with the MSE (mean square error) and CVIS criteria. For the PPF adapted JPEG2000 decoder, performance evaluation has also shown promising results against the JPEG2000 benchmarks. Based on subjective evaluation, when both PCDM and PPF are used in the JPEG2000 coding system, the overall perceived image quality is superior to the stand-alone JPEG2000 with the PCDM

Neuron-level dynamics of oscillatory network structure and markerless tracking of kinematics during grasping

Oscillatory synchrony is proposed to play an important role in flexible sensory-motor transformations. Thereby, it is assumed that changes in the oscillatory network structure at the level of single neurons lead to flexible information processing. Yet, how the oscillatory network structure at the neuron-level changes with different behavior remains elusive. To address this gap, we examined changes in the fronto-parietal oscillatory network structure at the neuron-level, while monkeys performed a flexible sensory-motor grasping task. We found that neurons formed separate subnetworks in the low frequency and beta bands. The beta subnetwork was active during steady states and the low frequency network during active states of the task, suggesting that both frequencies are mutually exclusive at the neuron-level. Furthermore, both frequency subnetworks reconfigured at the neuron-level for different grip and context conditions, which was mostly lost at any scale larger than neurons in the network. Our results, therefore, suggest that the oscillatory network structure at the neuron-level meets the necessary requirements for the coordination of flexible sensory-motor transformations. Supplementarily, tracking hand kinematics is a crucial experimental requirement to analyze neuronal control of grasp movements. To this end, a 3D markerless, gloveless hand tracking system was developed using computer vision and deep learning techniques. 2021-11-3

OpenApePose: a database of annotated ape photographs for pose estimation

Because of their close relationship with humans, non-human apes (chimpanzees, bonobos, gorillas, orangutans, and gibbons, including siamangs) are of great scientific interest. The goal of understanding their complex behavior would be greatly advanced by the ability to perform video-based pose tracking. Tracking, however, requires high-quality annotated datasets of ape photographs. Here we present OpenApePose, a new public dataset of 71,868 photographs, annotated with 16 body landmarks, of six ape species in naturalistic contexts. We show that a standard deep net (HRNet-W48) trained on ape photos can reliably track out-of-sample ape photos better than networks trained on monkeys (specifically, the OpenMonkeyPose dataset) and on humans (COCO) can. This trained network can track apes almost as well as the other networks can track their respective taxa, and models trained without one of the six ape species can track the held out species better than the monkey and human models can. Ultimately, the results of our analyses highlight the importance of large specialized databases for animal tracking systems and confirm the utility of our new ape database

How dogs hear us: perception of the human voice by domestic dogs (Canis familiaris)

Domestic dogs have co-habited with humans for at least 15000 years. Close social interaction between the two species has promoted inter-specific communication and dogs now show advanced skills in responding to human signals in comparison to wolves. However, research into dogs’ abilities to interpret human signals has predominantly focussed on visual gestures, while their responses to vocal signals remain under-investigated. Exploring the perception of human speech by dogs, a phylogenetically distant species, could provide new insights into the evolution of mammal communication. Therefore, the aim of this thesis is to assess human speech perception by dogs. Speech is composed of two main communicative components: the segmental phonemic cues carrying the linguistic content and the supra-segmental cues transmitting information about the speaker such as their gender, age and emotional state. I first explore how dogs perceive supra-segmental cues, determining that they are capable of the cross-modal discrimination of human gender. I then provide a review detailing the mechanisms underlying cross-modal associations in mammal communication, before testing which of these mechanisms may enable dogs to cross-modally associate cues to human age. The results indicate that dogs learn to match some voices to humans according to their age category, while also perceiving more general cross-modal correspondences in the environment. Finally, I investigate how dogs dissociate the main communicatory components of speech during processing, providing evidence that dogs differentially process segmental and supra-segmental cues according to their communicative content. In doing so, dogs appear to express parallel hemispheric biases to those reported in humans. Additionally, the results provide the first clear demonstration that dogs attend to the combinatory structure of the phonemic content in learnt commands. Overall, this thesis extends our knowledge of dogs’ perception of human signals, indicating that they are capable of perceiving each of the main components of speech in a functionally relevant manner. Together the results suggest that dogs share some of the cognitive and social processes involved in speech perception with human listeners