Humans and deep networks largely agree on which kinds of variation make object recognition harder

View-invariant object recognition is a challenging problem, which has attracted much attention among the psychology, neuroscience, and computer vision communities. Humans are notoriously good at it, even if some variations are presumably more difficult to handle than others (e.g. 3D rotations). Humans are thought to solve the problem through hierarchical processing along the ventral stream, which progressively extracts more and more invariant visual features. This feed-forward architecture has inspired a new generation of bio-inspired computer vision systems called deep convolutional neural networks (DCNN), which are currently the best algorithms for object recognition in natural images. Here, for the first time, we systematically compared human feed-forward vision and DCNNs at view-invariant object recognition using the same images and controlling for both the kinds of transformation as well as their magnitude. We used four object categories and images were rendered from 3D computer models. In total, 89 human subjects participated in 10 experiments in which they had to discriminate between two or four categories after rapid presentation with backward masking. We also tested two recent DCNNs on the same tasks. We found that humans and DCNNs largely agreed on the relative difficulties of each kind of variation: rotation in depth is by far the hardest transformation to handle, followed by scale, then rotation in plane, and finally position. This suggests that humans recognize objects mainly through 2D template matching, rather than by constructing 3D object models, and that DCNNs are not too unreasonable models of human feed-forward vision. Also, our results show that the variation levels in rotation in depth and scale strongly modulate both humans' and DCNNs' recognition performances. We thus argue that these variations should be controlled in the image datasets used in vision research

Understanding motor control in humans to improve rehabilitation robots

Recent reviews highlighted the limited results of robotic rehabilitation and the low quality of evidences in this field. Despite the worldwide presence of several robotic infrastructures, there is still a lack of knowledge about the capabilities of robotic training effect on the neural control of movement. To fill this gap, a step back to motor neuroscience is needed: the understanding how the brain works in the generation of movements, how it adapts to changes and how it acquires new motor skills is fundamental. This is the rationale behind my PhD project and the contents of this thesis: all the studies included in fact examined changes in motor control due to different destabilizing conditions, ranging from external perturbations, to self-generated disturbances, to pathological conditions. Data on healthy and impaired adults have been collected and quantitative and objective information about kinematics, dynamics, performance and learning were obtained for the investigation of motor control and skill learning. Results on subjects with cervical dystonia show how important assessment is: possibly adequate treatments are missing because the physiological and pathological mechanisms underlying sensorimotor control are not routinely addressed in clinical practice. These results showed how sensory function is crucial for motor control. The relevance of proprioception in motor control and learning is evident also in a second study. This study, performed on healthy subjects, showed that stiffness control is associated with worse robustness to external perturbations and worse learning, which can be attributed to the lower sensitiveness while moving or co-activating. On the other hand, we found that the combination of higher reliance on proprioception with \u201cdisturbance training\u201d is able to lead to a better learning and better robustness. This is in line with recent findings showing that variability may facilitate learning and thus can be exploited for sensorimotor recovery. Based on these results, in a third study, we asked participants to use the more robust and efficient strategy in order to investigate the control policies used to reject disturbances. We found that control is non-linear and we associated this non-linearity with intermittent control. As the name says, intermittent control is characterized by open loop intervals, in which movements are not actively controlled. We exploited the intermittent control paradigm for other two modeling studies. In these studies we have shown how robust is this model, evaluating it in two complex situations, the coordination of two joints for postural balance and the coordination of two different balancing tasks. It is an intriguing issue, to be addressed in future studies, to consider how learning affects intermittency and how this can be exploited to enhance learning or recovery. The approach, that can exploit the results of this thesis, is the computational neurorehabilitation, which mathematically models the mechanisms underlying the rehabilitation process, with the aim of optimizing the individual treatment of patients. Integrating models of sensorimotor control during robotic neurorehabilitation, might lead to robots that are fully adaptable to the level of impairment of the patient and able to change their behavior accordingly to the patient\u2019s intention. This is one of the goals for the development of rehabilitation robotics and in particular of Wristbot, our robot for wrist rehabilitation: combining proper assessment and training protocols, based on motor control paradigms, will maximize robotic rehabilitation effects

Integrated Structural And Functional Biomarkers For Neurodegeneration

Alzheimer\u27s Disease consists of a complex cascade of pathological processes, leading to the death of cortical neurons and development of dementia. Because it is impossible to regenerate neurons that have already died, a thorough understanding of the earlier stages of the disease, before significant neuronal death has occurred, is critical for developing disease-modifying therapies. The various components of Alzheimer\u27s Disease pathophysiology necessitate a variety of measurement techniques. Image-based measurements known as biomarkers can be used to assess cortical thinning and cerebral blood flow, but non-imaging characteristics such as performance on cognitive tests and age are also important determinants of risk of Alzheimer\u27s Disease. Incorporating the various imaging and non-imaging sources of information into a scientifically interpretable and statistically sound model is challenging. In this thesis, I present a method to include imaging data in standard regression analyses in a data-driven and anatomically interpretable manner. I also introduce a technique for disentangling the effect of cortical structure from blood flow, enabling a clearer picture of the signal carried by cerebral blood flow beyond the confounding effects of anatomical structure. In addition to these technical developments in multi-modal image analysis, I show the results of two clinically-oriented studies focusing on the relative importance of various biomarkers for predicting presence of Alzheimer\u27s Disease pathology in the earliest stages of disease. In the first, I present evidence that white matter hyperintensities, a marker of small vessel disease, are more highly associated with Alzheimer\u27s Disease pathology than current mainstream imaging biomarkers in elderly control patients. In the second, I show that once Alzheimer\u27s Disease has progressed to the point of noticeable cognitive decline, cognitive tests are as predictive of presence of Alzheimer\u27s pathology as standard imaging biomarkers. Taken together, these studies demonstrate that the relative importance of biomarkers and imaging modalities changes over the course of disease progression, and sophisticated data-driven methods for combining a variety of modalities is likely to lead to greater biological insight into the disease process than a single modality

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Mechanics of the Developing Brain: From Smooth-walled Tube to the Folded Cortex

Over the course of human development, the brain undergoes dramatic physical changes to achieve its final, convoluted shape. However, the forces underlying every cinch, bulge, and fold remain poorly understood. This doctoral research focuses on the mechanical processes responsible for early (embryonic) and late (preterm) brain development. First, we examine early brain development in the chicken embryo, which is similar to human at these stages. Research has primarily focused on molecular signals to describe morphogenesis, but mechanical analysis can also provide important insights. Using a combination of experiments and finite element modeling, we find that actomyosin contraction is responsible for initial segmentation of the forebrain. By considering mechanical forces from the internal and external environment, we propose a role for mechanical feedback in maintaining these segments during subsequent inflation and bending. Next, we extend our analysis to division of right and left cerebral hemispheres. In this case, we discover that morphogen signals and mechanical feedback act synergistically to shape the hemispheres. In human, cerebral hemispheres go on to form complex folds through a mechanical process that involves rapid expansion of the cortical surface. However, the spatiotemporal dynamics of cortical growth remain unknown in human. Here, we develop a novel strain energy minimization approach to measure regional growth in complex surfaces. By considering brain surfaces of preterm subjects, reconstructed from magnetic resonance imaging (MRI), this analysis reveals distinct patterns of cortical growth that evolve over the third trimester. This information provides a comprehensive view of cortical growth and folding, connecting what is known about patterns of development at the cellular and folding scales. Abnormal brain morphogenesis can lead to serious structural defects and neurological disorders such as epilepsy and autism. By integrating mechanics, biology, and neuroimaging, we gain a more complete understanding of brain development. By studying physical changes from the simple, microscopic embryo to the macroscopic, folded cortex, we gain insight into relevant biological and physical mechanisms across developmental stages

Shape analysis of the human brain.

Autism is a complex developmental disability that has dramatically increased in prevalence, having a decisive impact on the health and behavior of children. Methods used to detect and recommend therapies have been much debated in the medical community because of the subjective nature of diagnosing autism. In order to provide an alternative method for understanding autism, the current work has developed a 3-dimensional state-of-the-art shape based analysis of the human brain to aid in creating more accurate diagnostic assessments and guided risk analyses for individuals with neurological conditions, such as autism. Methods: The aim of this work was to assess whether the shape of the human brain can be used as a reliable source of information for determining whether an individual will be diagnosed with autism. The study was conducted using multi-center databases of magnetic resonance images of the human brain. The subjects in the databases were analyzed using a series of algorithms consisting of bias correction, skull stripping, multi-label brain segmentation, 3-dimensional mesh construction, spherical harmonic decomposition, registration, and classification. The software algorithms were developed as an original contribution of this dissertation in collaboration with the BioImaging Laboratory at the University of Louisville Speed School of Engineering. The classification of each subject was used to construct diagnoses and therapeutic risk assessments for each patient. Results: A reliable metric for making neurological diagnoses and constructing therapeutic risk assessment for individuals has been identified. The metric was explored in populations of individuals having autism spectrum disorders, dyslexia, Alzheimers disease, and lung cancer. Conclusion: Currently, the clinical applicability and benefits of the proposed software approach are being discussed by the broader community of doctors, therapists, and parents for use in improving current methods by which autism spectrum disorders are diagnosed and understood

Genetic determination and layout rules of visual cortical architecture

The functional architecture of the primary visual cortex is set up by neurons that preferentially respond to visual stimuli with contours of a specific orientation in visual space. In primates and placental carnivores, orientation preference is arranged into continuous and roughly repetitive (iso-) orientation domains. Exceptions are pinwheels that are surrounded by all orientation preferences. The configuration of pinwheels adheres to quantitative species-invariant statistics, the common design. This common design most likely evolved independently at least twice in the course of the past 65 million years, which might indicate a functionally advantageous trait. The possible acquisition of environment-dependent functional traits by genes, the Baldwin effect, makes it conceivable that visual cortical architecture is partially or redundantly encoded by genetic information. In this conception, genetic mechanisms support the emergence of visual cortical architecture or even establish it under unfavorable environments. In this dissertation, I examine the capability of genetic mechanisms for encoding visual cortical architecture and mathematically dissect the pinwheel configuration under measurement noise as well as in different geometries. First, I theoretically explore possible roles of genetic mechanisms in visual cortical development that were previously excluded from theoretical research, mostly because the information capacity of the genome appeared too small to contain a blueprint for wiring up the cortex. For the first time, I provide a biologically plausible scheme for quantitatively encoding functional visual cortical architecture by genetic information that circumvents the alleged information bottleneck. Key ingredients for this mechanism are active transport and trans-neuronal signaling as well as joined dynamics of morphogens and connectome. This theory provides predictions for experimental tests and thus may help to clarify the relative importance of genes and environments on complex human traits. Second, I disentangle the link between orientation domain ensembles and the species-invariant pinwheel statistics of the common design. This examination highlights informative measures of pinwheel configurations for model benchmarking. Third, I mathematically investigate the susceptibility of the pinwheel configuration to measurement noise. The results give rise to an extrapolation method of pinwheel densities to the zero noise limit and provide an approximated analytical expression for confidence regions of pinwheel centers. Thus, the work facilitates high-precision measurements and enhances benchmarking for devising more accurate models of visual cortical development. Finally, I shed light on genuine three-dimensional properties of functional visual cortical architectures. I devise maximum entropy models of three-dimensional functional visual cortical architectures in different geometries. This theory enables the examination of possible evolutionary transitions between different functional architectures for which intermediate organizations might still exist

Accuracy of rats in discriminating visual objects Is explained by the complexity of their perceptual strategy

Despite their growing popularity as models of visual functions, it is widely assumed that rodents deploy perceptual strategies not nearly as advanced as those of primates, when processing visual objects. Such belief is fostered by the conflicting findings about the complexity of rodent pattern vision, which appears to range from mere detection of overall object luminance to view-invariant processing of discriminant shape features. Here, we sought to clarify how refined object vision is in rodents, by measuring how well a group of rats discriminated a reference object from eleven distractors, spanning a spectrum of image-level similarity with the reference. We also presented the animals with random variations of the reference, and we processed their responses to these stimuli to obtain subject-specific models of rat perceptual choices. These models captured very well the highly variable discrimination performance observed across subjects and object conditions. In particular, they revealed how the animals that succeeded with the more challenging distractors were those that integrated the wider variety of discriminant features into their perceptual strategy. Critically, these features remained highly subject-specific and largely invariant under changes in object appearance (e.g., size variation), although they were properly reformatted (e.g., rescaled) to deal with the specific transformations the objects underwent. Overall, these findings show that rat object vision, far from being poorly developed, can be characterized as a feature-based filtering process (iterated across multiple scales, positions, etc.), similar to the one that is at work in primates and state-of-the-art machine vision systems, such as convolutional neural networks

Object Recognition

Vision-based object recognition tasks are very familiar in our everyday activities, such as driving our car in the correct lane. We do these tasks effortlessly in real-time. In the last decades, with the advancement of computer technology, researchers and application developers are trying to mimic the human's capability of visually recognising. Such capability will allow machine to free human from boring or dangerous jobs

Reference Frames in Human Sensory, Motor, and Cognitive Processing

Reference-frames, or coordinate systems, are used to express properties and relationships of objects in the environment. While the use of reference-frames is well understood in physical sciences, how the brain uses reference-frames remains a fundamental question. The goal of this dissertation is to reach a better understanding of reference-frames in human perceptual, motor, and cognitive processing. In the first project, we study reference-frames in perception and develop a model to explain the transition from egocentric (based on the observer) to exocentric (based outside the observer) reference-frames to account for the perception of relative motion. In a second project, we focus on motor behavior, more specifically on goal-directed reaching. We develop a model that explains how egocentric perceptual and motor reference-frames can be coordinated through exocentric reference-frames. Finally, in a third project, we study how the cognitive system can store and recognize objects by using sensorimotor schema that allows mental rotation within an exocentric reference-frame