Vision systems with the human in the loop

The emerging cognitive vision paradigm deals with vision systems that apply machine learning and automatic reasoning in order to learn from what they perceive. Cognitive vision systems can rate the relevance and consistency of newly acquired knowledge, they can adapt to their environment and thus will exhibit high robustness. This contribution presents vision systems that aim at flexibility and robustness. One is tailored for content-based image retrieval, the others are cognitive vision systems that constitute prototypes of visual active memories which evaluate, gather, and integrate contextual knowledge for visual analysis. All three systems are designed to interact with human users. After we will have discussed adaptive content-based image retrieval and object and action recognition in an office environment, the issue of assessing cognitive systems will be raised. Experiences from psychologically evaluated human-machine interactions will be reported and the promising potential of psychologically-based usability experiments will be stressed

Efficient Yet Deep Convolutional Neural Networks for Semantic Segmentation

Semantic Segmentation using deep convolutional neural network pose more complex challenge for any GPU intensive task. As it has to compute million of parameters, it results to huge memory consumption. Moreover, extracting finer features and conducting supervised training tends to increase the complexity. With the introduction of Fully Convolutional Neural Network, which uses finer strides and utilizes deconvolutional layers for upsampling, it has been a go to for any image segmentation task. In this paper, we propose two segmentation architecture which not only needs one-third the parameters to compute but also gives better accuracy than the similar architectures. The model weights were transferred from the popular neural net like VGG19 and VGG16 which were trained on Imagenet classification data-set. Then we transform all the fully connected layers to convolutional layers and use dilated convolution for decreasing the parameters. Lastly, we add finer strides and attach four skip architectures which are element-wise summed with the deconvolutional layers in steps. We train and test on different sparse and fine data-sets like Pascal VOC2012, Pascal-Context and NYUDv2 and show how better our model performs in this tasks. On the other hand our model has a faster inference time and consumes less memory for training and testing on NVIDIA Pascal GPUs, making it more efficient and less memory consuming architecture for pixel-wise segmentation.Comment: 8 page

Assessing Vividness of Mental Imagery: The Plymouth Sensory Imagery Questionnaire

Publisher allows archiving of submitted msMental imagery may occur in any sensory modality, although visual imagery has been most studied. A sensitive measure of the vividness of imagery across a range of modalities is needed: the shorter version of Bett’s QMI (Sheehan, 1967) uses outdated items and has an unreliable factor structure. We report the development and initial validation of the Plymouth Sensory Imagery Questionnaire (Psi-Q) comprising items for each of the following modalities: Vision, Sound, Smell, Taste, Touch, Bodily Sensation and Emotional Feeling. An Exploratory Factor Analysis on a 35-item form indicated that these modalities formed separate factors, rather than a single imagery factor, and this was replicated by confirmatory factor analysis. The Psi-Q was validated against the Spontaneous Use of Imagery Scale (Reisberg, Pearson & Kosslyn, 2003) and Marks’ (1995) Vividness of Visual Imagery Questionnaire-2. A short 21-item form comprising the best three items from the seven factors correlated with the total score and subscales of the full form, and with the VVIQ-2. Inspection of the data shows that while visual and sound imagery is most often rated as vivid, individuals who rate one modality as strong and the other as weak are not uncommon. Findings are interpreted within a working memory framework and point to the need for further research to identify the specific cognitive processes underlying the vividness of imagery across sensory modalities

Objects predict fixations better than early saliency

Humans move their eyes while looking at scenes and pictures. Eye movements correlate with shifts in attention and are thought to be a consequence of optimal resource allocation for high-level tasks such as visual recognition. Models of attention, such as “saliency maps,” are often built on the assumption that “early” features (color, contrast, orientation, motion, and so forth) drive attention directly. We explore an alternative hypothesis: Observers attend to “interesting” objects. To test this hypothesis, we measure the eye position of human observers while they inspect photographs of common natural scenes. Our observers perform different tasks: artistic evaluation, analysis of content, and search. Immediately after each presentation, our observers are asked to name objects they saw. Weighted with recall frequency, these objects predict fixations in individual images better than early saliency, irrespective of task. Also, saliency combined with object positions predicts which objects are frequently named. This suggests that early saliency has only an indirect effect on attention, acting through recognized objects. Consequently, rather than treating attention as mere preprocessing step for object recognition, models of both need to be integrated

Balancing the demands of two tasks: an investigation of cognitive–motor dual-tasking in relapsing remitting multiple sclerosis

Background: People with relapsing remitting multiple sclerosis (PwRRMS) suffer disproportionate decrements in gait under dual-task conditions, when walking and a cognitive task are combined. There has been much less investigation of the impact of cognitive demands on balance. Objectives: This study investigated whether: (1) PwRRMS show disproportionate decrements in postural stability under dual-task conditions compared to healthy controls, and (2) dual-task decrements are associated with everyday dual-tasking difficulties. The impact of mood, fatigue, and disease severity on dual-tasking was also examined. Methods: A total of 34 PwRRMS and 34 matched controls completed cognitive (digit span) and balance (movement of center of pressure on Biosway on stable and unstable surfaces) tasks under single- and dual-task conditions. Everyday dual-tasking was measured using the Dual-Tasking Questionnaire. Mood was measured by the Hospital Anxiety & Depression Scale. Fatigue was measured via the Modified Fatigue Index Scale. Results: No differences in age, gender, years of education, estimated pre-morbid IQ, or baseline digit span between groups. Compared with controls, PwRRMS showed significantly greater decrement in postural stability under dual-task conditions on an unstable surface (p=.007), but not a stable surface (p=.679). Balance decrement scores were not correlated with everyday dual-tasking difficulties or fatigue. Stable surface balance decrement scores were significantly associated with levels of anxiety (rho=0.527; p=.001) and depression (rho=0.451; p=.007). Conclusions: RRMS causes dual-tasking difficulties, impacting balance under challenging conditions, which may contribute to increased risk of gait difficulties and falls. The relationship between anxiety/depression and dual-task decrement suggests that emotional factors may be contributing to dual-task difficulties

A multifactorial approach for understanding fall risk in older people

OBJECTIVE: To identify the interrelationships and discriminatory value of a broad range of objectively measured explanatory risk factors for falls. DESIGN: Prospective cohort study with 12-month follow-up period. SETTING: Community sample. PARTICIPANTS: Five hundred community-dwelling people aged 70 to 90. MEASUREMENTS: All participants underwent assessments on medical, disability, physical, cognitive, and psychological measures. Fallers were defined as people who had at least one injurious fall or at least two noninjurious falls during a 12-month follow-up period. RESULTS: Univariate regression analyses identified the following fall risk factors: disability, poor performance on physical tests, depressive symptoms, poor executive function, concern about falling, and previous falls. Classification and regression tree analysis revealed that balance-related impairments were critical predictors of falls. In those with good balance, disability and exercise levels influenced future fall risk-people in the lowest and the highest exercise tertiles were at greater risk. In those with impaired balance, different risk factors predicted greater fall risk-poor executive function, poor dynamic balance, and low exercise levels. Absolute risks for falls ranged from 11% in those with no risk factors to 54% in the highest-risk group. CONCLUSIONS: A classification and regression tree approach highlighted interrelationships and discriminatory value of important explanatory fall risk factors. The information may prove useful in clinical settings to assist in tailoring interventions to maximize the potential benefit of falls prevention strategies

Spatial judgment in Parkinson's disease: Contributions of attentional and executive dysfunction

Spatial judgment is impaired in Parkinson's disease (PD), with previous research suggesting that disruptions in attention and executive function are likely contributors. If judgment of center places demands on frontal systems, performance on tests of attention/executive function may correlate with extent of bias in PD, and attentional disturbance may predict inconsistency in spatial judgment. The relation of spatial judgment to attention/executive function may differ for those with left-side versus right-side motor onset (LPD, RPD), reflecting effects of attentional lateralization. We assessed 42 RPD, 37 LPD, and 67 healthy control participants with a Landmark task (LM) in which a cursor moved horizontally from the right (right-LM) or left (left-LM). The task was to judge the center of the line. Participants also performed neuropsychological tests of attention and executive function. LM group differences were found on left-LM only, with both PD subgroups biased leftward of the control group (RPD p < .05; LPD p < .01; no RPD-LPD difference). For left-LM trials, extent of bias significantly correlated with performance on the cognitive tasks for PD but not for the control group. PD showed greater variability in perceived center than the control group; this variability correlated with performance on the cognitive tasks. The correlations between performance on the test of spatial judgment and the tests of attention/executive function suggest that frontal-based attentional dysfunction affects dynamic spatial judgment, both in extent of spatial bias and in consistency of response as indexed by intertrial variability. (PsycINFO Database Record (c) 2019 APA, all rights reserved).R01 NS067128 - NINDS NIH HHS; R21 NS043730 - NINDS NIH HHS; National Institute of Neurological Disorders and Stroke; American Parkinson's Disease Association; Massachusetts ChapterAccepted manuscrip

Deep Neural Networks for Anatomical Brain Segmentation

We present a novel approach to automatically segment magnetic resonance (MR) images of the human brain into anatomical regions. Our methodology is based on a deep artificial neural network that assigns each voxel in an MR image of the brain to its corresponding anatomical region. The inputs of the network capture information at different scales around the voxel of interest: 3D and orthogonal 2D intensity patches capture the local spatial context while large, compressed 2D orthogonal patches and distances to the regional centroids enforce global spatial consistency. Contrary to commonly used segmentation methods, our technique does not require any non-linear registration of the MR images. To benchmark our model, we used the dataset provided for the MICCAI 2012 challenge on multi-atlas labelling, which consists of 35 manually segmented MR images of the brain. We obtained competitive results (mean dice coefficient 0.725, error rate 0.163) showing the potential of our approach. To our knowledge, our technique is the first to tackle the anatomical segmentation of the whole brain using deep neural networks