Optimizing Perceptual Quality Prediction Models for Multimedia Processing Systems

L'abstract è presente nell'allegato / the abstract is in the attachmen

Reverse-engineering the cortical architecture for controlled semantic cognition.

We employ a reverse-engineering approach to illuminate the neurocomputational building blocks that combine to support controlled semantic cognition: the storage and context-appropriate use of conceptual knowledge. By systematically varying the structure of a computational model and assessing the functional consequences, we identified the architectural properties that best promote some core functions of the semantic system. Semantic cognition presents a challenging test case, as the brain must achieve two seemingly contradictory functions: abstracting context-invariant conceptual representations across time and modalities, while producing specific context-sensitive behaviours appropriate for the immediate task. These functions were best achieved in models possessing a single, deep multimodal hub with sparse connections from modality-specific regions, and control systems acting on peripheral rather than deep network layers. The reverse-engineered model provides a unifying account of core findings in the cognitive neuroscience of controlled semantic cognition, including evidence from anatomy, neuropsychology and functional brain imaging

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Visual Quality Assessment and Blur Detection Based on the Transform of Gradient Magnitudes

abstract: Digital imaging and image processing technologies have revolutionized the way in which we capture, store, receive, view, utilize, and share images. In image-based applications, through different processing stages (e.g., acquisition, compression, and transmission), images are subjected to different types of distortions which degrade their visual quality. Image Quality Assessment (IQA) attempts to use computational models to automatically evaluate and estimate the image quality in accordance with subjective evaluations. Moreover, with the fast development of computer vision techniques, it is important in practice to extract and understand the information contained in blurred images or regions. The work in this dissertation focuses on reduced-reference visual quality assessment of images and textures, as well as perceptual-based spatially-varying blur detection. A training-free low-cost Reduced-Reference IQA (RRIQA) method is proposed. The proposed method requires a very small number of reduced-reference (RR) features. Extensive experiments performed on different benchmark databases demonstrate that the proposed RRIQA method, delivers highly competitive performance as compared with the state-of-the-art RRIQA models for both natural and texture images. In the context of texture, the effect of texture granularity on the quality of synthesized textures is studied. Moreover, two RR objective visual quality assessment methods that quantify the perceived quality of synthesized textures are proposed. Performance evaluations on two synthesized texture databases demonstrate that the proposed RR metrics outperforms full-reference (FR), no-reference (NR), and RR state-of-the-art quality metrics in predicting the perceived visual quality of the synthesized textures. Last but not least, an effective approach to address the spatially-varying blur detection problem from a single image without requiring any knowledge about the blur type, level, or camera settings is proposed. The evaluations of the proposed approach on a diverse sets of blurry images with different blur types, levels, and content demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods qualitatively and quantitatively.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

Independent circuits in basal ganglia and cortex for the processing of reward and precision feedback

In order to understand human decision making it is necessary to understand how the brain uses feedback to guide goal-directed behavior. The ventral striatum (VS) appears to be a key structure in this function, responding strongly to explicit reward feedback. However, recent results have also shown striatal activity following correct task performance even in the absence of feedback. This raises the possibility that, in addition to processing external feedback, the dopamine-centered reward circuit might regulate endogenous reinforcement signals, like those triggered by satisfaction in accurate task performance. Here we use functional magnetic resonance imaging (fMRI) to test this idea. Participants completed a simple task that garnered both reward feedback and feedback about the precision of performance. Importantly, the design was such that we could manipulate information about the precision of performance within different levels of reward magnitude. Using parametric modulation and functional connectivity analysis we identified brain regions sensitive to each of these signals. Our results show a double dissociation: frontal and posterior cingulate regions responded to explicit reward but were insensitive to task precision, whereas the dorsal striatum - and putamen in particular - was insensitive to reward but responded strongly to precision feedback in reward-present trials. Both types of feedback activated the VS, and sensitivity in this structure to precision feedback was predicted by personality traits related to approach behavior and reward responsiveness. Our findings shed new light on the role of specific brain regions in integrating different sources of feedback to guide goal-directed behavior

The role of instructions and intention in learning

This thesis investigates how manipulating intention to learn (learning orientation) through verbal instructions affects learning in a range of putatively associative and implicit tasks. Within three different paradigms, learning orientation was manipulated so that learning was either incidental to, or aligned with (i.e. intentional) the aims of the task. The first series of experiments investigated sequence learning, as measured in the serial reaction time task. Sequence learning was found to result reliably under incidental conditions and was selectively improved by instructions promoting discovery of a relational rule describing a set of probabilistic contingencies. The second series of experiments used the prototype distortion task, where it has been claimed that implicit learning of a category of prototype-centered stimuli can occur automatically as a result of exposure. Using a visual search task as a means of incidental exposure, equivocal evidence for the implicit status of learning in the prototype distortion task was found, and instructions directing participants to memorize the stimuli resulted in greater evidence of learning the similarity structure of the category. Finally, the third series of experiments assessed generalization along stimulus dimensions following a difficult discrimination task. Instructions directing attention to a particular stimulus dimension promoted rule-based generalization and facilitated a dissociation in the pattern of generalization obtained as a result of reducing rule applicability on test. The results suggest that human learning is highly susceptible to learning orientation, which has implications for the way implicit learning should be viewed as a psychological construct. Theories of learning, whether single- or dual-process, need to better account for this seemingly pervasive role of learning orientation

Multimodal Sensory Integration for Perception and Action in High Functioning Children with Autism Spectrum Disorder

Movement disorders are the earliest observed features of autism spectrum disorder (ASD) present in infancy. Yet we do not understand the neural basis for impaired goal-directed movements in this population. To reach for an object, it is necessary to perceive the state of the arm and the object using multiple sensory modalities (e.g. vision, proprioception), to integrate those sensations into a motor plan, to execute the plan, and to update the plan based on the sensory consequences of action. In this dissertation, I present three studies in which I recorded hand paths of children with ASD and typically developing (TD) controls as they grasped the handle of a robotic device to control a cursor displayed on a video screen. First, participants performed discrete and continuous movements to capture targets. Cursor feedback was perturbed from the hand\u27s actual position to introduce visuo-spatial conflict between sensory and proprioceptive feedback. Relative to controls, children with ASD made greater errors, consistent with deficits of sensorimotor adaptive and strategic compensations. Second, participants performed a two-interval forced-choice discrimination task in which they perceived two movements of the visual cursor and/or the robot handle and then indicated which of the two movements was more curved. Children with ASD were impaired in their ability to discriminate movement kinematics when provided visual and proprioceptive information simultaneously, suggesting deficits of visuo-proprioceptive integration. Finally, participants made goal-directed reaching movements against a load while undergoing simultaneous functional magnetic resonance imaging (MRI). The load remained constant (predictable) within an initial block of trials and then varied randomly within four additional blocks. Children with ASD exhibited greater movement variability compared to controls during both constant and randomly-varying loads. MRI analysis identified marked differences in the extent and intensity of the neural activities supporting goal-directed reaching in children with ASD compared to TD children in both environmental conditions. Taken together, the three studies revealed deficits of multimodal sensory integration in children with ASD during perception and execution of goal-directed movements and ASD-related motor performance deficits have a telltale neural signature, as revealed by functional MR imaging

Visibility metrics and their applications in visually lossless image compression

Visibility metrics are image metrics that predict the probability that a human observer can detect differences between a pair of images. These metrics can provide localized information in the form of visibility maps, in which each value represents a probability of detection. An important application of the visibility metric is visually lossless image compression that aims at compressing a given image to the lowest fraction of bit per pixel while keeping the compression artifacts invisible at the same time. In previous works, most visibility metrics were modeled based on largely simplified assumptions and mathematical models of human visual systems. This approach generally fits well into experimental data measured with simple stimuli, such as Gabor patches. However, it cannot predict complex non-linear effects, such as contrast masking in natural images, particularly well. To predict visibility of image differences accurately, we collected the largest visibility dataset under fixed viewing conditions for calibrating existing visibility metrics and proposed a deep neural network-based visibility metric. We demonstrated in our experiments that the deep neural network-based visibility metric significantly outperformed existing visibility metrics. However, the deep neural network-based visibility metric cannot predict visibility under varying viewing conditions, such as display brightness and viewing distances that have great impacts on the visibility of distortions. To extend the deep neural network-based visibility metric to varying viewing conditions, we collected the largest visibility dataset under varying display brightness and viewing distances. We proposed incorporating white-box modules, in other words, luminance masking and viewing distance adaptation, into the black-box deep neural network, and we found that the combination of white-box modules and black-box deep neural networks could generalize our proposed visibility metric to varying viewing conditions. To demonstrate the application of our proposed deep neural network-based visibility metric to visually lossless image compression, we collected the visually lossless image compression dataset under fixed viewing conditions and significantly improved the deep neural network-based visibility metric's accuracy of predicting visually lossless image compression threshold by pre-training the visibility metric with a synthetic dataset generated by the state-of-the-art white-box visibility metric---HDR-VDP \cite{Mantiuk2011}. In a large-scale study of 1000 images, we found that with our improved visibility metric, we can save around 60\% to 70\% bits for visually lossless image compression encoding as compared to the default visually lossless quality level of 90. Because predicting image visibility and predicting image quality are closely related research topics, we also proposed a trained perceptually uniform transform for high dynamic range images and videos quality assessments by training a perceptual encoding function on a set of subjective quality assessment datasets. We have shown that when combining the trained perceptual encoding function with standard dynamic range image quality metrics, such as peak-signal-noise-ratio (PSNR), better performance was achieved compared to the untrained version