Seeing faces: evidence suggesting cortical disinhibition in the genesis of visual hallucinations.

The neural mechanisms responsible for triggering visual hallucinations are poorly understood. Here, we report a unique patient whose hallucinations consist exclusively of faces, and which could be reliably precipitated by looking at trees. Using functional Magnetic Resonance Imaging (fMRI), we found that, while face hallucinations was associated with increased neural activity in a number of cortical regions, including low-level visual areas, there was significant decreased activity in the right fusiform face area, a region that is empirically defined by increase activity during veridical perception of faces. These findings indicate key differences in how hallucinatory and veridical perceptions lead to the same phenomenological experience of seeing faces, and are consistent with the hypothesis that hallucinations may be generated by decreased inhibitory inputs to key cortical regions, in contrast to the excitatory synaptic inputs underlying veridical perception

Abnormal connectivity between the default mode and the visual system underlies the manifestation of visual hallucinations in Parkinson’s disease:A task-based fMRI study

Background: The neural substrates of visual hallucinations remain an enigma, due primarily to the difficulties associated with directly interrogating the brain during hallucinatory episodes. Aims: To delineate the functional patterns of brain network activity and connectivity underlying visual hallucinations in Parkinson’s disease. Methods: In this study, we combined functional magnetic resonance imaging (MRI) with a behavioral task capable of eliciting visual misperceptions, a confirmed surrogate for visual hallucinations, in 35 patients with idiopathic Parkinson’s disease. We then applied an independent component analysis to extract time series information for large-scale neuronal networks that have been previously implicated in the pathophysiology of visual hallucinations. These data were subjected to a task-based functional connectivity analysis, thus providing the first objective description of the neural activity and connectivity during visual hallucinations in patients with Parkinson’s disease. Results: Correct performance of the task was associated with increased activity in primary visual regions; however, during visual misperceptions, this same visual network became actively coupled with the default mode network (DMN). Further, the frequency of misperception errors on the task was positively correlated with the strength of connectivity between these two systems, as well as with decreased activity in the dorsal attention network (DAN), and with impaired connectivity between the DAN and the DMNs, and ventral attention networks. Finally, each of the network abnormalities identified in our analysis were significantly correlated with two independent clinical measures of hallucination severity. Conclusions: Together, these results provide evidence that visual hallucinations are due to increased engagement of the DMN with the primary visual system, and emphasize the role of dysfunctional engagement of attentional networks in the pathophysiology of hallucinations

Changes in structural network topology correlate with severity of hallucinatory behavior in Parkinson's disease

Inefficient integration between bottom-up visual input and higher order visual processing regions is implicated in visual hallucinations in Parkinson's disease (PD). Here, we investigated white matter contributions to this perceptual imbalance hypothesis. Twenty-nine PD patients were assessed for hallucinatory behavior. Hallucination severity was correlated to connectivity strength of the network using the network-based statistic approach. The results showed that hallucination severity was associated with reduced connectivity within a subnetwork that included the majority of the diverse club. This network showed overall greater between-module scores compared with nodes not associated with hallucination severity. Reduced between-module connectivity in the lateral occipital cortex, insula, and pars orbitalis and decreased within-module connectivity in the prefrontal, somatosensory, and primary visual cortices were associated with hallucination severity. Conversely, hallucination severity was associated with increased between- and within-module connectivity in the orbitofrontal and temporal cortex, as well as regions comprising the dorsal attentional and default mode network. These results suggest that hallucination severity is associated with marked alterations in structural network topology with changes in participation along the perceptual hierarchy. This may result in the inefficient transfer of information that gives rise to hallucinations in PD. Author SummaryInefficient integration of information between external stimuli and internal perceptual predictions may lead to misperceptions or visual hallucinations in Parkinson's disease (PD). In this study, we show that hallucinatory behavior in PD patients is associated with marked alterations in structural network topology. Severity of hallucinatory behavior was associated with decreased connectivity in a large subnetwork that included the majority of the diverse club, nodes with a high number of between-module connections. Furthermore, changes in between-module connectivity were found across brain regions involved in visual processing, top-down prediction centers, and endogenous attention, including the occipital, orbitofrontal, and posterior cingulate cortex. Together, these findings suggest that impaired integration across different sides across different perceptual processing regions may result in inefficient transfer of information

Clue: Cross-modal Coherence Modeling for Caption Generation

We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning. Using an annotation protocol specifically devised for capturing image--caption coherence relations, we annotate 10,000 instances from publicly-available image--caption pairs. We introduce a new task for learning inferences in imagery and text, coherence relation prediction, and show that these coherence annotations can be exploited to learn relation classifiers as an intermediary step, and also train coherence-aware, controllable image captioning models. The results show a dramatic improvement in the consistency and quality of the generated captions with respect to information needs specified via coherence relations.Comment: Accepted as a long paper to ACL 202

Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline

Recovering a high dynamic range (HDR) image from a single low dynamic range (LDR) input image is challenging due to missing details in under-/over-exposed regions caused by quantization and saturation of camera sensors. In contrast to existing learning-based methods, our core idea is to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We then propose to learn three specialized CNNs to reverse these steps. By decomposing the problem into specific sub-tasks, we impose effective physical constraints to facilitate the training of individual sub-networks. Finally, we jointly fine-tune the entire model end-to-end to reduce error accumulation. With extensive quantitative and qualitative experiments on diverse image datasets, we demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.Comment: CVPR 2020. Project page: https://www.cmlab.csie.ntu.edu.tw/~yulunliu/SingleHDR Code: https://github.com/alex04072000/SingleHD