Fast and Interpretable Nonlocal Neural Networks for Image Denoising via Group-Sparse Convolutional Dictionary Learning

Nonlocal self-similarity within natural images has become an increasingly popular prior in deep-learning models. Despite their successful image restoration performance, such models remain largely uninterpretable due to their black-box construction. Our previous studies have shown that interpretable construction of a fully convolutional denoiser (CDLNet), with performance on par with state-of-the-art black-box counterparts, is achievable by unrolling a dictionary learning algorithm. In this manuscript, we seek an interpretable construction of a convolutional network with a nonlocal self-similarity prior that performs on par with black-box nonlocal models. We show that such an architecture can be effectively achieved by upgrading the $\ell 1$ sparsity prior of CDLNet to a weighted group-sparsity prior. From this formulation, we propose a novel sliding-window nonlocal operation, enabled by sparse array arithmetic. In addition to competitive performance with black-box nonlocal DNNs, we demonstrate the proposed sliding-window sparse attention enables inference speeds greater than an order of magnitude faster than its competitors.Comment: 11 pages, 8 figures, 6 table

Lateralization in the dichotic listening of tones is influenced by the content of speech

Available online 10 February 2020.Cognitive functions, for example speech processing, are distributed asymmetrically in the two hemispheres that mostly have homologous anatomical structures. Dichotic listening is a well-established paradigm to investigate hemispherical lateralization of speech. However, the mixed results of dichotic listening, especially when using tonal languages as stimuli, complicates the investigation of functional lateralization. We hypothesized that the inconsistent results in dichotic listening are due to an interaction in processing a mixture of acoustic and linguistic attributes that are differentially processed over the two hemispheres. In this study, a within-subject dichotic listening paradigm was designed, in which different levels of speech and linguistic information was incrementally included in different conditions that required the same tone identification task. A left ear advantage (LEA), in contrast with the commonly found right ear advantage (REA) in dichotic listening, was observed in the hummed tones condition, where only the slow frequency modulation of tones was included. However, when phonemic and lexical information was added in simple vowel tone conditions, the LEA became unstable. Furthermore, ear preference became balanced when phonological and lexicalsemantic attributes were included in the consonant-vowel (CV), pseudo-word, and word conditions. Compared with the existing REA results that use complex vowel word tones, a complete pattern emerged gradually shifting from LEA to REA. These results support the hypothesis that an acoustic analysis of suprasegmental information of tones is preferably processed in the right hemisphere, but is influenced by phonological and lexical semantic processes residing in the left hemisphere. The ear preference in dichotic listening depends on the levels of speech and linguistic analysis and preferentially lateralizes across the different hemispheres. That is, the manifestation of functional lateralization depends on the integration of information across the two hemispheres.This study was supported by National Natural Science Foundation of China 31871131, Major Program of Science and Technology Commission of Shanghai Municipality (STCSM) 17JC1404104, Program of Introducing Talents of Discipline to Universities, Base B16018 to XT, and the JRI Seed Grants for Research Collaboration from NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai to XT and QC, and NIH 2R01DC05660 to David Poeppel at New York University supporting NM and AF and F32 DC011985 to AF

Reconstructing Speech from Human Auditory Cortex

Direct brain recordings from neurosurgical patients listening to speech reveal that the acoustic speech signals can be reconstructed from neural activity in auditory cortex

iEEG-BIDS, extending the Brain Imaging Data Structure specification to human intracranial electrophysiology

The Brain Imaging Data Structure (BIDS) is a community-driven specification for organizing neuroscience data and metadata with the aim to make datasets more transparent, reusable, and reproducible. Intracranial electroencephalography (iEEG) data offer a unique combination of high spatial and temporal resolution measurements of the living human brain. To improve internal (re)use and external sharing of these unique data, we present a specification for storing and sharing iEEG data: iEEG-BIDS

The electrophysiology of language perception and production

For over a century, an abundance of research has tried to elucidate the neurobiological basis of language processing in the human cortex. Neuroimaging and lesion studies have provided great insight into what functions different brain structures subserve. While these techniques provide a high spatial resolution they are limited in the temporal domain. Conversely, contributions from non-invasive electrophysiology provided a high temporal resolution with a limited ability to localize cortical sources. The combined spatial and temporal dynamics of cortical processing during language perception and production remains largely unknown. This dissertation addresses this issue by employing unique neuronal population recordings from neurosurgical patients performing linguistic tasks. The studies described here elucidate the timing, magnitude and spatial extent of cortical processing during perception and production of language. The results provide evidence on the level of single-trial that: 1) A rich network of independent and spatially distinct functional sub-regions of cortex subserve perception and production of language. 2) Neighboring sub-regions 4 mm apart can exhibit inverse functional specific responses to linguistic stimuli and self produced speech. 3) Broca's area is not involved in the actual act of articulation but rather in speech preparation and interfacing perception and production. Taken together, these results defy century old dogmas and suggest that language is supported by a complex network of independent sub-regions, with Broca's area acting as a mediator between perception and production rather than as the seat of articulation

Bridging the Gap Between Big Data and Social Services

The goal of health and human service agencies is to benefit the general public as well as protect at-risk populations from worsening social concerns. While there has been a growing focus on prevention, predictive models can be hard to translate into solutions that can be effectively implemented. The recent proliferation of big data sources has created an unprecedented opportunity to leverage data in order to focus work with vulnerable populations and provide predictive-based intervention prior to the worsening of an individual’s situation. For example, publically available court records indicating an imminent eviction can be used in order to identify a population at a greater risk of becoming homeless. Prevention services can be provided to these identified individuals prior to their becoming homeless. This intervention, which precedes actual homelessness, not only helps an individual or family, but is also cost effective for the city. Such an approach requires integrating solutions across multiple levels: data integrity, predictive analytics, and implementing an effective intervention process. There are not many organizations that have the necessary tools, ability and knowledge to follow through on all these levels in order to deliver an effective outcome. In this perspective we would like to introduce a predictive-based social intervention approach and examine the associated challenges that must be addressed

Bridging the Gap Between Big Data and Social Services

The goal of health and human service agencies is to benefit the general public as well as protect at-risk populations from worsening social concerns. While there has been a growing focus on prevention, predictive models can be hard to translate into solutions that can be effectively implemented. The recent proliferation of big data sources has created an unprecedented opportunity to leverage data in order to focus work with vulnerable populations and provide predictive-based intervention prior to the worsening of an individual’s situation. For example, publically available court records indicating an imminent eviction can be used in order to identify a population at a greater risk of becoming homeless. Prevention services can be provided to these identified individuals prior to their becoming homeless. This intervention, which precedes actual homelessness, not only helps an individual or family, but is also cost effective for the city. Such an approach requires integrating solutions across multiple levels: data integrity, predictive analytics, and implementing an effective intervention process. There are not many organizations that have the necessary tools, ability and knowledge to follow through on all these levels in order to deliver an effective outcome. In this perspective we would like to introduce a predictive-based social intervention approach and examine the associated challenges that must be addressed

Bridging the Gap Between Big Data and Social Services

The goal of health and human service agencies is to benefit the general public as well as protect at-risk populations from worsening social concerns. While there has been a growing focus on prevention, predictive models can be hard to translate into solutions that can be effectively implemented. The recent proliferation of big data sources has created an unprecedented opportunity to leverage data in order to focus work with vulnerable populations and provide predictive-based intervention prior to the worsening of an individual’s situation. For example, publically available court records indicating an imminent eviction can be used in order to identify a population at a greater risk of becoming homeless. Prevention services can be provided to these identified individuals prior to their becoming homeless. This intervention, which precedes actual homelessness, not only helps an individual or family, but is also cost effective for the city. Such an approach requires integrating solutions across multiple levels: data integrity, predictive analytics, and implementing an effective intervention process. There are not many organizations that have the necessary tools, ability and knowledge to follow through on all these levels in order to deliver an effective outcome. In this perspective we would like to introduce a predictive-based social intervention approach and examine the associated challenges that must be addressed

The electrophysiology of language perception and production

For over a century, an abundance of research has tried to elucidate the neurobiological basis of language processing in the human cortex. Neuroimaging and lesion studies have provided great insight into what functions different brain structures subserve. While these techniques provide a high spatial resolution they are limited in the temporal domain. Conversely, contributions from non-invasive electrophysiology provided a high temporal resolution with a limited ability to localize cortical sources. The combined spatial and temporal dynamics of cortical processing during language perception and production remains largely unknown. This dissertation addresses this issue by employing unique neuronal population recordings from neurosurgical patients performing linguistic tasks. The studies described here elucidate the timing, magnitude and spatial extent of cortical processing during perception and production of language. The results provide evidence on the level of single-trial that: 1) A rich network of independent and spatially distinct functional sub-regions of cortex subserve perception and production of language. 2) Neighboring sub-regions 4 mm apart can exhibit inverse functional specific responses to linguistic stimuli and self produced speech. 3) Broca's area is not involved in the actual act of articulation but rather in speech preparation and interfacing perception and production. Taken together, these results defy century old dogmas and suggest that language is supported by a complex network of independent sub-regions, with Broca's area acting as a mediator between perception and production rather than as the seat of articulation