An Internal Learning Approach to Video Inpainting

We propose a novel video inpainting algorithm that simultaneously hallucinates missing appearance and motion (optical flow) information, building upon the recent 'Deep Image Prior' (DIP) that exploits convolutional network architectures to enforce plausible texture in static images. In extending DIP to video we make two important contributions. First, we show that coherent video inpainting is possible without a priori training. We take a generative approach to inpainting based on internal (within-video) learning without reliance upon an external corpus of visual data to train a one-size-fits-all model for the large space of general videos. Second, we show that such a framework can jointly generate both appearance and flow, whilst exploiting these complementary modalities to ensure mutual consistency. We show that leveraging appearance statistics specific to each video achieves visually plausible results whilst handling the challenging problem of long-term consistency.Comment: Accepted by ICCV 2019. Website: https://cs.stanford.edu/~haotianz/publications/video_inpainting

FreConv: Frequency Branch-and-Integration Convolutional Networks

Recent researches indicate that utilizing the frequency information of input data can enhance the performance of networks. However, the existing popular convolutional structure is not designed specifically for utilizing the frequency information contained in datasets. In this paper, we propose a novel and effective module, named FreConv (frequency branch-and-integration convolution), to replace the vanilla convolution. FreConv adopts a dual-branch architecture to extract and integrate high- and low-frequency information. In the high-frequency branch, a derivative-filter-like architecture is designed to extract the high-frequency information while a light extractor is employed in the low-frequency branch because the low-frequency information is usually redundant. FreConv is able to exploit the frequency information of input data in a more reasonable way to enhance feature representation ability and reduce the memory and computational cost significantly. Without any bells and whistles, experimental results on various tasks demonstrate that FreConv-equipped networks consistently outperform state-of-the-art baselines.Comment: Accepted by ICME202

Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment

Multimedia summarization with multimodal output (MSMO) is a recently explored application in language grounding. It plays an essential role in real-world applications, i.e., automatically generating cover images and titles for news articles or providing introductions to online videos. However, existing methods extract features from the whole video and article and use fusion methods to select the representative one, thus usually ignoring the critical structure and varying semantics. In this work, we propose a Semantics-Consistent Cross-domain Summarization (SCCS) model based on optimal transport alignment with visual and textual segmentation. In specific, our method first decomposes both video and article into segments in order to capture the structural semantics, respectively. Then SCCS follows a cross-domain alignment objective with optimal transport distance, which leverages multimodal interaction to match and select the visual and textual summary. We evaluated our method on three recent multimodal datasets and demonstrated the effectiveness of our method in producing high-quality multimodal summaries

Neural, electrophysiological and anatomical basis of brain-network variability and its characteristic changes in mental disorders

Functional brain networks demonstrate significant temporal variability and dynamic reconfiguration even in the resting state. Currently, most studies investigate temporal variability of brain networks at the scale of single (micro) or whole-brain (macro) connectivity. However, the mechanism underlying time-varying properties remains unclear, as the coupling between brain network variability and neural activity is not readily apparent when analysed at either micro or macroscales. We propose an intermediate 15 (meso) scale analysis and characterize temporal variability of the functional architecture associated with a particular region. This yields a topography of variability that reflects the whole-brain and, most importantly, creates an analytical framework to establish the fundamental relationship between variability of regional functional architecture and its neural activity or structural connectivity. We find that temporal variability reflects the dynamical reconfiguration of a brain region into distinct functional modules at different times and may be indicative of brain flexibility and adaptability. Primary and unimodal sensory-motor cortices demon- 20 strate low temporal variability, while transmodal areas, including heteromodal association areas and limbic system, demonstrate the high variability. In particular, regions with highest variability such as hippocampus/parahippocampus, inferior and middle temporal gyrus, olfactory gyrus and caudate are all related to learning, suggesting that the temporal variability may indicate the level of brain adaptability. With simultaneously recorded electroencephalography/functional magnetic resonance imaging and functional magnetic resonance imaging/diffusion tensor imaging data, we also find that variability of regional functional architec- 25 ture is modulated by local blood oxygen level-dependent activity and a-band oscillation, and is governed by the ratio of intra- to inter-community structural connectivity. Application of the mesoscale variability measure to multicentre datasets of three mental disorders and matched controls involving 1180 subjects reveals that those regions demonstrating extreme, i.e. highest/lowest variability in controls are most liable to change in mental disorders. Specifically, we draw attention to the identification of diametrically opposing patterns of variability changes between schizophrenia and attention deficit hyperactivity disorder/autism. 30 Regions of the default-mode network demonstrate lower variability in patients with schizophrenia, but high variability in patients with autism/attention deficit hyperactivity disorder, compared with respective controls. In contrast, subcortical regions, especially the thalamus, show higher variability in schizophrenia patients, but lower variability in patients with attention deficit hyperactivity disorder. The changes in variability of these regions are also closely related to symptom scores. Our work provides insights into the dynamic organization of the resting brain and how it changes in brain disorders. The nodal variability measure may also be 35 potentially useful as a predictor for learning and neural rehabilitation

Stereotaxical Infusion of Rotenone: A Reliable Rodent Model for Parkinson's Disease

A clinically-related animal model of Parkinson's disease (PD) may enable the elucidation of the etiology of the disease and assist the development of medications. However, none of the current neurotoxin-based models recapitulates the main clinical features of the disease or the pathological hallmarks, such as dopamine (DA) neuron specificity of degeneration and Lewy body formation, which limits the use of these models in PD research. To overcome these limitations, we developed a rat model by stereotaxically (ST) infusing small doses of the mitochondrial complex-I inhibitor, rotenone, into two brain sites: the right ventral tegmental area and the substantia nigra. Four weeks after ST rotenone administration, tyrosine hydroxylase (TH) immunoreactivity in the infusion side decreased by 43.7%, in contrast to a 75.8% decrease observed in rats treated systemically with rotenone (SYS). The rotenone infusion also reduced the DA content, the glutathione and superoxide dismutase activities, and induced alpha-synuclein expression, when compared to the contralateral side. This ST model displays neither peripheral toxicity or mortality and has a high success rate. This rotenone-based ST model thus recapitulates the slow and specific loss of DA neurons and better mimics the clinical features of idiopathic PD, representing a reliable and more clinically-related model for PD research

Redundancy-Reduction-Based Hierarchical Design in Synchronization of Multi-Agent Systems

In this paper, a layered, undirected-network-structure, optimization approach is proposed to reduce the redundancy in multi-agent information synchronization and improve the computing rate. Based on the traversing binary tree and aperiodic sampling of the complex delayed networks theory, we proposed a network-partitioning method for pinning dynamic networks, with a more simplified, analyzable structure, and all of the traversed nodes are mathematically asymptotically synchronized at the same time. Moreover, a systematic implementable approach is proposed for application. The approach could be separated into two main steps: the first is a method of network partition that reduces the trivial interaction, which does not affect the information traversal, and the second involves applying the time-dependent functional approach of Lyapunov to give global exponential conditions, under the criteria for the synchronization of multiple agents, with a lower conservatism of the decision variables compared to some other results, so that the information available could fully benefit from the actual discrete-time communication pattern. Both mathematical proofs and numerical example evidence are presented to demonstrate the effectiveness of the implemented approach. This class contains a number of practically interesting systems, for instance, unmanned aerial vehicle (UAV) formation systems or the ground-air coordinated unmanned aerial system