Scene Consistency Representation Learning for Video Scene Segmentation

A long-term video, such as a movie or TV show, is composed of various scenes, each of which represents a series of shots sharing the same semantic story. Spotting the correct scene boundary from the long-term video is a challenging task, since a model must understand the storyline of the video to figure out where a scene starts and ends. To this end, we propose an effective Self-Supervised Learning (SSL) framework to learn better shot representations from unlabeled long-term videos. More specifically, we present an SSL scheme to achieve scene consistency, while exploring considerable data augmentation and shuffling methods to boost the model generalizability. Instead of explicitly learning the scene boundary features as in the previous methods, we introduce a vanilla temporal model with less inductive bias to verify the quality of the shot features. Our method achieves the state-of-the-art performance on the task of Video Scene Segmentation. Additionally, we suggest a more fair and reasonable benchmark to evaluate the performance of Video Scene Segmentation methods. The code is made available.Comment: Accepted to CVPR 202

Joint Prediction of Audio Event and Annoyance Rating in an Urban Soundscape by Hierarchical Graph Representation Learning

Sound events in daily life carry rich information about the objective world. The composition of these sounds affects the mood of people in a soundscape. Most previous approaches only focus on classifying and detecting audio events and scenes, but may ignore their perceptual quality that may impact humans’ listening mood for the environment, e.g. annoyance. To this end, this paper proposes a novel hierarchical graph representation learning (HGRL) approach which links objective audio events (AE) with subjective annoyance ratings (AR) of the soundscape perceived by humans. The hierarchical graph consists of fine-grained event (fAE) embeddings with single-class event semantics, coarse-grained event (cAE) embeddings with multi-class event semantics, and AR embeddings. Experiments show the proposed HGRL successfully integrates AE with AR for AEC and ARP tasks, while coordinating the relations between cAE and fAE and further aligning the two different grains of AE information with the AR

Nested Event Extraction upon Pivot Element Recogniton

Nested Event Extraction (NEE) aims to extract complex event structures where an event contains other events as its arguments recursively. Nested events involve a kind of Pivot Elements (PEs) that simultaneously act as arguments of outer events and as triggers of inner events, and thus connect them into nested structures. This special characteristic of PEs brings challenges to existing NEE methods, as they cannot well cope with the dual identities of PEs. Therefore, this paper proposes a new model, called PerNee, which extracts nested events mainly based on recognizing PEs. Specifically, PerNee first recognizes the triggers of both inner and outer events and further recognizes the PEs via classifying the relation type between trigger pairs. In order to obtain better representations of triggers and arguments to further improve NEE performance, it incorporates the information of both event types and argument roles into PerNee through prompt learning. Since existing NEE datasets (e.g., Genia11) are limited to specific domains and contain a narrow range of event types with nested structures, we systematically categorize nested events in generic domain and construct a new NEE dataset, namely ACE2005-Nest. Experimental results demonstrate that PerNee consistently achieves state-of-the-art performance on ACE2005-Nest, Genia11 and Genia13

Depth-resolved microscopy of cortical hemodynamics with optical coherence tomography

We describe depth-resolved microscopy of cortical hemodynamics with high-speed spectral/Fourier domain optical coherence tomography (OCT). Stimulus-evoked changes in blood vessel diameter, flow, and total hemoglobin were measured in the rat somatosensory cortex. The results show OCT measurements of hemodynamic changes during functional activation and represent an important step toward understanding functional hyperemia at the microscopic level.National Institutes of Health (U.S.) (R01-NS057476)National Institutes of Health (U.S.) (P01NS055104)National Institutes of Health (U.S.) (P50NS010828)National Institutes of Health (U.S.) (K99NS067050)National Institutes of Health (U.S.) (R01-CA075289-12)United States. Air Force Office of Scientific Research (FA9550-07-1-0014)United States. Dept. of Defense. Medical Free Electron Laser Program (FA9550-07-1-0101

Absence of surrogate light chain results in spontaneous autoreactive germinal centres expanding VH81X-expressing B cells

Random recombination of antibody heavy- and light-chain genes results in a diverse B-cell receptor (BCR) repertoire including self-reactive BCRs. However, tolerance mechanisms that prevent the development of self-reactive B cells remain incompletely understood. The absence of the surrogate light chain, which assembles with antibody heavy chain forming a pre-BCR, leads to production of antinuclear antibodies (ANAs). Here we show that the naive follicular B-cell pool is enriched for cells expressing prototypic ANA heavy chains in these mice in a non-autoimmune background with a broad antibody repertoire. This results in the spontaneous formation of T-cell-dependent germinal centres that are enriched with B cells expressing prototypic ANA heavy chains. However, peripheral tolerance appears maintained by selection thresholds on cells entering the memory B-cell and plasma cell pools, as exemplified by the exclusion of cells expressing the intrinsically self-reactive VH81X from both pool

Genome-wide detection of human variants that disrupt intronic branchpoints

The search for candidate variants underlying human disease in massive parallel sequencing data typically focuses on coding regions and essential splice sites, mostly ignoring noncoding variants. The RNA spliceosome recognizes intronic branchpoint (BP) motifs at the beginning of splicing and operates mostly within introns to define the exon-intron boundaries; however, BP variants have been paid little attention. We established a comprehensive genome-wide database and knowledgebase of BP and developed BPHunter for systematic and informative genome-wide detection of intronic variants that may disrupt BP and splicing, together with an effective strategy for prioritizing BP variant candidates. BPHunter not only constitutes an important resource for understanding BP, but should also drive discovery of BP variants in human genetic diseases and traits. Pre-messenger RNA splicing is initiated with the recognition of a single-nucleotide intronic branchpoint (BP) within a BP motif by spliceosome elements. Forty-eight rare variants in 43 human genes have been reported to alter splicing and cause disease by disrupting BP. However, until now, no computational approach was available to efficiently detect such variants in massively parallel sequencing data. We established a comprehensive human genome-wide BP database by integrating existing BP data and generating new BP data from RNA sequencing of lariat debranching enzyme DBR1-mutated patients and from machine-learning predictions. We characterized multiple features of BP in major and minor introns and found that BP and BP-2 (two nucleotides upstream of BP) positions exhibit a lower rate of variation in human populations and higher evolutionary conservation than the intronic background, while being comparable to the exonic background. We developed BPHunter as a genome-wide computational approach to systematically and efficiently detect intronic variants that may disrupt BP recognition. BPHunter retrospectively identified 40 of the 48 known pathogenic BP variants, in which we summarized a strategy for prioritizing BP variant candidates. The remaining eight variants all create AG-dinucleotides between the BP and acceptor site, which is the likely reason for missplicing. We demonstrated the practical utility of BPHunter prospectively by using it to identify a novel germline heterozygous BP variant of STAT2 in a patient with critical COVID-19 pneumonia and a novel somatic intronic 59-nucleotide deletion of ITPKB in a lymphoma patient, both of which were validated experimentally. BPHunter is publicly available from an

Quantitative cerebral blood flow with optical coherence tomography

Absolute measurements of cerebral blood flow (CBF) are an important endpoint in studies of cerebral pathophysiology. Currently no accepted method exists for in vivo longitudinal monitoring of CBF with high resolution in rats and mice. Using three-dimensional Doppler Optical Coherence Tomography and cranial window preparations, we present methods and algorithms for regional CBF measurements in the rat cortex. Towards this end, we develop and validate a quantitative statistical model to describe the effect of static tissue on velocity sensitivity. This model is used to design scanning protocols and algorithms for sensitive 3D flow measurements and angiography of the cortex. We also introduce a method of absolute flow calculation that does not require explicit knowledge of vessel angles. We show that OCT estimates of absolute CBF values in rats agree with prior measures by autoradiography, suggesting that Doppler OCT can perform absolute flow measurements in animal models.National Institutes of Health (U.S.) (Grant number R01-NS057476)National Institutes of Health (U.S.) (Grant number P01NS055104)National Institutes of Health (U.S.) (Grant number P50NS010828)ational Institutes of Health (U.S.) (Grant number K99NS067050)National Institutes of Health (U.S.) (Grant number R01-CA075289-13)United States. Air Force Office of Scientific Research (FA9550-07-1-0014)United States. Dept. of Defense. Medical Free Electron Laser Program (FA9550-07-1-0101