Intel Labs at Ego4D Challenge 2022: A Better Baseline for Audio-Visual Diarization

This report describes our approach for the Audio-Visual Diarization (AVD) task of the Ego4D Challenge 2022. Specifically, we present multiple technical improvements over the official baselines. First, we improve the detection performance of the camera wearer's voice activity by modifying the training scheme of its model. Second, we discover that an off-the-shelf voice activity detection model can effectively remove false positives when it is applied solely to the camera wearer's voice activities. Lastly, we show that better active speaker detection leads to a better AVD outcome. Our final method obtains 65.9% DER on the test set of Ego4D, which significantly outperforms all the baselines. Our submission achieved 1st place in the Ego4D Challenge 2022.Comment: Validation report for the Ego4D challenge at ECCV 202

Possible quantum phase-manipulation of a two-leg ladder in mixed-dimensional fermionic cold atoms

The recent realization of mixed-dimensional systems of cold atoms has attracted much attention from both experimentalists and theorists. Different effective interactions and novel correlated quantum many-body phases may be engineered in these systems, with the different phases being tunable via external parameters. In this article we investigate a two-species Fermi atom mixture: one species of atom exists in two hyperfine states and is confined to move in a two-leg ladder, interacting with an on-site interaction, and the other moves freely in a two dimensional square lattice that contains the two-leg ladder. The two species of atoms interact via an on-site interaction on the ladder. In the limit of weak inter-species interactions, the two-dimensional gas can be integrated out, leading to an effective long-range mediated interaction in the ladder, generated by to the on-site inter-species interaction. We show that the form of the mediated interaction can be controlled by the density of the two-dimensional gas and that it enhances the charge density wave instability in the two-leg ladder after the renormalization group transformation. Parameterizing the phase diagram with various experimentally controllable quantities, we discuss the possible tuning of the macroscopic quantum many-body phases of the two-leg ladder in this mixed-dimensional fermionic cold atom system.Comment: 4 pages and 3 figure

The impact of urban sprawl on journey to work times for mass transit and all other commuters in the United States: A research note

As government budgets get tighter, there has been considerable public outcry about the continued investment in public mass transit systems and their financial viability. Amid this outcry, a number of studies have been conducted to determine which factors influence the use and efficiency of publiclyfunded mass transit systems. These factors include population density and less sprawl (or greater urban compactness). However, their impact on mass transit usage is somewhat contradictory in that the heavy concentration of populations in the urban area and greater compactness is believed to increase mass transit usage due to a bigger number of potential passengers. In fact, greater compactness and greater transit ridership have played a role in lengthening the journey to work for most commuters and thus discouraged the use of mass transit systems. Thus, some questioned the wisdom of mass transit subsidies and “smart growth” policies. To attempt to answer this question and avoid any further confusion, this paper examines how urban sprawl affects the journey to work commute time of mass transit riders and other commuters throughout the United States after controlling for variables such as the volume of ridership, local per capita income, the presence of a local rail transit system, and local weather. The findings for this research note defy some conventional wisdom and point to several public policy recommendations on how to improve public mass transit at the local level. For instance, we find that greater urban compactness can be turned into a mass transit advantage if mass transit riders can use a commuter rail option

Disruption of the nuclear p53-GAPDH complex protects against ischemia-induced neuronal damage

Glyceraldehyde 3-phosphate dehydrogenase (GAPDH) is conventionally considered a critical enzyme that involves in glycolysis for energy production. Recent previous studies have suggested that GAPDH is important in glutamate-induced neuronal excitotoxicity, while accumulated evidence also demonstrated that GAPDH nuclear translocation plays a critical role in cell death. However, the molecular mechanisms underlying this process remain largely unknown. In this study, we showed that GAPDH translocates to the nucleus in a Siah1-dependent manner upon glutamate stimulation. The nuclear GAPDH forms a protein complex with p53 and enhances p53 expression and phosphorylation. Disruption of the GAPDH-p53 interaction with an interfering peptide blocks glutamate-induced cell death and GAPDH-mediated up-regulation of p53 expression and phosphorylation. Furthermore, administration of the interfering peptide in vivo protects against ischemia-induced cell death in rats subjected to tMCAo. Our data suggest that the nuclear p53-GAPDH complex is important in regulating glutamate-mediated neuronal death and could serve as a potential therapeutic target for ischemic stroke treatment

SViTT: Temporal Learning of Sparse Video-Text Transformers

Do video-text transformers learn to model temporal relationships across frames? Despite their immense capacity and the abundance of multimodal training data, recent work has revealed the strong tendency of video-text models towards frame-based spatial representations, while temporal reasoning remains largely unsolved. In this work, we identify several key challenges in temporal learning of video-text transformers: the spatiotemporal trade-off from limited network size; the curse of dimensionality for multi-frame modeling; and the diminishing returns of semantic information by extending clip length. Guided by these findings, we propose SViTT, a sparse video-text architecture that performs multi-frame reasoning with significantly lower cost than naive transformers with dense attention. Analogous to graph-based networks, SViTT employs two forms of sparsity: edge sparsity that limits the query-key communications between tokens in self-attention, and node sparsity that discards uninformative visual tokens. Trained with a curriculum which increases model sparsity with the clip length, SViTT outperforms dense transformer baselines on multiple video-text retrieval and question answering benchmarks, with a fraction of computational cost. Project page: http://svcl.ucsd.edu/projects/svitt.Comment: CVPR 202

Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection

Active speaker detection (ASD) in videos with multiple speakers is a challenging task as it requires learning effective audiovisual features and spatial-temporal correlations over long temporal windows. In this paper, we present SPELL, a novel spatial-temporal graph learning framework that can solve complex tasks such as ASD. To this end, each person in a video frame is first encoded in a unique node for that frame. Nodes corresponding to a single person across frames are connected to encode their temporal dynamics. Nodes within a frame are also connected to encode inter-person relationships. Thus, SPELL reduces ASD to a node classification task. Importantly, SPELL is able to reason over long temporal contexts for all nodes without relying on computationally expensive fully connected graph neural networks. Through extensive experiments on the AVA-ActiveSpeaker dataset, we demonstrate that learning graph-based representations can significantly improve the active speaker detection performance owing to its explicit spatial and temporal structure. SPELL outperforms all previous state-of-the-art approaches while requiring significantly lower memory and computational resources. Our code is publicly available at https://github.com/SRA2/SPELLComment: ECCV 2022 camera ready (Supplementary videos: on ECVA soon). This paper supersedes arXiv:2112.0147

Pay for Success: A Roadmap for Implementation in Minnesota

Professional paper for the fulfillment of the Master of Public Policy degree.Pay for Success (PFS) is a promising financing model that encourages investment in programs that produce improved social outcomes resulting in future cost-savings for the government. In a PFS project, investors provide initial capital to scale-up effective social programs and the government pays back the investors only if the desired outcomes are achieved. Minnesota emerged as a pioneer in this field, being the first U.S. state to enact legislation authorizing a Pay for Performance pilot in 2011, even before the first PFS program was launched in New York. However, despite having the legislation in effect for more than 6 years now, no PFS project has been implemented in Minnesota. Meanwhile, over 20 PFS programs have been launched in other states such as Illinois, Ohio, Colorado, and South Carolina among others, some of which have also seen their first success payments made out to the investors

Pay for Success: A Roadmap for Implementation in Minnesota

Pay for Success (PFS) is a promising financing model that encourages investment in programs that produce improved social outcomes resulting in future cost-savings for the government. In a PFS project, investors provide initial capital to scale-up effective social programs and the government pays back the investors only if the desired outcomes are achieved. Minnesota emerged as a pioneer in this field, being the first U.S. state to enact legislation authorizing a Pay for Performance pilot in 2011, even before the first PFS program was launched in New York. However, despite having the legislation in effect for more than 6 years now, no PFS project has been implemented in Minnesota. Meanwhile, over 20 PFS programs have been launched in other states such as Illinois, Ohio, Colorado, and South Carolina among others, some of which have also seen their first success payments made out to the investors. This report describes the PFS financing mechanism and provides a set of recommendations for the state government and other stakeholders to advance the implementation of PFS projects in Minnesota recognizing the roadblocks that stalled implementation of the state Pay for Performance Act. We present a set of steps through which PFS funding can be approached in Minnesota and provide a list of program areas where PFS projects can be launched. The report also highlights the legislative action that could move PFS projects ahead in the state of Minnesota and discusses some ways to move forward in the absence of legislative involvement