Local Visual Microphones: Improved Sound Extraction from Silent Video

Sound waves cause small vibrations in nearby objects. A few techniques exist in the literature that can extract sound from video. In this paper we study local vibration patterns at different image locations. We show that different locations in the image vibrate differently. We carefully aggregate local vibrations and produce a sound quality that improves state-of-the-art. We show that local vibrations could have a time delay because sound waves take time to travel through the air. We use this phenomenon to estimate sound direction. We also present a novel algorithm that speeds up sound extraction by two to three orders of magnitude and reaches real-time performance in a 20KHz video.Comment: Accepted to BMVC 201

지식 증류를위한 다단계 교사

학위논문(석사)--서울대학교 대학원 :공과대학 전기·컴퓨터공학부,2019. 8. Lee, Kyoung Mu.지식 증류 (Knowledge Distillation, KD)는 교사로부터 학생 모델로 지식을 전 달하는 잘 알려진 방법입니다. 본 논문에서는 계층 적 진보적 교사 (Layer-wise Pro- gressive Teacher)를 도입하여 지식 증류를위한 새로운 틀을 제안하고자한다. 이와 관련하여 우리는 교사의 중간 계층에서 확률을 구함으로써 서로 다른 경도 수준에 서 부드러운 목표를 만드는 방법을 제안합니다. 우리의 방법은 교사와 학생 사이에 큰 차이가있어 학생이 교사를 모방하는 것을 더 어렵게하는 경우를 위해 특별히 고 안되었습니다. 우리는 또한 학생의 온도를 제거하고 교사의 온도를 유지하는 것이 좋습니다. 실험 결과는 기존의 증류법과 비교할 때 우리의 방법이 훨씬 더 우수한 결과를 얻음을 보여줍니다.Knowledge Distillation (KD) is a well-known method for transferring knowledge from a teacher to a student model. In this thesis, we propose a new framework for Knowledge Distillation by introducing a Layer-wise Progressive Teacher. In this regard, we propose a method to create soft targets in different levels of complexity by obtaining the probabilities from the intermediate layers of the teacher network. Our method is specially designed for the cases that there is a large gap between the teacher and the student which makes it harder for the student to mimic the teacher. In addition, we proposed focalized teacher as a method to train a better teacher for the student. The experimental results show that our method gets significantly better results in comparison with existing knowledge distillation methods.1 Introduction 1 1.1 Background. 1 1.2 Motivation 3 1.3 ProposedMethod 4 1.4 Datasets. 5 2 Related Work 7 2.1 Theory of Transfer Learning 7 2.2 Applications. 8 3 Focalized Teacher 10 3.1 Overview 10 3.2 LabelCorrection 11 3.3 FocalizedTeacher 12 3.4 Experimental Results 13 4 Layer-wise Progressive Knowledge Distillation 16 4.1 BackgroundandNotations . 16 4.2 Layer-wiseKnowledgeDistillation. 17 4.3 ProgressiveTeacher. 19 4.4 Experimental Results 20 4.4.1 TemperatureAnalysis . 21 4.4.2 DistanceMetric. 23 4.4.3 Distilled Knowledge from an intermediate layer . . . . . . . . 24 4.4.4 ProgressiveTeacher 27 4.4.5 ComparisonwithotherKDmethods 29 5 Conclusion 5.1 SummaryoftheThesis . 32 5.2 FutureWorks 32 5.2.1 Progressive Teacher Assistant based Knowledge Distillation . 33Maste

HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising

The paper presents a novel approach for vector-floorplan generation via a diffusion model, which denoises 2D coordinates of room/door corners with two inference objectives: 1) a single-step noise as the continuous quantity to precisely invert the continuous forward process; and 2) the final 2D coordinate as the discrete quantity to establish geometric incident relationships such as parallelism, orthogonality, and corner-sharing. Our task is graph-conditioned floorplan generation, a common workflow in floorplan design. We represent a floorplan as 1D polygonal loops, each of which corresponds to a room or a door. Our diffusion model employs a Transformer architecture at the core, which controls the attention masks based on the input graph-constraint and directly generates vector-graphics floorplans via a discrete and continuous denoising process. We have evaluated our approach on RPLAN dataset. The proposed approach makes significant improvements in all the metrics against the state-of-the-art with significant margins, while being capable of generating non-Manhattan structures and controlling the exact number of corners per room. A project website with supplementary video and document is here https://aminshabani.github.io/housediffusion

Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting

The performance of time series forecasting has recently been greatly improved by the introduction of transformers. In this paper, we propose a general multi-scale framework that can be applied to the state-of-the-art transformer-based time series forecasting models (FEDformer, Autoformer, etc.). By iteratively refining a forecasted time series at multiple scales with shared weights, introducing architecture adaptations, and a specially-designed normalization scheme, we are able to achieve significant performance improvements, from 5.5% to 38.5% across datasets and transformer architectures, with minimal additional computational overhead. Via detailed ablation studies, we demonstrate the effectiveness of each of our contributions across the architecture and methodology. Furthermore, our experiments on various public datasets demonstrate that the proposed improvements outperform their corresponding baseline counterparts. Our code is publicly available in https://github.com/BorealisAI/scaleformer

Developing a Prediction Model for Author Collaboration in Bioinformatics Research Using Graph Mining Techniques and Big Data Applications

Nowadays, scientific collaboration has dramatically increased due to web-based technologies, advanced communication systems, and information and scientific databases. The present study aims to provide a predictive model for author collaborations in bioinformatics research output using graph mining techniques and big data applications. The study is applied-developmental research adopting a mixed-method approach, i.e., a mix of quantitative and qualitative measures. The research population consisted of all bioinformatics research documents indexed in PubMed (n=699160). The correlations of bioinformatics articles were examined in terms of weight and strength based on article sections including title, abstract, keywords, journal title, and author affiliation using graph mining techniques and big data applications. Eventually, the prediction model of author collaboration in bioinformatics research was developed using the abovementioned tools and expert-assigned weights. The calculations and data analysis were carried out using Expert Choice, Excel, Spark, and Scala, and Python programming languages in a big data server. Accordingly, the research was conducted in three phases: 1) identifying and weighting the factors contributing to authors’ similarity measurement; 2) implementing co-authorship prediction model; and 3) integrating the first and second phases (i.e., integrating the weights obtained in the previous phases). The results showed that journal title, citation, article title, author affiliation, keywords, and abstract scored 0.374, 0.374, 0.091, 0.075, 0.055, and 0.031. Moreover, the journal title achieved the highest score in the model for the co-author recommender system. As the data in bibliometric information networks is static, it was proved remarkably effective to use content-based features for similarity measures. So that the recommender system can offer the most suitable collaboration suggestions. It is expected that the model works efficiently in other databases and provides suitable recommendations for author collaborations in other subject areas. By integrating expert opinion and systemic weights, the model can help alleviate the current information overload and facilitate collaborator lookup by authors.https://dorl.net/dor/20.1001.1.20088302.2021.19.2.1.

JigsawPlan: Room Layout Jigsaw Puzzle Extreme Structure from Motion using Diffusion Models

This paper presents a novel approach to the Extreme Structure from Motion (E-SfM) problem, which takes a set of room layouts as polygonal curves in the top-down view, and aligns the room layout pieces by estimating their 2D translations and rotations, akin to solving the jigsaw puzzle of room layouts. The biggest discovery and surprise of the paper is that the simple use of a Diffusion Model solves this challenging registration problem as a conditional generation process. The paper presents a new dataset of room layouts and floorplans for 98,780 houses. The qualitative and quantitative evaluations demonstrate that the proposed approach outperforms the competing methods by significant margins

A Comprehensive Survey on Graph Summarization with Graph Neural Networks

As large-scale graphs become more widespread, more and more computational challenges with extracting, processing, and interpreting large graph data are being exposed. It is therefore natural to search for ways to summarize these expansive graphs while preserving their key characteristics. In the past, most graph summarization techniques sought to capture the most important part of a graph statistically. However, today, the high dimensionality and complexity of modern graph data are making deep learning techniques more popular. Hence, this paper presents a comprehensive survey of progress in deep learning summarization techniques that rely on graph neural networks (GNNs). Our investigation includes a review of the current state-of-the-art approaches, including recurrent GNNs, convolutional GNNs, graph autoencoders, and graph attention networks. A new burgeoning line of research is also discussed where graph reinforcement learning is being used to evaluate and improve the quality of graph summaries. Additionally, the survey provides details of benchmark datasets, evaluation metrics, and open-source tools that are often employed in experimentation settings, along with a discussion on the practical uses of graph summarization in different fields. Finally, the survey concludes with a number of open research challenges to motivate further study in this area.Comment: 20 pages, 4 figures, 3 tables, Journal of IEEE Transactions on Artificial Intelligenc

Human Umbilical Cord Mesenchymal Stem Cells-Derived Exosomes Can Alleviate the Proctitis Model Through TLR4/NF-Κb Pathway

Background: Proctitis is a significant concern of inflammatory bowel diseases, especially ulcerative colitis. Exosomes are a new method for treating many diseases by their immunosuppressive and tissue-repairing potential. Here, we tried Mesenchymal stem cells (MSCs)-derived Exosomes for treating the proctitis model of rats. Materials and Methods: Rats were assigned into four groups: sham, control group, rectal, and intraperitoneal exosome injection. The proctitis model was induced by rectal administration of 4% acetic acid. The exosome was derived from human MSCs isolated from human umbilical cords. After seven days, rectum samples were assessed for histopathological, IHC, and PCR analysis. Results: The histopathologic scores, collagen deposition, and the expression of NF-κB, TLR4, TNFα, IL-6, and TGFβ were decreased in intraperitoneal exosome compared to controls. The result was not promising for the rectal administration of exosomes. Conclusion: Exosomes can suppress the inflammatory response in the proctitis model and improve the rectum's healing process. Exosomes can inhabit the NF-κB/TLR4 pathway and downstream pro-inflammatory cytokines. This study implicates the therapeutic benefits of exomes in treating proctitis