70 research outputs found
Offline-to-Online Knowledge Distillation for Video Instance Segmentation
In this paper, we present offline-to-online knowledge distillation (OOKD) for
video instance segmentation (VIS), which transfers a wealth of video knowledge
from an offline model to an online model for consistent prediction. Unlike
previous methods that having adopting either an online or offline model, our
single online model takes advantage of both models by distilling offline
knowledge. To transfer knowledge correctly, we propose query filtering and
association (QFA), which filters irrelevant queries to exact instances. Our KD
with QFA increases the robustness of feature matching by encoding
object-centric features from a single frame supplemented by long-range global
information. We also propose a simple data augmentation scheme for knowledge
distillation in the VIS task that fairly transfers the knowledge of all classes
into the online model. Extensive experiments show that our method significantly
improves the performance in video instance segmentation, especially for
challenging datasets including long, dynamic sequences. Our method also
achieves state-of-the-art performance on YTVIS-21, YTVIS-22, and OVIS datasets,
with mAP scores of 46.1%, 43.6%, and 31.1%, respectively
A Study on the Generality of Neural Network Structures for Monocular Depth Estimation
Monocular depth estimation has been widely studied, and significant
improvements in performance have been recently reported. However, most previous
works are evaluated on a few benchmark datasets, such as KITTI datasets, and
none of the works provide an in-depth analysis of the generalization
performance of monocular depth estimation. In this paper, we deeply investigate
the various backbone networks (e.g.CNN and Transformer models) toward the
generalization of monocular depth estimation. First, we evaluate
state-of-the-art models on both in-distribution and out-of-distribution
datasets, which have never been seen during network training. Then, we
investigate the internal properties of the representations from the
intermediate layers of CNN-/Transformer-based models using synthetic
texture-shifted datasets. Through extensive experiments, we observe that the
Transformers exhibit a strong shape-bias rather than CNNs, which have a strong
texture-bias. We also discover that texture-biased models exhibit worse
generalization performance for monocular depth estimation than shape-biased
models. We demonstrate that similar aspects are observed in real-world driving
datasets captured under diverse environments. Lastly, we conduct a dense
ablation study with various backbone networks which are utilized in modern
strategies. The experiments demonstrate that the intrinsic locality of the CNNs
and the self-attention of the Transformers induce texture-bias and shape-bias,
respectively.Comment: Accepted in TPAM
Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency
We present an end-to-end joint training framework that explicitly models
6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular
camera setup without supervision. Our technical contributions are three-fold.
First, we highlight the fundamental difference between inverse and forward
projection while modeling the individual motion of each rigid object, and
propose a geometrically correct projection pipeline using a neural forward
projection module. Second, we design a unified instance-aware photometric and
geometric consistency loss that holistically imposes self-supervisory signals
for every background and object region. Lastly, we introduce a general-purpose
auto-annotation scheme using any off-the-shelf instance segmentation and
optical flow models to produce video instance segmentation maps that will be
utilized as input to our training pipeline. These proposed elements are
validated in a detailed ablation study. Through extensive experiments conducted
on the KITTI and Cityscapes dataset, our framework is shown to outperform the
state-of-the-art depth and motion estimation methods. Our code, dataset, and
models are available at https://github.com/SeokjuLee/Insta-DM .Comment: Accepted to AAAI 2021. Code/dataset/models are available at
https://github.com/SeokjuLee/Insta-DM. arXiv admin note: substantial text
overlap with arXiv:1912.0935
Implicit Neural Image Stitching With Enhanced and Blended Feature Reconstruction
Existing frameworks for image stitching often provide visually reasonable
stitchings. However, they suffer from blurry artifacts and disparities in
illumination, depth level, etc. Although the recent learning-based stitchings
relax such disparities, the required methods impose sacrifice of image
qualities failing to capture high-frequency details for stitched images. To
address the problem, we propose a novel approach, implicit Neural Image
Stitching (NIS) that extends arbitrary-scale super-resolution. Our method
estimates Fourier coefficients of images for quality-enhancing warps. Then, the
suggested model blends color mismatches and misalignment in the latent space
and decodes the features into RGB values of stitched images. Our experiments
show that our approach achieves improvement in resolving the low-definition
imaging of the previous deep image stitching with favorable accelerated
image-enhancing methods. Our source code is available at
https://github.com/minshu-kim/NIS
Foraging Behavior and Diet of Northward Migrating Bar-Tailed Godwits (Limosa laponica) and Great Knots (Calidris tenuirostris) at a Key Stopover Site
After the completion of the Saemangeum reclamation, the Geum Estuary has emerged as a key feeding and rooting site (staging site) for shorebirds in South Korea. However, there has yet to be any study conducted on the behavior and diet of shorebirds in this region. In this study, we first compared behavior and diet of two representative shorebird species with different morphologies, Bar-tailed Godwits (Limosa laponica) and Great Knots (Calidris tenuirostris) that co-occur in the tidal flat of Yubu Island near the Geum Estuary. During the April to May of 2016, using a camera equipped with a telescope, we recorded 36 individuals during low tide. Behavior was significantly different between the two shorebird species (Chi-square test, P = 0.037), although feeding success rate was similar between the two species (ANOVA, P > 0.05); the Bar-tailed Godwit showed more probings and less peckings than the Great Knot. In addition, Bar-tailed Godwits walked less (ANOVA, P < 0.001) and were less alert (ANOVA, P < 0.005) than Great Knots. On the other hand, diet composition was significantly different between them (Chi-square test, P = 0.010); Both species fed mainly upon Mollusca but the Great Knot fed on noticeably more Annelida (lugworms) than the Bar-tailed Godwit. Among Mollusca, both species consumed more gastropods than bivalves (ANOVA, P < 0.001).Based on these results, it is thought that Bar-tailed Godwits may save energy through less pecking together with less walking and less being alert, since it fed on less food than Great Knots. These results on behavior and diet will be helpful for conservation of the two species in the Geum Estuary tidal flats, a key stopover site of many shorebirds in the East Asian-Australasian Flyway (EAAF)
Whole Genome Analysis of Lactobacillus plantarum Strains Isolated From Kimchi and Determination of Probiotic Properties to Treat Mucosal Infections by Candida albicans and Gardnerella vaginalis
Three Lactobacillus plantarum strains ATG-K2, ATG-K6, and ATG-K8 were isolated from Kimchi, a Korean traditional fermented food, and their probiotic potentials were examined. All three strains were free of antibiotic resistance, hemolysis, and biogenic amine production and therefore assumed to be safe, as supported by whole genome analyses. These strains demonstrated several basic probiotic functions including a wide range of antibacterial activity, bile salt hydrolase activity, hydrogen peroxide production, and heat resistance at 70°C for 60 s. Further studies of antimicrobial activities against Candida albicans and Gardnerella vaginalis revealed growth inhibitory effects from culture supernatants, coaggregation effects, and killing effects of the three probiotic strains, with better efficacy toward C. albicans. In vitro treatment of bacterial lysates of the probiotic strains to the RAW264.7 murine macrophage cell line resulted in innate immunity enhancement via IL-6 and TNF-α production without lipopolysaccharide (LPS) treatment and anti-inflammatory effects via significantly increased production of IL-10 when co-treated with LPS. However, the degree of probiotic effect was different for each strain as the highest TNF-α and the lowest IL-10 production by the RAW264.7 cell were observed in the K8 lysate treated group compared to the K2 and K6 lysate treated groups, which may be related to genomic differences such as chromosome size (K2: 3,034,884 bp, K6: 3,205,672 bp, K8: 3,221,272 bp), plasmid numbers (K2: 3, K6 and K8: 1), or total gene numbers (K2: 3,114, K6: 3,178, K8: 3,186). Although more correlative inspections to connect genomic information and biological functions are needed, genomic analyses of the three strains revealed distinct genomic compositions of each strain. Also, this finding suggests genome level analysis may be required to accurately identify microorganisms. Nevertheless, L. plantarum ATG-K2, ATG-K6, and ATG-K8 demonstrated their potential as probiotics for mucosal health improvement in both microbial and immunological contexts
Profiling age-related epigenetic markers of stomach adenocarcinoma in young and old subjects
The purpose of our study is to identify epigenetic markers that are differently expressed in the stomach adenocarcinoma (STAD) condition. Based on data from The Cancer Genome Atlas (TCGA), we were able to detect an age-related difference in methylation patterns and changes in gene and miRNA expression levels in young (n = 14) and old (n = 70) STAD subjects. Our analysis identified 323 upregulated and 653 downregulated genes in old STAD subjects. We also found 76 miRNAs with age-related expression patterns and 113 differentially methylated genes (DMGs), respectively. Our further analysis revealed that significant upregulated genes (n = 35) were assigned to the cell cycle, while the muscle system process (n = 27) and cell adhesion-related genes (n = 57) were downregulated. In addition, by comparing gene and miRNA expression with methylation change, we identified that three upregulated genes (ELF3, IL1??, and MMP13) known to be involved in inflammatory responses and cell growth were significantly hypomethylated in the promoter region. We further detected target candidates for age-related, downregulated miRNAs (hsa-mir-124-3, hsa-mir-204, and hsa-mir-125b-2) in old STAD subjects. This is the first report of the results from a study exploring age-related epigenetic biomarkers of STAD using high-throughput data and provides evidence for a complex clinicopathological condition expressed by the age-related STAD progression. &copy; the authors, publisher and licensee Libertas Academica Limitedopen
- …