Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition

Recent studies show that vision models pre-trained in generic visual learning tasks with large-scale data can provide useful feature representations for a wide range of visual perception problems. However, few attempts have been made to exploit pre-trained foundation models in visual place recognition (VPR). Due to the inherent difference in training objectives and data between the tasks of model pre-training and VPR, how to bridge the gap and fully unleash the capability of pre-trained models for VPR is still a key issue to address. To this end, we propose a novel method to realize seamless adaptation of pre-trained models for VPR. Specifically, to obtain both global and local features that focus on salient landmarks for discriminating places, we design a hybrid adaptation method to achieve both global and local adaptation efficiently, in which only lightweight adapters are tuned without adjusting the pre-trained model. Besides, to guide effective adaptation, we propose a mutual nearest neighbor local feature loss, which ensures proper dense local features are produced for local matching and avoids time-consuming spatial verification in re-ranking. Experimental results show that our method outperforms the state-of-the-art methods with less training data and training time, and uses about only 3% retrieval runtime of the two-stage VPR methods with RANSAC-based spatial verification. It ranks 1st on the MSLS challenge leaderboard (at the time of submission). The code is released at https://github.com/Lu-Feng/SelaVPR.Comment: ICLR202

Deep Homography Estimation for Visual Place Recognition

Visual place recognition (VPR) is a fundamental task for many applications such as robot localization and augmented reality. Recently, the hierarchical VPR methods have received considerable attention due to the trade-off between accuracy and efficiency. They usually first use global features to retrieve the candidate images, then verify the spatial consistency of matched local features for re-ranking. However, the latter typically relies on the RANSAC algorithm for fitting homography, which is time-consuming and non-differentiable. This makes existing methods compromise to train the network only in global feature extraction. Here, we propose a transformer-based deep homography estimation (DHE) network that takes the dense feature map extracted by a backbone network as input and fits homography for fast and learnable geometric verification. Moreover, we design a re-projection error of inliers loss to train the DHE network without additional homography labels, which can also be jointly trained with the backbone network to help it extract the features that are more suitable for local matching. Extensive experiments on benchmark datasets show that our method can outperform several state-of-the-art methods. And it is more than one order of magnitude faster than the mainstream hierarchical VPR methods using RANSAC. The code is released at https://github.com/Lu-Feng/DHE-VPR.Comment: Accepted by AAAI202

SNP-S3: Shared Network Pre-training and Significant Semantic Strengthening for Various Video-Text Tasks

We present a framework for learning cross-modal video representations by directly pre-training on raw data to facilitate various downstream video-text tasks. Our main contributions lie in the pre-training framework and proxy tasks. First, based on the shortcomings of two mainstream pixel-level pre-training architectures (limited applications or less efficient), we propose Shared Network Pre-training (SNP). By employing one shared BERT-type network to refine textual and cross-modal features simultaneously, SNP is lightweight and could support various downstream applications. Second, based on the intuition that people always pay attention to several "significant words" when understanding a sentence, we propose the Significant Semantic Strengthening (S3) strategy, which includes a novel masking and matching proxy task to promote the pre-training performance. Experiments conducted on three downstream video-text tasks and six datasets demonstrate that, we establish a new state-of-the-art in pixel-level video-text pre-training; we also achieve a satisfactory balance between the pre-training efficiency and the fine-tuning performance. The codebase are available at https://github.com/alipay/Ant-Multi-Modal-Framework/tree/main/prj/snps3_vtp.Comment: Accepted by TCSVT (IEEE Transactions on Circuits and Systems for Video Technology

The Ninth Visual Object Tracking VOT2021 Challenge Results

acceptedVersionPeer reviewe

Epigenetics Recording Varied Environment and Complex Cell Events is an Origin of Cellular Aging

Although the phenomenal relationship between epigenetics and aging phenotypic changes is built up, an intrinsic connection between the epigenetics and aging requires to be theoretically illuminated. In this study, we propose epigenetic recording of varied cell environment and complex history could be an origin of cellular aging. Through epigenetic modifications, the environment and historical events can induce the chromatin template into activated or repressive accessible structure, thereby shaping the DNA template into a spectrum of chromatin states. The inner nature of diversity and conflicts born by cell environment and its historical events are hence recorded into the chromatin template. This could result in a dissipated spectrum of chromatin state and chaos of overall gene expressions. An unavoidable degradation of epigenome entropy, similar to Shannon entropy, would be consequently induced. The resulted disorder in epigenome, characterized by corrosion of epigenome entropy as reflected in chromatin template, can be stably memorized and propagated through cell divisions. Furthermore, hysteresis nature of epigenetics responding to emerging environment could exacerbate the degradation of epigenome entropy. Besides stochastic errors, we propose that epigenetics disorder and chaos derived from unordered environment and complex cell experiences play an essential role in epigenetic drift and the as-resulted cellular aging

Epigenetic Dynamics and Regulation of Plant Male Reproduction

Flowering plant male germlines develop within anthers and undergo epigenetic reprogramming with dynamic changes in DNA methylation, chromatin modifications, and small RNAs. Profiling the epigenetic status using different technologies has substantially accumulated information on specific types of cells at different stages of male reproduction. Many epigenetically related genes involved in plant gametophyte development have been identified, and the mutation of these genes often leads to male sterility. Here, we review the recent progress on dynamic epigenetic changes during pollen mother cell differentiation, microsporogenesis, microgametogenesis, and tapetal cell development. The reported epigenetic variations between male fertile and sterile lines are summarized. We also summarize the epigenetic regulation-associated male sterility genes and discuss how epigenetic mechanisms in plant male reproduction can be further revealed

Chemical, Energetic, and Structural Characteristics of Hydrothermal Carbonization Solid Products for Lawn Grass

Hydrothermal carbonization (HTC) of lawn grass was carried out at 200 °C and 240 °C for 30 to 180 min. The chemical, energetic, and structural characteristics of HTC solid residues were investigated. Results from HTC experiments indicate that solid mass yield of all solid residues was 31 to 50%. The hydrogen/carbon (H/C) and oxygen/carbon (O/C) atomic ratios of all solid residues were 1.17 to 1.64 and 0.45 to 0.65, respectively. The higher heating value (HHV) increased up to 20.54 MJ/kg with increasing HTC residence time at 240 °C for 180 min. Both XRD patterns and FTIR spectra show that differences occur with samples treated as compared to the raw material. Solid hydrochar exhibited higher ordered structure characteristics and was mainly derived from amorphous components degradation when the residence time was increased from 30 to 180 min at 200 °C, while hydrochar formed from cellulose components degradation with increased residence time at 240 °C. According to the results studied, it was found that prolonged residence time was favorable to the formation of hydrochar from lawn grass

A Case Report of X-Linked Hyperimmunoglobulin M Syndrome with Lipoma Arborescens of Knees

The X-linked hyperimmunoglobulin M syndrome (HIGM), caused by mutations in the CD40LG gene, is a kind of primary immunodeficiency disease (PID). Patients with X-linked HIGM are susceptible to infection as well as autoimmune diseases. Lipoma arborescens (LA) is a rare benign tumor, of which the pathogenesis mechanism has not been clearly understood. We report a case of HIGM combined with LA in a 22-year-old male patient. A new deletion mutation of CD40LG gene was detected in this case. The possible relationship between HIGM and LA was also discussed

A wide survey of heavy metals-induced in-vitro DNA replication stress characterized by rate-limited replication

Heavy metals (HMs) are environmental pollutants that pose a threat to human health and have been accepted to cause various diseases, including cancer and developmental disorders. DNA replication stress has been identified to be associated with such diseases. However, the effect of HMs exclusively on DNA replication stress is still not well understood. In this study, DNA replication stress induced by thirteen HMs was assessed using a simplified in-vitro DNA replication model. Two parameters, Cte/Ctc reflecting the cycle threshold value alteration and Ke/Kc reflecting the linear phase slope change, were calculated based on the DNA replication amplification curve to evaluate the rate of exponential and linear phases. These parameters were used to detect the replication rate reflecting in-vitro DNA replication stress induced by tested HMs. According to the effective concentrations and rate-limiting degree, HMs were ranked as follows: Hg, Ce > Pb > Zn > Cr > Cd > Co > Fe > Mn, Cu, Bi, Sr, Ni. Additionally, EDTA could relieve the DNA replication stress induced by some HMs. In conclusion, this study highlights the potential danger of HMs themselves on DNA replication and provides new insight into the possible links between HMs and DNA replication-related diseases

Genetic structure and molecular mechanism underlying the stalk lodging traits in maize (Zea mays L.)

Stalk lodging seriously affects yield and quality of crops, and it can be caused by several factors, such as environments, developmental stages, and internal chemical components of plant stalks. Breeding of stalk lodging-resistant varieties is thus an important task for maize breeders. To better understand the genetic basis underlying stalk lodging resistance, several methods such as quantitative trait locus (QTL) mapping and genome-wide association study (GWAS) have been used to mine potential gene resources. Based on different types of genetic populations and mapping methods, many significant loci associated with stalk lodging resistance have been identified so far. However, few work has been performed to compare and integrate these reported genetic loci. In this study, we first collected hundreds of QTLs and quantitative trait nucleotides (QTNs) related to stalk lodging traits in maize. Then we mapped and integrated the QTLs and QTNs in maize genome to identify overlapped hotspot regions. Based on the genomic confidence intervals harboring these overlapped hotspot regions, we predicted candidate genes related to stalk lodging traits. Meanwhile, we mapped reported genes to these hotspot regions. Finally, we constructed molecular regulatory networks underlying stalk lodging resistance in maize. Collectively, this study provides not only useful genetic loci for deeply exploring molecular mechanisms of stalk lodging resistance traits, but also potential candidate genes and targeted strategies for improving stalk lodging resistance to increase crop yields in future