Retrieval Oriented Masking Pre-training Language Model for Dense Passage Retrieval

Pre-trained language model (PTM) has been shown to yield powerful text representations for dense passage retrieval task. The Masked Language Modeling (MLM) is a major sub-task of the pre-training process. However, we found that the conventional random masking strategy tend to select a large number of tokens that have limited effect on the passage retrieval task (e,g. stop-words and punctuation). By noticing the term importance weight can provide valuable information for passage retrieval, we hereby propose alternative retrieval oriented masking (dubbed as ROM) strategy where more important tokens will have a higher probability of being masked out, to capture this straightforward yet essential information to facilitate the language model pre-training process. Notably, the proposed new token masking method will not change the architecture and learning objective of original PTM. Our experiments verify that the proposed ROM enables term importance information to help language model pre-training thus achieving better performance on multiple passage retrieval benchmarks.Comment: Search LM part of the "AliceMind SLM + HLAR" method in MS MARCO Passage Ranking Leaderboard Submissio

Densovirus is a mutualistic symbiont of a global crop pest (Helicoverpa armigera) and protects against a baculovirus and Bt biopesticide

Mutualistic associations between symbiotic bacteria and their hosts are common within insect systems. However, viruses are often considered as pathogens even though some have been reported to be beneficial to their hosts. Herein, we report a novel densovirus, Helicoverpa armigera densovirus-1 (HaDNV-1) that appears to be beneficial to its host. HaDNV-1 was found to be widespread in wild populations of H. armigera adults (.67% prevalence between 2008 and 2012). In wild larval populations, there was a clear negative interaction between HaDNV-1 and H. armigera nucleopolyhedrovirus (HaNPV), a baculovirus that is widely used as a biopesticide. Laboratory bioassays revealed that larvae hosting HaDNV-1 had significantly enhanced resistance to HaNPV (and lower viral loads), and that resistance to Bacillus thuringiensis (Bt) toxin was also higher at low doses. Laboratory assays indicated that the virus was mainly distributed in the fat body, and could be both horizontally- and vertically-transmitted, though the former occurred only at large challenge doses. Densovirus-positive individuals developed more quickly and had higher fecundity than uninfected insects. We found no evidence for a negative effect of HaDNV-1 infection on H. armigera fitness-related traits, strongly suggesting a mutualistic interaction between the cotton bollworm and its densovirus

Crack healing utilising bacterial spores in concrete

This self repair system is based upon harmless ground borne bacteria as the self healing agent. The bacteria is activated after the concrete is cracked and the bacterial spores are exposed to moisture and air. The bacterial reproduction process creates a calcite by-product which fills the cracks in the concrete. By sealing the cracks in concrete, an effective barrier to air or liquid borne deleterious materials is formed and as a consequence of his, enhanced durability is achieved in the structure, resulting in lower life cycle costs. The concrete/mortar prisms were cracked and tested for water flow. They were then left for 56 days to heal and were subject to a test for water tightness. Healing was observed and a reduced water flow (74% and 32% healed) measured with the healed samples when compared to the specimens that were cracked and subjected to a water flow test without any healing agent. The number of samples were limited and a larger scale test is recommended for further work, however this is proof of concept of the process of healing and testing

A dicistrovirus increases pupal mortality in Spodoptera frugiperda by suppressing protease activity and inhibiting larval diet consumption

Understanding interactions between viruses and their hosts is conducive to enabling better application of viruses as biocontrol agents. Certain viruses carried by parasitic wasps enhance the parasitic efficiency of wasp-larvae by protecting them against the immune system of their Lepidopteran host. However, the relationship between prey pests and viruses found in predatory natural enemies remains unclear. Herein, we report the interaction between Arma chinensis virus-1 (AcV-1), originally isolated from a predatory natural enemy, Arma chinensis (Hemiptera: Pentatomidae), and one of its prey species, Spodoptera frugiperda (Lepidoptera: Noctuidae). The results showed that the AcV-1 virus appeared harmful to the novel host S. frugiperda by inhibiting larval diet consumption and increasing pupal mortality. Meanwhile, sequencing data indicated that the virus altered the gene expression profiles of S. frugiperda. KEGG analysis showed that the proteasome and phagosome pathways related to protein degradation and immune response were significantly enriched. Although the expression levels of digestive enzyme genes did not change significantly, the total protease activity of AcV-1 virus-positive individuals was significantly decreased, suggesting that the virus inhibited diet consumption of S. frugiperda via the down-regulation of digestive enzyme activities. These results indicate that a virus initially isolated in a predatory natural enemy can decrease the fitness of its prey species. The virus was found to impact the host proteasome and phagosome pathways related to protein degradation and immunity, providing a potential mechanism to enhance controlling efficiency.</p