Query-as-context Pre-training for Dense Passage Retrieval

Recently, methods have been developed to improve the performance of dense passage retrieval by using context-supervised pre-training. These methods simply consider two passages from the same document to be relevant, without taking into account the possibility of weakly correlated pairs. Thus, this paper proposes query-as-context pre-training, a simple yet effective pre-training technique to alleviate the issue. Query-as-context pre-training assumes that the query derived from a passage is more likely to be relevant to that passage and forms a passage-query pair. These passage-query pairs are then used in contrastive or generative context-supervised pre-training. The pre-trained models are evaluated on large-scale passage retrieval benchmarks and out-of-domain zero-shot benchmarks. Experimental results show that query-as-context pre-training brings considerable gains and meanwhile speeds up training, demonstrating its effectiveness and efficiency. Our code will be available at https://github.com/caskcsg/ir/tree/main/cotmae-qc .Comment: EMNLP 2023 Main Conferenc

PUNR: Pre-training with User Behavior Modeling for News Recommendation

News recommendation aims to predict click behaviors based on user behaviors. How to effectively model the user representations is the key to recommending preferred news. Existing works are mostly focused on improvements in the supervised fine-tuning stage. However, there is still a lack of PLM-based unsupervised pre-training methods optimized for user representations. In this work, we propose an unsupervised pre-training paradigm with two tasks, i.e. user behavior masking and user behavior generation, both towards effective user behavior modeling. Firstly, we introduce the user behavior masking pre-training task to recover the masked user behaviors based on their contextual behaviors. In this way, the model could capture a much stronger and more comprehensive user news reading pattern. Besides, we incorporate a novel auxiliary user behavior generation pre-training task to enhance the user representation vector derived from the user encoder. We use the above pre-trained user modeling encoder to obtain news and user representations in downstream fine-tuning. Evaluations on the real-world news benchmark show significant performance improvements over existing baselines.Comment: Accepted by Findings of EMNLP23. Github Repo: https://github.com/ma787639046/pun

Growth patterns and environmental adaptions of the tree species planted for ecological remediation in typhoon-disturbed areas—A case study in Zhuhai, China

Typhoon frequently results in various mechanical damages to urban forest ecosystems. Imperative forest remediation projects were launched to restore the environmental conditions in cities, in which massive trees were newly planted. However, it was rarely answered whether the newly planted trees could acclimate to typhoon circumstances and enhance the wind resistance of the local ecosystem. Therefore, it was necessary to achieve information on the physical growth and windy environmental adaption of newly planted trees, which could promote a profound understanding of the efficiency of post-typhoon ecological remediation. In this study, we selected Zhuhai's urban-forest remediation district as our research area that suffered severely from Typhoon Hato (2017). The six newly-planted tree species for the ecological remediation were measured for their above- and below-ground processes from June 2018 to December 2019, including their development of tree height, ground diameter, crown size, and fine root biomass. Additionally, the variations of the soil's physical and chemical properties were also measured to assess the impact of plantation on soil conditions. Our results showed that the six surveyed tree species had different above- and below-ground growth patterns. With robust root development at horizontal and vertical levels combined with relatively short and thick above-ground profiles, Sterculia lanceolata Cav. and Cinnamomum camphora (Linn) were likely to cope well with typhoon disturbances. Ilex rotunda Thunb. and Schima superba Gardn. et Champ. exhibited moderate acclimation to windy environment, while Elaeocarpus sylvestris (Lour.) Poir. and Elaeocarpus apiculatus Mast. were not recommended to be planted in typhoon-disturbed areas concerning their unstable root development. In addition, the ecological remediation did improve the soil properties, specifically for the chemical characteristics including available nitrogen, available potassium, and soil organic matter. To improve the effectiveness of forest remediation in the future, it was better to choose those tree species with vigorous root development and steady values of root:shoot ratios, which might be advantageous for coping with typhoon disturbances. The tree species with prosperous above-ground growth were not suitable for areas facing strong winds directly but could be planted in leeward regions to amplify their landscape functions

Rehabilitation of Xiaozhou water village

published_or_final_versionArchitectureMasterMaster of Landscape Architectur

Symmetric Metric Learning with Adaptive Margin for Recommendation

Metric learning based methods have attracted extensive interests in recommender systems. Current methods take the user-centric way in metric space to ensure the distance between user and negative item to be larger than that between the current user and positive item by a fixed margin. While they ignore the relations among positive item and negative item. As a result, these two items might be positioned closely, leading to incorrect results. Meanwhile, different users usually have different preferences, the fixed margin used in those methods can not be adaptive to various user biases, and thus decreases the performance as well. To address these two problems, a novel Symmetic Metric Learning with adaptive margin (SML) is proposed. In addition to the current user-centric metric, it symmetically introduces a positive item-centric metric which maintains closer distance from positive items to user, and push the negative items away from the positive items at the same time. Moreover, the dynamically adaptive margins are well trained to mitigate the impact of bias. Experimental results on three public recommendation datasets demonstrate that SML produces a competitive performance compared with several state-of-the-art methods

Laws Governing Free and Actual Drying Shrinkage of 50 mm Thick Mongolian Scotch Pine Timber

The relationships between free shrinkage and actual shrinkage of different layers in Mongolian Scotch pine (Pinus sylvestris var. mongolica Litv.) were explored to provide basic data for the further study of drying shrinkage properties. The free shrinkage coefficients at different temperatures and the actual shrinkage strain of each layer were examined under conventional drying. The results showed high precision of free drying shrinkage of corresponding layers of thin small test strips in each layer of sawn timber. The free shrinkage increased linearly as moisture content declined. At the same temperature, the free shrinkage coefficient reached the largest values for the first layer (above 0.267%), while the smallest values were recorded for the ninth layer (below 0.249%). Except for the ninth layer, the free shrinkage coefficients in width directions of other representative layers decreased as temperature increased. At constant temperature, the difference in free shrinkage coefficient of test materials in the length direction of sawn timber was small for the first layer, but slightly larger and changed irregularly in the fifth and ninth layer direction. At the end of conventional drying, the plastic deformation of each layer in the early stage of drying showed a reducing trend or even reversal due to the effects of reverse stress and later damp heat. In sum, these findings look promising for future optimization of wood drying process

Silica-Coated, Waxberry-like Surface-Enhanced Raman Resonant Scattering Tag-Pair with Near-Infrared Raman Dye Encoding: Toward In Vivo Duplexing Detection

Surface-enhanced Raman resonant scattering (SERRS) tags encoded with near-infrared (NIR) Raman reporters showed great potential for in vivo detection owing to their ultrasensitivity. However, in vivo signal stability of such tags is a remaining problem due to the lack of suitable silica coating method because the weakly adsorbed NIR reporters tend to detach from traditional gold nanosubstrates in the ethanol-rich and high pH conditions, which are commonly used for silica coating. Herein, we propose a silica coating method for NIR SERRS tags by using waxberry-like gold nanoparticles (NPs) as substrates. The lipid bilayer of the NPs played a crucial role in the coating, which can encapsulate the NIR Raman reporter via hydrophobic interactions and prevent the interference from a harsh medium. Thus, the silica-coated tags well preserved ultrasensitivity of bare tags and simultaneously gained satisfactory signal stability in vivo. Moreover, the coating method is compatible for the encapsulation of a variety of thiol group-free NIR reporters (as exemplified by DTTC, Cy7, IR792, and DIR), relying on which a tag-pair with distinguishable peaks can be screened (labeling with DTTC and Cy7, respectively). In vivo duplexing detection revealed that the tag-pair-labeled liposome was cleared faster in the liver than polydopamine NPs within one mouse. The developed method paves an easy way for gaining high-quality SERRS tags and will promote their in vivo multiplex analysis and diagnostics applications