Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval

Grounded on pre-trained language models (PLMs), dense retrieval has been studied extensively on plain text. In contrast, there has been little research on retrieving data with multiple aspects using dense models. In the scenarios such as product search, the aspect information plays an essential role in relevance matching, e.g., category: Electronics, Computers, and Pet Supplies. A common way of leveraging aspect information for multi-aspect retrieval is to introduce an auxiliary classification objective, i.e., using item contents to predict the annotated value IDs of item aspects. However, by learning the value embeddings from scratch, this approach may not capture the various semantic similarities between the values sufficiently. To address this limitation, we leverage the aspect information as text strings rather than class IDs during pre-training so that their semantic similarities can be naturally captured in the PLMs. To facilitate effective retrieval with the aspect strings, we propose mutual prediction objectives between the text of the item aspect and content. In this way, our model makes more sufficient use of aspect information than conducting undifferentiated masked language modeling (MLM) on the concatenated text of aspects and content. Extensive experiments on two real-world datasets (product and mini-program search) show that our approach can outperform competitive baselines both treating aspect values as classes and conducting the same MLM for aspect and content strings. Code and related dataset will be available at the URL \footnote{https://github.com/sunxiaojie99/ATTEMPT}.Comment: accepted by cikm202

Easy Begun is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropout

Spatial-temporal (ST) graph modeling, such as traffic speed forecasting and taxi demand prediction, is an important task in deep learning area. However, for the nodes in graph, their ST patterns can vary greatly in difficulties for modeling, owning to the heterogeneous nature of ST data. We argue that unveiling the nodes to the model in a meaningful order, from easy to complex, can provide performance improvements over traditional training procedure. The idea has its root in Curriculum Learning which suggests in the early stage of training models can be sensitive to noise and difficult samples. In this paper, we propose ST-Curriculum Dropout, a novel and easy-to-implement strategy for spatial-temporal graph modeling. Specifically, we evaluate the learning difficulty of each node in high-level feature space and drop those difficult ones out to ensure the model only needs to handle fundamental ST relations at the beginning, before gradually moving to hard ones. Our strategy can be applied to any canonical deep learning architecture without extra trainable parameters, and extensive experiments on a wide range of datasets are conducted to illustrate that, by controlling the difficulty level of ST relations as the training progresses, the model is able to capture better representation of the data and thus yields better generalization

Methyltransferase Dnmt3a upregulates HDAC9 to deacetylate the kinase TBK1 for activation of antiviral innate immunity

The DNA methyltransferase Dnmt3a has high expression in terminally differentiated macrophages; however, its role in innate immunity remains unknown. Here we report that deficiency in Dnmt3a selectively impaired the production of type I interferons triggered by pattern-recognition receptors (PRRs), but not that of the proinflammatory cytokines TNF and IL-6. Dnmt3a-deficient mice exhibited enhanced susceptibility to viral challenge. Dnmt3a did not directly regulate the transcription of genes encoding type I interferons; instead, it increased the production of type I interferons through an epigenetic mechanism by maintaining high expression of the histone deacetylase HDAC9. In turn, HDAC9 directly maintained the deacetylation status of the key PRR signaling molecule TBK1 and enhanced its kinase activity. Our data add mechanistic insight into the crosstalk between epigenetic modifications and post-translational modifications in the regulation of PRR signaling and activation of antiviral innate immune responses

High-Performance Polarization Imaging Reconstruction in Scattering System under Natural Light Conditions with an Improved U-Net

Imaging through scattering media faces great challenges. Object information will be seriously degraded by scattering media, and the final imaging quality will be poor. In order to improve imaging quality, we propose using the transmitting characteristics of an object’s polarization information, to achieve imaging through scattering media under natural light using an improved U-net. In this paper, we choose ground glass as the scattering medium and capture the polarization images of targets through the scattering medium by a polarization camera. Experimental results show that the proposed model can reconstruct target information from highly damaged images, and for the same material object, the trained network model has a superior generalization without considering its structural shapes. Meanwhile, we have also investigated the effect of the distance between the target and the ground glass on the reconstructing performance, in which, and although the mismatch distance between the training set and the testing sample expands to 1 cm, the modified U-net can also efficaciously reconstruct the targets

Active Source Free Domain Adaptation

Source free domain adaptation (SFDA) aims to transfer a trained source model to the unlabeled target domain without accessing the source data. However, the SFDA setting faces an effect bottleneck due to the absence of source data and target supervised information, as evidenced by the limited performance gains of newest SFDA methods. In this paper, for the first time, we introduce a more practical scenario called active source free domain adaptation (ASFDA) that permits actively selecting a few target data to be labeled by experts. To achieve that, we first find that those satisfying the properties of neighbor-chaotic, individual-different, and target-like are the best points to select, and we define them as the minimum happy (MH) points. We then propose minimum happy points learning (MHPL) to actively explore and exploit MH points. We design three unique strategies: neighbor ambient uncertainty, neighbor diversity relaxation, and one-shot querying, to explore the MH points. Further, to fully exploit MH points in the learning process, we design a neighbor focal loss that assigns the weighted neighbor purity to the cross-entropy loss of MH points to make the model focus more on them. Extensive experiments verify that MHPL remarkably exceeds the various types of baselines and achieves significant performance gains at a small cost of labeling.Comment: 9 pages (not including references and checklist), 4 figures

A multi-layer fuzzy model based on fuzzy-rule clustering for prediction tasks

Fuzzy systems are widely used for solving complex and non-linear problems that cannot be addressed using precise mathematical models. Their performance, however, is critically affected by how they are constructed as well as their fuzzy rule base. Inspired by neural networks that apply a multi-layer structure to improve their performance, we propose a multi-layer fuzzy model with modified fuzzy rules to improve the approximation ability of fuzzy systems without losing efficiency. In practical applications, the fuzzy rule base extracted from numerical data is often incomplete, which makes a fuzzy system less robust. To address this problem, a non-linear function is used as the consequent of each fuzzy rule based on fuzzy-rule clustering to enhance the approximation ability of the fuzzy rule base. In addition, exact matching of fuzzy rules is employed based on the fuzzy rule's antecedent for prediction. By doing so, only one rule will be triggered in each layer, which is very efficient. Experimental results from two simulated functions and three practical applications confirm that our proposed multi-layer fuzzy model can outperform other well-established fuzzy models in terms of accuracy and robustness without sacrificing efficiency

A Novel Ensemble Learning Approach for Stock Market Prediction Based on Sentiment Analysis and the Sliding Window Method

Financial news disclosures provide valuable information for traders and investors while making stock market investment decisions. Essential but challenging, the stock market prediction problem has attracted significant attention from both researchers and practitioners. Conventional machine learning models often fail to interpret the content of financial news due to the complexity and ambiguity of natural language used in the news. Inspired by the success of recurrent neural networks (RNNs) in sequential data processing, we propose an ensemble RNN approach (long short-term memory, gated recurrent unit, and SimpleRNN) to predict stock market movements. To avoid extracting tens of thousands of features using traditional natural language processing methods, we apply sentiment analysis and the sliding window method to extract only the most representative features. Our experimental results confirm the effectiveness of these two methods for feature extraction and show that the proposed ensemble approach is able to outperform other models under comparison

A Comprehensive Risk Assessment Framework for Inland Waterway Transportation of Dangerous Goods

A framework for risk assessment due to inland waterway transportation of dangerous goods is designed based on all possible event types that may be caused by the inland transportation of dangerous goods. The objective of this study is to design a framework for calculating the risks associated with changes in the transportation of dangerous goods along inland waterways. The framework is based on the traditional definition of risk and is designed for sensitive riverside environmental conditions in inland waterways. From the perspective of transportation management, this paper introduced the concept of transportability of dangerous goods and constructed a transportability assessment framework, which consists of a multi-index evaluation system and a single metric model. The result of the assessment is as an auxiliary basis to determine the transportation permit and control intensity of dangerous goods in an inland waterway specific voyage. The methodology is illustrated using a case study of transporting fireworks in the Yangtze River

Optimization Design And Simulation Of Microgrid In Amdjarass Town, Chad

How to supply electricity to the remote areas has become a very pressing issue for some countries which do not have the ability to connect all power grids to the whole country temporarily. At the same time, with the increase of fossil fuel costs and the continuous development of renewable energy generation technology, the construction of a hybrid renewable energy microgrid system seems to become an economic and technical approach to resolve the power shortage problem in the remote areas of some countries. Based on the analysis of local natural resources and load conditions, this paper designed a microgrid system which contains the wind turbines, PV systems, a diesel generator and an energy storage module to meet the power supply needs of the small town Amdjarass in Chad. Then the authors optimized the capacity of this microgrid and estimated the cost of this system with the utilization of HOMER software. In the end, this paper set the optimal configuration scheme with the target of the lowest COE and analyzed the sensitivity of some important parameters which could affect the economic performance