TIER-A: Denoising Learning Framework for Information Extraction

With the development of deep neural language models, great progress has been made in information extraction recently. However, deep learning models often overfit on noisy data points, leading to poor performance. In this work, we examine the role of information entropy in the overfitting process and draw a key insight that overfitting is a process of overconfidence and entropy decreasing. Motivated by such properties, we propose a simple yet effective co-regularization joint-training framework TIER-A, Aggregation Joint-training Framework with Temperature Calibration and Information Entropy Regularization. Our framework consists of several neural models with identical structures. These models are jointly trained and we avoid overfitting by introducing temperature and information entropy regularization. Extensive experiments on two widely-used but noisy datasets, TACRED and CoNLL03, demonstrate the correctness of our assumption and the effectiveness of our framework

Global vegetation variability and its response to elevated CO2, global warming, and climate variability – a study using the offline SSiB4/TRIFFID model and satellite data

Abstract. The climate regime shift during the 1980s had a substantial impact on the terrestrial ecosystems and vegetation at different scales. However, the mechanisms driving vegetation changes, before and after the shift, remain unclear. In this study, we used a biophysical-dynamic vegetation model to estimate large-scale trends in terms of carbon fixation, vegetation growth, and expansion during the period 1958–2007, and to attribute these changes to environmental drivers including elevated atmospheric CO2 concentration (hereafter eCO2), global warming, and climate variability (hereafter CV). Simulated Leaf Area Index (LAI) and Gross Primary Product (GPP) were evaluated against observation-based data. Significant spatial correlations are found (correlations > 0.87), along with regionally varying temporal correlations of 0.34–0.80 for LAI and 0.45–0.83 for GPP. More than 40 % of the global land area shows significant trends in LAI and GPP since the 1950s: 11.7 % and 19.3 % of land has consistently positive LAI and GPP trends, respectively; while 17.1 % and 20.1 % of land, saw LAI and GPP trends respectively, reverse during the 1980s. Vegetation fraction cover (FRAC) trends, representing vegetation expansion/shrinking, are found at the edges of semi-arid areas and polar areas. Overall, eCO2 consistently contributes to positive LAI and GPP trends in the tropics. Global warming is shown to mostly affected LAI, with positive effects in high latitudes and negative effects in subtropical semi-arid areas. CV is found to dominate the variability of FRAC, LAI, and GPP in the semi-humid and semi-arid areas. The eCO2 and global warming effects increased after the 1980s, while the CV effect reversed during the 1980s. In addition, plant competition is shown to have played an important role in determining which driver dominated the regional trends. This paper presents a new insight into ecosystem variability and changes in the varying climate since the 1950s

Altering nodes types in controlling complex networks

Controlling a complex network towards a desired state is of great importance in many applications. A network can be controlled by inputting suitable external signals into some selected nodes, which are called driver nodes. Previous works found there exist two control modes in dense networks: distributed and centralized modes. For networks with the distributed mode, most of the nodes can be act as driver nodes; and those with the centralized mode, most of the nodes never be the driver nodes. Here we present an efficient algorithm to change the control type of nodes, from input nodes to redundant nodes, which is done by reversing edges of the network. We conclude four possible cases when reversing an edge and show the control mode can be changed by reversing very few in-edges of driver nodes. We evaluate the performance of our algorithm on both synthetic and real networks. The experimental results show that the control mode of a network can be easily changed by reversing a few elaborately selected edges, and the number of possible driver nodes is dramatically decreased. Our methods provide the ability to design the desired control modes of the network for different control scenarios, which may be used in many application regions

Hyper-Relational Knowledge Graph Neural Network for Next POI

With the advancement of mobile technology, Point of Interest (POI) recommendation systems in Location-based Social Networks (LBSN) have brought numerous benefits to both users and companies. Many existing works employ Knowledge Graph (KG) to alleviate the data sparsity issue in LBSN. These approaches primarily focus on modeling the pair-wise relations in LBSN to enrich the semantics and thereby relieve the data sparsity issue. However, existing approaches seldom consider the hyper-relations in LBSN, such as the mobility relation (a 3-ary relation: user-POI-time). This makes the model hard to exploit the semantics accurately. In addition, prior works overlook the rich structural information inherent in KG, which consists of higher-order relations and can further alleviate the impact of data sparsity.To this end, we propose a Hyper-Relational Knowledge Graph Neural Network (HKGNN) model. In HKGNN, a Hyper-Relational Knowledge Graph (HKG) that models the LBSN data is constructed to maintain and exploit the rich semantics of hyper-relations. Then we proposed a Hypergraph Neural Network to utilize the structural information of HKG in a cohesive way. In addition, a self-attention network is used to leverage sequential information and make personalized recommendations. Furthermore, side information, essential in reducing data sparsity by providing background knowledge of POIs, is not fully utilized in current methods. In light of this, we extended the current dataset with available side information to further lessen the impact of data sparsity. Results of experiments on four real-world LBSN datasets demonstrate the effectiveness of our approach compared to existing state-of-the-art methods

ELIP: Efficient Language-Image Pre-training with Fewer Vision Tokens

Learning a versatile language-image model is computationally prohibitive under a limited computing budget. This paper delves into the \emph{efficient language-image pre-training}, an area that has received relatively little attention despite its importance in reducing computational cost and footprint. To that end, we propose a vision token pruning and merging method ELIP, to remove less influential tokens based on the supervision of language outputs. Our method is designed with several strengths, such as being computation-efficient, memory-efficient, and trainable-parameter-free, and is distinguished from previous vision-only token pruning approaches by its alignment with task objectives. We implement this method in a progressively pruning manner using several sequential blocks. To evaluate its generalization performance, we apply ELIP to three commonly used language-image pre-training models and utilize public image-caption pairs with 4M images for pre-training. Our experiments demonstrate that with the removal of ~30$\%$ vision tokens across 12 ViT layers, ELIP maintains significantly comparable performance with baselines ($\sim$0.32 accuracy drop on average) over various downstream tasks including cross-modal retrieval, VQA, image captioning, \emph{etc}. In addition, the spared GPU resources by our ELIP allow us to scale up with larger batch sizes, thereby accelerating model pre-training and even sometimes enhancing downstream model performance

A novel control system design for automatic feed drilling operation of the PLC-based oil rig

Aiming at the difficulties in realizing the accurate control due to the nonlinearity of the automatic drilling system of oil drilling rig, a design scheme is proposed by giving a constant drilled-pressure to the rig for fuzzy control. Sampling error with changes in the signal was sent into the fuzzy controller, which turned the signal into a fuzzy volume. Subsequently, a precise volume was obtained accordingly and then added to an actuator for the motor control. According to the MATLAB simulation results, the response could be faster and more stable compared with the traditional control

Evaluate What You Can't Evaluate: Unassessable Quality for Generated Response

LLMs (large language models) such as ChatGPT have shown remarkable language understanding and generation capabilities. Although reference-free evaluators based on LLMs show better human alignment than traditional reference-based evaluators, there are many challenges in using reference-free evaluators based on LLMs. Reference-free evaluators are more suitable for open-ended examples with different semantics responses. But not all examples are open-ended. For closed-ended examples with unique correct semantic response, reference-free evaluators will still consider it high quality when giving a response that is inconsistent with the facts and the semantic of reference. In order to comprehensively evaluate the reliability of evaluators based on LLMs, we construct two adversarial meta-evaluation dialogue generation datasets KdConv-ADV and DSTC7-ADV based on KdConv and DSTC7-AVSD, respectively. Compared to previous meta-evaluation benchmarks, KdConv-ADV and DSTC7-ADV are much more challenging since they requires evaluators to be able to reasonably evaluate closed-ended examples with the help of external knowledge or even its own knowledge. Empirical results show that the ability of LLMs to identify unreasonable responses is insufficient. There are risks in using eference-free evaluators based on LLMs to evaluate the quality of dialogue responses.Comment: preprin