Sentence level relation extraction via relation embedding

Relation extraction is a task of information extraction that extracts semantic relations from text, which usually occur between two named entities. It is a crucial step for converting unstructured text into structured data that forms a knowledge base, so that it may be used to build systems with special purposes such as business decision making and legal case-based reasoning. Relation extraction in sentence-level is the most common type, because relationships can be usually discovered within single sentences. One obvious example is the relationship between the subject and the object. As it has been studied for years, there are various methods for relation extraction such as feature based methods, distant supervision and recurrent neural networks. However, the following problems have been found in these approaches. (i) These methods require large amounts of human labelled data to train the model in order to get high accuracy. (ii) These methods are hard to be applied in real applications, especially in specialised domains where experts are required for both labelling and validating the data. In this thesis, we address these problems in two aspects: academic research and application development. In terms of academic research, we propose models that can be trained with less amount of labelled training data. The first approach trains the relation feature embedding, then it uses the feature embeddings for obtaining relation embeddings. To minimise the effect of designing handcraft features, the second approach adopts RNNs to automatically learn features from the text. In these methods, relation embeddings are reduced to a smaller vector space, and the relations with similar meanings form clusters. Therefore, the model can be trained with a smaller number of labelled data. The last approach adopts seq2seq regularisation, which can improve the accuracy of the relation extraction models. In terms of application development, we construct a prototype web service for searching semantic triples using relations extracted by third-party extraction tools. In the last chapter, we run all our proposed models on real-world legal documents. We also build a web application for extracting relations in legal text based on the trained models, which can help lawyers investigate the key information in legal cases more quickly. We believe that the idea of relation embeddings can be applied in domains that require relation extraction but with limited labelled data

Modulation of Androgen Receptor by FOXA1 and FOXO1 Factors in Prostate Cancer

© Ivyspring International Publisher. This is an open-access article distributed under the terms of the Creative Commons Licens

Learning Data Quality Analytics for Financial Services

Financial institutions put tremendous efforts on the data analytics work associated with the risk data in recent years. Their analytical reports are yet to be accepted by regulators in financial services industry till early 2019. In particular, the enhancement needs to meet the regulatory requirement the APRA CPG 235. To improve data quality, we assist in the data quality analytics by developing a machine learning model to identify current issues and predict future issues. This helps to remediate data as early as possible for the mitigation of risk of re-occurrence. The analytical dimensions are customer related risks (market, credit, operational & liquidity risks) and business segments (private, wholesale & retail banks). The model is implemented with multiple Long Short-Term Memory ( LSTM ) Recurrent Neural Network ( RNNs ) to find the best one for the quality & prediction analytics. They are evaluated by divergent algorithms and cross-validation techniques

Equivariant Transporter Network

Transporter Net is a recently proposed framework for pick and place that is able to learn good manipulation policies from a very few expert demonstrations. A key reason why Transporter Net is so sample efficient is that the model incorporates rotational equivariance into the pick module, i.e. the model immediately generalizes learned pick knowledge to objects presented in different orientations. This paper proposes a novel version of Transporter Net that is equivariant to both pick and place orientation. As a result, our model immediately generalizes place knowledge to different place orientations in addition to generalizing pick knowledge as before. Ultimately, our new model is more sample efficient and achieves better pick and place success rates than the baseline Transporter Net model.Comment: Project Website: https://haojhuang.github.io/etp_page

Leveraging Symmetries in Pick and Place

Robotic pick and place tasks are symmetric under translations and rotations of both the object to be picked and the desired place pose. For example, if the pick object is rotated or translated, then the optimal pick action should also rotate or translate. The same is true for the place pose; if the desired place pose changes, then the place action should also transform accordingly. A recently proposed pick and place framework known as Transporter Net captures some of these symmetries, but not all. This paper analytically studies the symmetries present in planar robotic pick and place and proposes a method of incorporating equivariant neural models into Transporter Net in a way that captures all symmetries. The new model, which we call Equivariant Transporter Net, is equivariant to both pick and place symmetries and can immediately generalize pick and place knowledge to different pick and place poses. We evaluate the new model empirically and show that it is much more sample efficient than the non-symmetric version, resulting in a system that can imitate demonstrated pick and place behavior using very few human demonstrations on a variety of imitation learning tasks.Comment: arXiv admin note: substantial text overlap with arXiv:2202.0940

Elevated CO2 causes different growth stimulation, water- and nitrogen-use efficiencies, and leaf ultrastructure responses in two conifer species under intra- and interspecific competition

The continuously increasing atmospheric carbon dioxide concentration ([CO2]) has substantial effects on plant growth, and on the composition and structure of forests. However, how plants respond to elevated [CO2] (e[CO2]) under intra- and interspecific competition has been largely overlooked. In this study, we employed Abies faxoniana Rehder & Wilson and Picea purpurea Mast. seedlings to explore the effects of e[CO2] (700 p.p.m.) and plant-plant competition on plant growth, physiological and morphological traits, and leaf ultrastructure. We found that e[CO2] stimulated plant growth, photosynthesis and nonstructural carbohydrates (NSC), affected morphological traits and leaf ultrastructure, and enhanced water- and nitrogen (N)- use efficiencies in A. faxoniana and P. purpurea. Under interspecific competition and e[CO2], P. purpurea showed a higher biomass accumulation, photosynthetic capacity and rate of ectomycorrhizal infection, and higher water- and N-use efficiencies compared with A. faxoniana. However, under intraspecific competition and e[CO2], the two conifers showed no differences in biomass accumulation, photosynthetic capacity, and water- and N-use efficiencies. In addition, under interspecific competition and e[CO2], A. faxoniana exhibited higher NSC levels in leaves as well as more frequent and greater starch granules, which may indicate carbohydrate limitation. Consequently, we concluded that under interspecific competition, P. purpurea possesses a positive growth and adjustment strategy (e.g. a higher photosynthetic capacity and rate of ectomycorrhizal infection, and higher water- and N-use efficiencies), while A. faxoniana likely suffers from carbohydrate limitation to cope with rising [CO2]. Our study highlights that plant-plant competition should be taken into consideration when assessing the impact of rising [CO2] on the plant growth and physiological performance.Peer reviewe

Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains

Pre-trained language models have been applied to various NLP tasks with considerable performance gains. However, the large model sizes, together with the long inference time, limit the deployment of such models in real-time applications. One line of model compression approaches considers knowledge distillation to distill large teacher models into small student models. Most of these studies focus on single-domain only, which ignores the transferable knowledge from other domains. We notice that training a teacher with transferable knowledge digested across domains can achieve better generalization capability to help knowledge distillation. Hence we propose a Meta-Knowledge Distillation (Meta-KD) framework to build a meta-teacher model that captures transferable knowledge across domains and passes such knowledge to students. Specifically, we explicitly force the meta-teacher to capture transferable knowledge at both instance-level and feature-level from multiple domains, and then propose a meta-distillation algorithm to learn single-domain student models with guidance from the meta-teacher. Experiments on public multi-domain NLP tasks show the effectiveness and superiority of the proposed Meta-KD framework. Further, we also demonstrate the capability of Meta-KD in the settings where the training data is scarce

YOLOCS: Object Detection based on Dense Channel Compression for Feature Spatial Solidification

In this study, we examine the associations between channel features and convolutional kernels during the processes of feature purification and gradient backpropagation, with a focus on the forward and backward propagation within the network. Consequently, we propose a method called Dense Channel Compression for Feature Spatial Solidification. Drawing upon the central concept of this method, we introduce two innovative modules for backbone and head networks: the Dense Channel Compression for Feature Spatial Solidification Structure (DCFS) and the Asymmetric Multi-Level Compression Decoupled Head (ADH). When integrated into the YOLOv5 model, these two modules demonstrate exceptional performance, resulting in a modified model referred to as YOLOCS. Evaluated on the MSCOCO dataset, the large, medium, and small YOLOCS models yield AP of 50.1%, 47.6%, and 42.5%, respectively. Maintaining inference speeds remarkably similar to those of the YOLOv5 model, the large, medium, and small YOLOCS models surpass the YOLOv5 model's AP by 1.1%, 2.3%, and 5.2%, respectively