591,994 research outputs found

    Two knowledge-based methods for High-Performance Sense Distribution Learning

    Get PDF
    Knowing the correct distribution of senses within a corpus can potentially boost the performance of Word Sense Disambiguation (WSD) systems by many points. We present two fully automatic and language-independent methods for computing the distribution of senses given a raw corpus of sentences. Intrinsic and extrinsic evaluations show that our methods outperform the current state of the art in sense distribution learning and the strongest baselines for the most frequent sense in multiple languages and on domain-specific test sets. Our sense distributions are available at http://trainomatic.org

    An Adaptive Deep Learning for Causal Inference Based on Support Points With High-Dimensional Data

    Get PDF
    The Sample splitting method in semiparametric statistics could introduce inconsistency in inference and estimation. Thus, to make adaptive learning based on observational data and establish valid learning that helps in the estimation and inference of the parameters and hyperparameters using double machine learning, this study introduces an efficient sample splitting technique for causal inference in the semiparametric framework, in other words, the support points sample splitting( SPSS), a subsampling method based on the energy distance concept is employed for causal inference under double machine learning paradigm. This work is based on the idea that the support points sample splitting (SPSS) is an optimal representative point of the data in a random sample versus the counterpart of random splitting, which implies that the support points sample splitting is an optimal sub-representation of the underlying data generating distribution. To my best knowledge, the conceptual foundation of the support points-based sample splitting is a cutting-edge method of subsampling and the best representation of a full big data set in the sense that the unit structural information of the underlying distribution via the traditional random data splitting is most likely not preserved. Three estimators were applied for double/debiased machine learning causal inference a paradigm that estimates the causal treatment effect from observational data based on machine learning algorithms with the support points sample splitting (SPSS). This study is considering Support Vector Machine (SVM) and Deep Learning (DL) as the predictive estimators. A comparative study is conducted between the SVM and DL with the support points technique to the benchmark results of Chernozhukov et al. (2018) that used instead, the random forest, the neural network, and the regression trees with random k-fold cross-fitting technique. An ensemble machine learning algorithm is proposed that is a hybrid of the super learner and the deep learning with the support points splitting to compare it to the results of Chernozhukov et al. (2018). Finally, a socio-economic real-world dataset, for the 401(k)-pension plan, is used to investigate and evaluate the proposed methods to those in Chernozhukov et al. (2018). The result of this study was under 162 simulations, shows that the three proposed models converge, support vector machine (SVM) with support points sample splitting (SPSS) under double machine learning (DML), the deep learning (DL) with support points sample splitting under double machine learning (DML), and the hybrid of super learning (SL) and deep learning with support points sample splitting under double machine learning. However, the performance of the three models differs. The first model, support vector machine (SVM) with support points sample splitting (SPSS) under double machine learning (DML) has the lowest performance compared to the other two models. In terms of the quality of the causal estimators, it has a higher MSE and inconsistency of the simulation results on all three data dimension levels, low-high-dimensional (p = 20,50,80), moderate-high-dimensional (p = 100, 200, 500), and big-high-dimensional p = (1000, 2000, 5000). The two other models, deep learning (DL) with support points sample splitting under double machine learning (DML), and the hybrid of super learning (SL) and deep learning with support points sample splitting under double machine learning have produced a competing performance and results in terms of the best estimation compared to the two other methods. The first model was time efficient to estimate the causal inference compared to the third one. But the third model was better performing in terms of the estimation quality by producing the lowest MSE compared to the other two models. The results of this research are consistent with the recent development of machine learning. The support vector machine learning has been introduced in the previous century, and it looks like it is no longer showing efficiency and quality estimation with the recent emerging double machine learning. However, cutting-edge methods such as deep learning and super learner have shown superior performance in the estimation of the causal double machine learning target estimator, and efficiency in the time of computation

    A Very Brief Introduction to Machine Learning With Applications to Communication Systems

    Get PDF
    Given the unprecedented availability of data and computing resources, there is widespread renewed interest in applying data-driven machine learning methods to problems for which the development of conventional engineering solutions is challenged by modelling or algorithmic deficiencies. This tutorial-style paper starts by addressing the questions of why and when such techniques can be useful. It then provides a high-level introduction to the basics of supervised and unsupervised learning. For both supervised and unsupervised learning, exemplifying applications to communication networks are discussed by distinguishing tasks carried out at the edge and at the cloud segments of the network at different layers of the protocol stack

    Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation

    Get PDF
    The success of deep learning methods hinges on the availability of large training datasets annotated for the task of interest. In contrast to human intelligence, these methods lack versatility and struggle to learn and adapt quickly to new tasks, where labeled data is scarce. Meta-learning aims to solve this problem by training a model on a large number of few-shot tasks, with an objective to learn new tasks quickly from a small number of examples. In this paper, we propose a meta-learning framework for few-shot word sense disambiguation (WSD), where the goal is to learn to disambiguate unseen words from only a few labeled instances. Meta-learning approaches have so far been typically tested in an NN-way, KK-shot classification setting where each task has NN classes with KK examples per class. Owing to its nature, WSD deviates from this controlled setup and requires the models to handle a large number of highly unbalanced classes. We extend several popular meta-learning approaches to this scenario, and analyze their strengths and weaknesses in this new challenging setting.Comment: Added additional experiment

    Semantic Sort: A Supervised Approach to Personalized Semantic Relatedness

    Full text link
    We propose and study a novel supervised approach to learning statistical semantic relatedness models from subjectively annotated training examples. The proposed semantic model consists of parameterized co-occurrence statistics associated with textual units of a large background knowledge corpus. We present an efficient algorithm for learning such semantic models from a training sample of relatedness preferences. Our method is corpus independent and can essentially rely on any sufficiently large (unstructured) collection of coherent texts. Moreover, the approach facilitates the fitting of semantic models for specific users or groups of users. We present the results of extensive range of experiments from small to large scale, indicating that the proposed method is effective and competitive with the state-of-the-art.Comment: 37 pages, 8 figures A short version of this paper was already published at ECML/PKDD 201

    MUSE: Modularizing Unsupervised Sense Embeddings

    Full text link
    This paper proposes to address the word sense ambiguity issue in an unsupervised manner, where word sense representations are learned along a word sense selection mechanism given contexts. Prior work focused on designing a single model to deliver both mechanisms, and thus suffered from either coarse-grained representation learning or inefficient sense selection. The proposed modular approach, MUSE, implements flexible modules to optimize distinct mechanisms, achieving the first purely sense-level representation learning system with linear-time sense selection. We leverage reinforcement learning to enable joint training on the proposed modules, and introduce various exploration techniques on sense selection for better robustness. The experiments on benchmark data show that the proposed approach achieves the state-of-the-art performance on synonym selection as well as on contextual word similarities in terms of MaxSimC
    • …
    corecore