7,854 research outputs found
QuesNet: A Unified Representation for Heterogeneous Test Questions
Understanding learning materials (e.g. test questions) is a crucial issue in
online learning systems, which can promote many applications in education
domain. Unfortunately, many supervised approaches suffer from the problem of
scarce human labeled data, whereas abundant unlabeled resources are highly
underutilized. To alleviate this problem, an effective solution is to use
pre-trained representations for question understanding. However, existing
pre-training methods in NLP area are infeasible to learn test question
representations due to several domain-specific characteristics in education.
First, questions usually comprise of heterogeneous data including content text,
images and side information. Second, there exists both basic linguistic
information as well as domain logic and knowledge. To this end, in this paper,
we propose a novel pre-training method, namely QuesNet, for comprehensively
learning question representations. Specifically, we first design a unified
framework to aggregate question information with its heterogeneous inputs into
a comprehensive vector. Then we propose a two-level hierarchical pre-training
algorithm to learn better understanding of test questions in an unsupervised
way. Here, a novel holed language model objective is developed to extract
low-level linguistic features, and a domain-oriented objective is proposed to
learn high-level logic and knowledge. Moreover, we show that QuesNet has good
capability of being fine-tuned in many question-based tasks. We conduct
extensive experiments on large-scale real-world question data, where the
experimental results clearly demonstrate the effectiveness of QuesNet for
question understanding as well as its superior applicability
AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks
The fields of both Natural Language Processing (NLP) and Automated Machine
Learning (AutoML) have achieved remarkable results over the past years. In NLP,
especially Large Language Models (LLMs) have experienced a rapid series of
breakthroughs very recently. We envision that the two fields can radically push
the boundaries of each other through tight integration. To showcase this
vision, we explore the potential of a symbiotic relationship between AutoML and
LLMs, shedding light on how they can benefit each other. In particular, we
investigate both the opportunities to enhance AutoML approaches with LLMs from
different perspectives and the challenges of leveraging AutoML to further
improve LLMs. To this end, we survey existing work, and we critically assess
risks. We strongly believe that the integration of the two fields has the
potential to disrupt both fields, NLP and AutoML. By highlighting conceivable
synergies, but also risks, we aim to foster further exploration at the
intersection of AutoML and LLMs
- …