37 research outputs found
実応用を志向した機械翻訳システムの設計と評価
Tohoku University博士(情報科学)thesi
Assessing the Ability of Self-Attention Networks to Learn Word Order
Self-attention networks (SAN) have attracted a lot of interests due to their
high parallelization and strong performance on a variety of NLP tasks, e.g.
machine translation. Due to the lack of recurrence structure such as recurrent
neural networks (RNN), SAN is ascribed to be weak at learning positional
information of words for sequence modeling. However, neither this speculation
has been empirically confirmed, nor explanations for their strong performances
on machine translation tasks when "lacking positional information" have been
explored. To this end, we propose a novel word reordering detection task to
quantify how well the word order information learned by SAN and RNN.
Specifically, we randomly move one word to another position, and examine
whether a trained model can detect both the original and inserted positions.
Experimental results reveal that: 1) SAN trained on word reordering detection
indeed has difficulty learning the positional information even with the
position embedding; and 2) SAN trained on machine translation learns better
positional information than its RNN counterpart, in which position embedding
plays a critical role. Although recurrence structure make the model more
universally-effective on learning word order, learning objectives matter more
in the downstream tasks such as machine translation.Comment: ACL 201
Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challenges
Gender inclusivity in language technologies has become a prominent research topic. In this study, we explore gender-neutral translation (GNT) as a form of gender inclusivity and a goal to be achieved by machine translation (MT) models, which have been found to perpetuate gender bias and discrimination. Specifically, we focus on translation from English into Italian, a language pair representative of salient gender-related linguistic transfer problems. To define GNT, we review a selection of relevant institutional guidelines for gender-inclusive language, discuss its scenarios of use, and examine the technical challenges of performing GNT in MT, concluding with a discussion of potential solutions to encourage advancements toward greater inclusivity in MT
MarioGPT: Open-Ended Text2Level Generation through Large Language Models
Procedural Content Generation (PCG) algorithms provide a technique to
generate complex and diverse environments in an automated way. However, while
generating content with PCG methods is often straightforward, generating
meaningful content that reflects specific intentions and constraints remains
challenging. Furthermore, many PCG algorithms lack the ability to generate
content in an open-ended manner. Recently, Large Language Models (LLMs) have
shown to be incredibly effective in many diverse domains. These trained LLMs
can be fine-tuned, re-using information and accelerating training for new
tasks. In this work, we introduce MarioGPT, a fine-tuned GPT2 model trained to
generate tile-based game levels, in our case Super Mario Bros levels. We show
that MarioGPT can not only generate diverse levels, but can be text-prompted
for controllable level generation, addressing one of the key challenges of
current PCG techniques. As far as we know, MarioGPT is the first text-to-level
model. We also combine MarioGPT with novelty search, enabling it to generate
diverse levels with varying play-style dynamics (i.e. player paths). This
combination allows for the open-ended generation of an increasingly diverse
range of content
Evaluating Information Retrieval and Access Tasks
This open access book summarizes the first two decades of the NII Testbeds and Community for Information access Research (NTCIR). NTCIR is a series of evaluation forums run by a global team of researchers and hosted by the National Institute of Informatics (NII), Japan. The book is unique in that it discusses not just what was done at NTCIR, but also how it was done and the impact it has achieved. For example, in some chapters the reader sees the early seeds of what eventually grew to be the search engines that provide access to content on the World Wide Web, today’s smartphones that can tailor what they show to the needs of their owners, and the smart speakers that enrich our lives at home and on the move. We also get glimpses into how new search engines can be built for mathematical formulae, or for the digital record of a lived human life. Key to the success of the NTCIR endeavor was early recognition that information access research is an empirical discipline and that evaluation therefore lay at the core of the enterprise. Evaluation is thus at the heart of each chapter in this book. They show, for example, how the recognition that some documents are more important than others has shaped thinking about evaluation design. The thirty-three contributors to this volume speak for the many hundreds of researchers from dozens of countries around the world who together shaped NTCIR as organizers and participants. This book is suitable for researchers, practitioners, and students—anyone who wants to learn about past and present evaluation efforts in information retrieval, information access, and natural language processing, as well as those who want to participate in an evaluation task or even to design and organize one
Image sensing with multilayer, nonlinear optical neural networks
Optical imaging is commonly used for both scientific and technological
applications across industry and academia. In image sensing, a measurement,
such as of an object's position, is performed by computational analysis of a
digitized image. An emerging image-sensing paradigm breaks this delineation
between data collection and analysis by designing optical components to perform
not imaging, but encoding. By optically encoding images into a compressed,
low-dimensional latent space suitable for efficient post-analysis, these image
sensors can operate with fewer pixels and fewer photons, allowing
higher-throughput, lower-latency operation. Optical neural networks (ONNs)
offer a platform for processing data in the analog, optical domain. ONN-based
sensors have however been limited to linear processing, but nonlinearity is a
prerequisite for depth, and multilayer NNs significantly outperform shallow NNs
on many tasks. Here, we realize a multilayer ONN pre-processor for image
sensing, using a commercial image intensifier as a parallel optoelectronic,
optical-to-optical nonlinear activation function. We demonstrate that the
nonlinear ONN pre-processor can achieve compression ratios of up to 800:1 while
still enabling high accuracy across several representative computer-vision
tasks, including machine-vision benchmarks, flow-cytometry image
classification, and identification of objects in real scenes. In all cases we
find that the ONN's nonlinearity and depth allowed it to outperform a purely
linear ONN encoder. Although our experiments are specialized to ONN sensors for
incoherent-light images, alternative ONN platforms should facilitate a range of
ONN sensors. These ONN sensors may surpass conventional sensors by
pre-processing optical information in spatial, temporal, and/or spectral
dimensions, potentially with coherent and quantum qualities, all natively in
the optical domain