986 research outputs found
Hierarchical Quantized Representations for Script Generation
Scripts define knowledge about how everyday scenarios (such as going to a
restaurant) are expected to unfold. One of the challenges to learning scripts
is the hierarchical nature of the knowledge. For example, a suspect arrested
might plead innocent or guilty, and a very different track of events is then
expected to happen. To capture this type of information, we propose an
autoencoder model with a latent space defined by a hierarchy of categorical
variables. We utilize a recently proposed vector quantization based approach,
which allows continuous embeddings to be associated with each latent variable
value. This permits the decoder to softly decide what portions of the latent
hierarchy to condition on by attending over the value embeddings for a given
setting. Our model effectively encodes and generates scripts, outperforming a
recent language modeling-based method on several standard tasks, and allowing
the autoencoder model to achieve substantially lower perplexity scores compared
to the previous language modeling-based method.Comment: EMNLP 201
Understanding Chat Messages for Sticker Recommendation in Messaging Apps
Stickers are popularly used in messaging apps such as Hike to visually
express a nuanced range of thoughts and utterances to convey exaggerated
emotions. However, discovering the right sticker from a large and ever
expanding pool of stickers while chatting can be cumbersome. In this paper, we
describe a system for recommending stickers in real time as the user is typing
based on the context of the conversation. We decompose the sticker
recommendation (SR) problem into two steps. First, we predict the message that
the user is likely to send in the chat. Second, we substitute the predicted
message with an appropriate sticker. Majority of Hike's messages are in the
form of text which is transliterated from users' native language to the Roman
script. This leads to numerous orthographic variations of the same message and
makes accurate message prediction challenging. To address this issue, we learn
dense representations of chat messages employing character level convolution
network in an unsupervised manner. We use them to cluster the messages that
have the same meaning. In the subsequent steps, we predict the message cluster
instead of the message. Our approach does not depend on human labelled data
(except for validation), leading to fully automatic updation and tuning
pipeline for the underlying models. We also propose a novel hybrid message
prediction model, which can run with low latency on low-end phones that have
severe computational limitations. Our described system has been deployed for
more than months and is being used by millions of users along with hundreds
of thousands of expressive stickers
PoET-BiN: Power Efficient Tiny Binary Neurons
The success of neural networks in image classification has inspired various
hardware implementations on embedded platforms such as Field Programmable Gate
Arrays, embedded processors and Graphical Processing Units. These embedded
platforms are constrained in terms of power, which is mainly consumed by the
Multiply Accumulate operations and the memory accesses for weight fetching.
Quantization and pruning have been proposed to address this issue. Though
effective, these techniques do not take into account the underlying
architecture of the embedded hardware. In this work, we propose PoET-BiN, a
Look-Up Table based power efficient implementation on resource constrained
embedded devices. A modified Decision Tree approach forms the backbone of the
proposed implementation in the binary domain. A LUT access consumes far less
power than the equivalent Multiply Accumulate operation it replaces, and the
modified Decision Tree algorithm eliminates the need for memory accesses. We
applied the PoET-BiN architecture to implement the classification layers of
networks trained on MNIST, SVHN and CIFAR-10 datasets, with near state-of-the
art results. The energy reduction for the classifier portion reaches up to six
orders of magnitude compared to a floating point implementations and up to
three orders of magnitude when compared to recent binary quantized neural
networks.Comment: Accepted in MLSys 2020 conferenc
Paralinguistic Privacy Protection at the Edge
Voice user interfaces and digital assistants are rapidly entering our lives
and becoming singular touch points spanning our devices. These always-on
services capture and transmit our audio data to powerful cloud services for
further processing and subsequent actions. Our voices and raw audio signals
collected through these devices contain a host of sensitive paralinguistic
information that is transmitted to service providers regardless of deliberate
or false triggers. As our emotional patterns and sensitive attributes like our
identity, gender, mental well-being, are easily inferred using deep acoustic
models, we encounter a new generation of privacy risks by using these services.
One approach to mitigate the risk of paralinguistic-based privacy breaches is
to exploit a combination of cloud-based processing with privacy-preserving,
on-device paralinguistic information learning and filtering before transmitting
voice data. In this paper we introduce EDGY, a configurable, lightweight,
disentangled representation learning framework that transforms and filters
high-dimensional voice data to identify and contain sensitive attributes at the
edge prior to offloading to the cloud. We evaluate EDGY's on-device performance
and explore optimization techniques, including model quantization and knowledge
distillation, to enable private, accurate and efficient representation learning
on resource-constrained devices. Our results show that EDGY runs in tens of
milliseconds with 0.2% relative improvement in ABX score or minimal performance
penalties in learning linguistic representations from raw voice signals, using
a CPU and a single-core ARM processor without specialized hardware.Comment: 14 pages, 7 figures. arXiv admin note: text overlap with
arXiv:2007.1506
Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos
Every moment counts in action recognition. A comprehensive understanding of
human activity in video requires labeling every frame according to the actions
occurring, placing multiple labels densely over a video sequence. To study this
problem we extend the existing THUMOS dataset and introduce MultiTHUMOS, a new
dataset of dense labels over unconstrained internet videos. Modeling multiple,
dense labels benefits from temporal relations within and across classes. We
define a novel variant of long short-term memory (LSTM) deep networks for
modeling these temporal relations via multiple input and output connections. We
show that this model improves action labeling accuracy and further enables
deeper understanding tasks ranging from structured retrieval to action
prediction.Comment: To appear in IJC
- …