6 research outputs found
Recognizing Uncertainty in Speech
We address the problem of inferring a speaker's level of certainty based on
prosodic information in the speech signal, which has application in
speech-based dialogue systems. We show that using phrase-level prosodic
features centered around the phrases causing uncertainty, in addition to
utterance-level prosodic features, improves our model's level of certainty
classification. In addition, our models can be used to predict which phrase a
person is uncertain about. These results rely on a novel method for eliciting
utterances of varying levels of certainty that allows us to compare the utility
of contextually-based feature sets. We elicit level of certainty ratings from
both the speakers themselves and a panel of listeners, finding that there is
often a mismatch between speakers' internal states and their perceived states,
and highlighting the importance of this distinction.Comment: 11 page
Framework for Human Computer Interaction for Learning Dialogue Strategies using Controlled Natural Language in Information Systems
Spoken Language systems are going to have a tremendous impact in all
the real world applications, be it healthcare enquiry, public transportation
system or airline booking system maintaining the language ethnicity for
interaction among users across the globe. These system have the capability
of interacting with the user in di erent languages that the system
supports. Normally when a person interacts with another person there are
many non-verbal clues which guide the dialogue and all the utterances have
a contextual relationship, which manage the dialogue as its mixed by the
two speakers. Human Computer Interaction has a wide impact on the design
of the applications and has become one of the emerging interest area of
the researchers. All of us are witness to an explosive electronic revolution
where lots of gadgets and gizmo's have surrounded us, advanced not only
in power, design, applications but the ease of access or what we call user
friendly interfaces are designed that we can easily use and control all the
functionality of the devices. Since speech is one of the most intuitive form
of interaction that humans use. It provides potential bene ts such as handfree
access to machines, ergonomics and greater e ciency of interaction.
Yet, speech-based interfaces design has been an expert job for a long time.
Lot of research has been done in building real spoken Dialogue Systems
which can interact with humans using voice interactions and help in performing
various tasks as are done by humans. Last two decades have seen
utmost advanced research in the automatic speech recognition, dialogue
management, text to speech synthesis and Natural Language Processing
for various applications which have shown positive results. This dissertation
proposes to apply machine learning (ML) techniques to the problem
of optimizing the dialogue management strategy selection in the Spoken
Dialogue system prototype design. Although automatic speech recognition
and system initiated dialogues where the system expects an answer in the
form of `yes' or `no' have already been applied to Spoken Dialogue Systems(
SDS), no real attempt to use those techniques in order to design a
new system from scratch has been made. In this dissertation, we propose
some novel ideas in order to achieve the goal of easing the design of Spoken
Dialogue Systems and allow novices to have access to voice technologies.
A framework for simulating and evaluating dialogues and learning optimal
dialogue strategies in a controlled Natural Language is proposed. The simulation
process is based on a probabilistic description of a dialogue and
on the stochastic modelling of both arti cial NLP modules composing a
SDS and the user. This probabilistic model is based on a set of parameters
that can be tuned from the prior knowledge from the discourse or learned
from data. The evaluation is part of the simulation process and is based
on objective measures provided by each module. Finally, the simulation
environment is connected to a learning agent using the supplied evaluation
metrics as an objective function in order to generate an optimal behaviour
for the SDS
Simulations of language in individuals with and without Autism Spectrum Disorder (ASD)
The current thesis provides an exploration of mental simulations of language in individuals with and without Autism Spectrum Disorder (ASD). The experiential explanation of language proposes that language comprehension is facilitated through the construction of mental simulations of described events, which are embodied in cognition; grounded in action and perception. This high order cognitive process is thought to be underpinned by the mirror neuron system and other neural networks in the typically developed (TD) population. In a series of six experiments combining behavioural, EEG and eye-tracking measures with psycholinguistic paradigms, this thesis examines for the first time whether individuals with ASD activate mental simulations of language that are comparable to those of TD individuals. The main findings suggest that individuals with ASD are able to simulate written and spoken language, and do so in the same way as TD individuals; relying on the same neurological correlates. These simulations are activated in real-time as the described event unfolds and are constrained by the linguistic input. However, the findings point to a possible deficit or bias in interpreting prosodic content in ASD. Moreover, difficulties in simulating described events in ASD emerge when the temporal sequence of events are interrupted. Moreover, while individuals with ASD are able to simulate language online, subtle differences in processing compared to TD individuals may explain the social communication associated with the disorder. The findings offer support for a complex information processing explanation of ASD and are discussed in relation to existing cognitive theories of ASD and the impact of social skills and language ability on mental simulations
Replication data for: Recognizing Uncertainty in Speech
We address the problem of inferring a speaker's level of certainty based on prosodic information in the speech signal, which has application in speech-based dialogue systems. We show that using phrase-level prosodic features centered around the phrases causing uncertainty, in addition to utterance-level prosodic features, improves our model's level of certainty classification. In addition, our models can be used to predict which phrase a person is uncertain about. These results rely on a novel method for eliciting utterances of varying levels of certainty that allows us to compare the utility of contextually-based feature sets. We elicit level of certainty ratings from both the speakers themselves and a panel of listeners, finding that there is often a mismatch between speakers' internal states and their perceived states, and highlighting the importance of this distinction