Improving generalisation to new speakers in spoken dialogue state tracking

Users with disabilities can greatly benefit from personalised voice-enabled environmental-control interfaces, but for users with speech impairments (e.g. dysarthria) poor ASR performance poses a challenge to successful dialogue. Statistical dialogue management has shown resilience against high ASR error rates, hence making it useful to improve the performance of these interfaces. However, little research was devoted to dialogue management personalisation to specific users so far. Recently, data driven discriminative models have been shown to yield the best performance in dialogue state tracking (the inference of the user goal from the dialogue history). However, due to the unique characteristics of each speaker, training a system for a new user when user specific data is not available can be challenging due to the mismatch between training and working conditions. This work investigates two methods to improve the performance with new speakers of a LSTM-based personalised state tracker: The use of speaker specific acoustic and ASRrelated features; and dropout regularisation. It is shown that in an environmental control system for dysarthric speakers, the combination of both techniques yields improvements of 3.5% absolute in state tracking accuracy. Further analysis explores the effect of using different amounts of speaker specific data to train the tracking system

An introduction to STRIKE : STRuctured Interpretation of the Knowledge Environment

Knowledge forms a critical part of the income generation of the system and the complex environment in which actors participate in the creation of knowledge assets merits robust, eclectic consideration. STRIKE - STRuctured Interpretation of the Knowledge Environment affords an unobtrusive and systematic framework to observe, record, evaluate and articulate concrete and abstract elements of a setting, across internal and external dimensions. Inter-relationships between actor and environment are preserved. STRIKE is supported by underlying techniques to enrich data and enhance the authenticity of its representation. Adoption of photography and videography tools provides illustrative and interpretive benefits and facilitates researcher reflexivity. This structured approach to data analysis and evaluation mitigates criticisms of methodological rigour in observational research and affords standardisation potential, germane for application in a verification or longitudinal capacity. Advancing exploratory validation studies, the method is employed to evaluate the knowledge environments of two enterprises in the UK creative sector. These occupy a critical role in fostering entrepreneurial innovation alongside participant self-efficacy. Access Space in Sheffield and the Bristol Hackspace are committed to open software, open knowledge and open participation; sharing peer learning, creativity and socio-technical aims to address broadly similar community needs. Drawing on Wittgenstein’s Picture Theory of Meaning, the knowledge management perspective is abstracted from the STRIKE assessment. It is argued that the tiered analytical approach which considers a breadth of dimensions enhances representation and interpretation of the knowledge environment and presents a diagnostic and prescriptive capability to actualise change. The paper concludes by evaluating framework effectiveness, findings application and future direction

Personalised Dialogue Management for Users with Speech Disorders

Many electronic devices are beginning to include Voice User Interfaces (VUIs) as an alternative to conventional interfaces. VUIs are especially useful for users with restricted upper limb mobility, because they cannot use keyboards and mice. These users, however, often suffer from speech disorders (e.g. dysarthria), making Automatic Speech Recognition (ASR) challenging, thus degrading the performance of the VUI. Partially Observable Markov Decision Process (POMDP) based Dialogue Management (DM) has been shown to improve the interaction performance in challenging ASR environments, but most of the research in this area has focused on Spoken Dialogue Systems (SDSs) developed to provide information, where the users interact with the system only a few times. In contrast, most VUIs are likely to be used by a single speaker over a long period of time, but very little research has been carried out on adaptation of DM models to specific speakers. This thesis explores methods to adapt DM models (in particular dialogue state tracking models and policy models) to a specific user during a longitudinal interaction. The main differences between personalised VUIs and typical SDSs are identified and studied. Then, state-of-the-art DM models are modified to be used in scenarios which are unique to long-term personalised VUIs, such as personalised models initialised with data from different speakers or scenarios where the dialogue environment (e.g. the ASR) changes over time. In addition, several speaker and environment related features are shown to be useful to improve the interaction performance. This study is done in the context of homeService, a VUI developed to help users with dysarthria to control their home devices. The study shows that personalisation of the POMDP-DM framework can greatly improve the performance of these interfaces

‘Alexa, Adv(ert)ise us!’: How Smart Speakers and Digital Assistants Challenge Advertising Literacy Amongst Young People

Digitale Sprachassistenten erreichen zunehmend auch die Lebenswelt von Kindern. Sie erinnern in ihren Funktionen teilweise an Spielkameraden, Kindermädchen und Hauslehrer. Die Pädagogik hat sich mit diesem neuen Medienphänomen bisher nur marginal auseinandergesetzt. Dabei liegen in der Nutzung von Smart Speakern durch junge Menschen viele Chancen, aber auch Herausforderungen. Diese werden in diesem Artikel herausgearbeitet und medienpädagogisch eingeordnet. Wir beginnen mit einer Definition von Smart Speakern und digitalen Sprachassistenten, um dann deren Gebrauch anhand von Nutzungsdaten zu beleuchten. Im nächsten Schritt widmen wir uns der Frage, inwieweit diese smarten Technologien in der Lebenswelt von jungen Menschen eine Rolle spielen. Welche Formen von Werbung gibt es dort? Welche Daten sammeln Sprachassistenten? Und: Wie können Eltern, Pädagoginnen, Pädagogen und Unternehmen für eine kindgerechte, datenschutzkonforme Nutzung smarter Technologien sorgen? Unser Ziel ist es, das Phänomen Smart Speaker und digitale Sprachassistenten ein Stück weiter ins medienpädagogische Visier zu rücken. Der Umgang mit den spezifischen Eigenheiten von Smart Speakern bedarf eines hohen Masses an (kindlicher) Nutzungskompetenz. Wie wir abschliessend aufzeigen werden, ergeben sich aus Sicht der Werbekompetenzförderung weitere pädagogisch gewinnbringende Ansatzpunkte.Digital assistants increasingly infiltrate the world of children. The way they function reminds us somewhat of playmates, nannies and tutors. So far, educators have only marginally dealt with this new media phenomenon, yet the use of smart speakers by young people offers many opportunities as well as challenges. These are elaborated in this article and classified in terms of media education. Firstly, we will address a definition of smart speakers and digital speech assistants, and then examine their use by means of usage data. We will then concentrate on examining the extent to which these smart technologies play a role in the environments of young people. What forms of advertising are there? What data do digital assistants collect? And finally, how can parents, educators and companies ensure that smart technologies are used in a child-friendly manner that complies with data protection regulations? Our aim is to nudge the phenomenon of smart speakers and speech assistants into the media-pedagogical focus. Dealing with the specific characteristics of smart speakers requires a high degree of (child) user competence. As we will show in the conclusion, there are further pedagogically beneficial approaches from the point of view of promoting advertising literacy