On Distant Speech Recognition for Home Automation

The official version of this draft is available at Springer via http://dx.doi.org/10.1007/978-3-319-16226-3_7International audienceIn the framework of Ambient Assisted Living, home automation may be a solution for helping elderly people living alone at home. This study is part of the Sweet-Home project which aims at developing a new home automation system based on voice command to improve support and well-being of people in loss of autonomy. The goal of the study is vocal order recognition with a focus on two aspects: distance speech recognition and sentence spotting. Several ASR techniques were evaluated on a realistic corpus acquired in a 4-room flat equipped with microphones set in the ceiling. This distant speech French corpus was recorded with 21 speakers who acted scenarios of activities of daily living. Techniques acting at the decoding stage, such as our novel approach called Driven Decoding Algorithm (DDA), gave better speech recognition results than the baseline and other approaches. This solution which uses the two best SNR channels and a priori knowledge (voice commands and distress sentences) has demonstrated an increase in recognition rate without introducing false alarms

Distant speech recognition for home automation: Preliminary experimental results in a smart home

International audienceThis paper presents a study that is part of the Sweet-Home project which aims at developing a new home automation system based on voice command. The study focused on two tasks: distant speech recognition and sentence spotting (e.g., recognition of domotic orders). Regarding the first task, different combinations of ASR systems, language and acoustic models were tested. Fusion of ASR outputs by consensus and with a triggered language model (using a priori knowledge) were investigated. For the sentence spotting task, an algorithm based on distance evaluation between the current ASR hypotheses and the predefine set of keyword patterns was introduced in order to retrieve the correct sentences in spite of the ASR errors. The techniques were assessed on real daily living data collected in a 4-room smart home that was fully equipped with standard tactile commands and with 7 wireless microphones set in the ceiling. Thanks to Driven Decoding Algorithm techniques, a classical ASR system reached 7.9% WER against 35% WER in standard configuration and 15% with MLLR adaptation only. The best keyword pattern classification result obtained in distant speech conditions was 7.5% CER

A Study of Accomodation of Prosodic and Temporal Features in Spoken Dialogues in View of Speech Technology Applications

Inter-speaker accommodation is a well-known property of human speech and human interaction in general. Broadly it refers to the behavioural patterns of two (or more) interactants and the effect of the (verbal and non-verbal) behaviour of each to that of the other(s). Implementation of thisbehavior in spoken dialogue systems is desirable as an improvement on the naturalness of humanmachine interaction. However, traditional qualitative descriptions of accommodation phenomena do not provide sufficient information for such an implementation. Therefore, a quantitativedescription of inter-speaker accommodation is required. This thesis proposes a methodology of monitoring accommodation during a human or humancomputer dialogue, which utilizes a moving average filter over sequential frames for each speaker. These frames are time-aligned across the speakers, hence the name Time Aligned Moving Average (TAMA). Analysis of spontaneous human dialogue recordings by means of the TAMA methodology reveals ubiquitous accommodation of prosodic features (pitch, intensity and speech rate) across interlocutors, and allows for statistical (time series) modeling of the behaviour, in a way which is meaningful for implementation in spoken dialogue system (SDS) environments.In addition, a novel dialogue representation is proposed that provides an additional point of view to that of TAMA in monitoring accommodation of temporal features (inter-speaker pause length and overlap frequency). This representation is a percentage turn distribution of individual speakercontributions in a dialogue frame which circumvents strict attribution of speaker-turns, by considering both interlocutors as synchronously active. Both TAMA and turn distribution metrics indicate that correlation of average pause length and overlap frequency between speakers can be attributed to accommodation (a debated issue), and point to possible improvements in SDS “turntaking” behaviour. Although the findings of the prosodic and temporal analyses can directly inform SDS implementations, further work is required in order to describe inter-speaker accommodation sufficiently, as well as to develop an adequate testing platform for evaluating the magnitude ofperceived improvement in human-machine interaction. Therefore, this thesis constitutes a first step towards a convincingly useful implementation of accommodation in spoken dialogue systems

Broadening Exposure to Socio-Political Opinions via a Pushy Smart Home Device

Motivated by the effects of the filter bubble and echo chamber phenomena on social media, we developed a smart home device, Spkr, that unpredictably “pushes” socio-political discussion topics into the home. The device utilised trending Twitter discussions, categorised by their socio-political alignment, to present people with a purposefully assorted range of viewpoints. We deployed Spkr in 10 homes for 28 days with a diverse range of participants and interviewed them about their experiences. Our results show that Spkr presents a novel means of combating selective exposure to socio-political issues, providing participants with identifiably diverse viewpoints. Moreover, Spkr acted as a conversational prompt for discussion within the home, initiating collective processes and engaging those who would not often be involved in political discussions. We demonstrate how smart home assistants can be used as a catalyst for provocation by altering and pluralising political discussions within households

Advancing automation and robotics technology for the Space Station and for the US economy, volume 2

In response to Public Law 98-371, dated July 18, 1984, the NASA Advanced Technology Advisory Committee has studied automation and robotics for use in the Space Station. The Technical Report, Volume 2, provides background information on automation and robotics technologies and their potential and documents: the relevant aspects of Space Station design; representative examples of automation and robotics; applications; the state of the technology and advances needed; and considerations for technology transfer to U.S. industry and for space commercialization

Multi-modal interface for offline robot programming

This thesis presents an approach for improving robot offline programming using input methods based on the human natural skills. The approach is focused to teach basic assembly and manipulation operations using a pair of industrial robots in an existing simulation environment and is meant to be improved in future works, that are also proposed in this thesis. In order to develop this approach, and regarding the available resources, an Add-In for the simulation and offline programming software RobotStudio was developed. This Add-In combines human pose, a graphical user interface and optionally speech to teach the robot a sequence of targets, along with the simulation environment, to automatically generate instructions. Two different kinds of sensors, Kinect and Leap Motion Sensor have been evaluated based on references in order to select the most suitable one for the implementation of this work. The executions of the programmed instructions have been evaluated in simulation.Este trabajo presenta una propuesta para mejorar al programación de robots fuera de línea usando métodos de entrada basados en habilidades humanas naturales. La propuesta se enfoca en enseñar operaciones básicas de ensamblaje y manipulación, utilizando un par de robots industriales en un entorno de simulación ya existente y se dispone para ser mejorado en trabajos futuros, los cuales también se proponen en este trabajo. Con el fin de desarrollar esta propuesta y teniendo en cuenta los recursos disponibles, se ha desarrollado un Add-In para el programa de simulación y programación fuera de línea Robot Studio. Este Add-In combina pose humana, una interfaz gráfica de usuario y opcionalmente habla para enseñar al robot una secuencia de objetivos junto con el entorno de simulación para automáticamente generar instrucciones. Dos diferentes tipos de sensores, Kinect y Leap Motion Sensor han sido evaluados en base a referencias para seleccionar el más adecuado para la implementación de este trabajo. Las ejecuciones de las instrucciones programadas han sido evaluadas en simulación.Máster Universitario en Ingeniería Industrial (M141