Personal storytelling: Using Natural Language Generation for children with complex communication needs, in the wild...

This paper describes a Natural Language Generation system (NLG), How was School Today? that automatically creates a personal narrative from sensor data and other media (photos and audio). It can be used by children with complex communication needs in schools to support interactive narrative about personal experiences. The robustness of story generation to missing data was identified as a key area for improvement in a feasibility study of the system at a first special needs school. This paper therefore suggests three possible methods for generating stories from unstructured data: clustering by voice recording, by location, or by time. Clustering based on voice recordings resulted in stories that were perceived as most easy to read, and to make most sense, by parents in a quantitative evaluation. This method was implemented in the live system, which was developed and evaluated iteratively at a second special needs school with children with different usage profiles. Open challenges and possibilities for NLG in augmented and alternative communication are also discussed

What is the potential for context aware communication aids?

Use of voice output communication aids (VOCAs) can be a very effective strategy to assist people with speech impairments in communicating. Despite this, people who use communication aids often express frustration with VOCAs—desiring devices that are simpler, quicker and more effective to use. Whilst it is not possible to resolve all these issues with technology, it is argued that significant progress can be made. The use of contextual information is one development that could improve the simplicity and effectiveness of communication aid design. Improving the effectiveness of communication aids, including through the use of context support, is a goal of the NIHR Devices for Dignity Assistive Technology Theme. This discussion paper examines the potential for creating ‘context aware’ communication aids. Three projects in which the authors have been involved are described to illustrate different approaches to the use of contextual information

Management and visualisation: seeing beyond the strategic

As organisations of all sizes become increasingly digitalised, a core management challenge remains unresolved. The ability to successfully and sustainably connect the stated vision of an organisation with its strategic plans and, in turn, with the reported reality of day-to-day operations, is largely an elusive ambition, despite the many stated advantages provided by contemporary technologies. In this book, the case is made for visual management as a method of communications, planning, learning and reporting that connects the organisation in a single, meaningful and seamless way. Throughout this book, visual management is theorised around the position that all forms of management documentation are an artefact of human construction and of the organisation itself that reflect learned patterns of activity. The book places visual management as a more intuitive and seamless method of coordinating, learning and communicating across an organisation than more traditional formats of presenting management documents. Consciously assembling the artefacts of an organisation in order to manage it introduces a layer of criticality that encourages reflection and consistency that is often absent from current management practice. The benefits that a visual approach brings to organisational management are an increasing necessity, as machine learning, robotics and process automation remove traditional roles from organisations and necessitate new views on how individuals now fit into a data-informed business. The book contributes to the academic debate regarding resource-based and knowledge-based views of the organisation by advocating a different, more holistic viewpoint and will thus appeal to academics and researchers in this area. It would also benefit students across business disciplines, whilst the practical models and tools offered will benefit directors and managers looking to implement their own visual organisational language

Enhancing student learning journeys with semantically annotated content

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThere is an increasing interest in developing existing Special Educational Needs (SEN) teaching methods due to recent concerns regarding the number of SEN pupils in schools. Communication is difficult for students when they have little or no clear speech. Consequently, a range of communication systems are used as an alternative to speech, including symbols, pictures or gestures. Importantly, helping students to better communicate also improves their education, friendships and independence. However, it is acknowledged that creating these educational resources is time consuming and expensive, and the learning results are not recognised as being as effective as required. Semantic Web technology has had an impact in the educational field and offers the required linkages for more engagement with Web content. There is, however, a considerable gap in Semantic Web research between the contributions in the mainstream educational field and research undertaken into special educational needs (SEN) students. This thesis presents an augmented World Wide Web (WWW) vision utilising annotation to more effectively support diverse special educational needs students. Students are supported in part by a SEN Teaching Platform (SENTP), one artefact from this design science research. Poetry is used as a website teaching material because of its significant impact on special needs students as it is a difficult topic to understand. The first stage of the research is to select the appropriate tools for testing annotation techniques in a real SEN environment. Later, a design of the proposed SEN teaching platform is built based on a Semantic Web annotation tool (Amaya) coordinated with a web application. Design is evaluated by conducting a pilot study in schools caring for special needs students (SEN). Evaluations were carried out at two schools, interviewing nine participants (Teachers, Teaching Assistant) in the UK. SENTP is tested for using Semantic Web technology to benefit the education of SEN students by utilizing Semantic Web annotation tools. This research further improves the SENTP with additional support for cognitive load using specific annotation formats within the Amaya annotation tool. Field testing is carried out at six UK schools with twenty-two participants being interviewed. Cognitive load principles are shown to improve both learning and class behaviour, also supporting teachers in the production of educational content. The pilot study and field testing results reveal that the proposed approach is effective. Following this, designed artefacts are synthesised within a wider design blueprint that articulates how this new world of annotated digital media is designed, deployed and consumed. Finally, SENTP ontology is created using OWL language and Protégé 5. The main goal of this ontology is to produce a wider design SENTP ontology that can be adapted to wider teaching purposes

Sistema de diálogo para la comunicación aumentativa y alternativa con pictogramas

En primer lugar, destacar que este Trabajo Fin de Máster se ha desarrollado dentro del grupo de investigación ViVoLab (Voice Input Voice Output Laboratory) de la Universidad de Zaragoza. Este proyecto se enmarca dentro del campo de la comunicación aumentativa y alternativa, la cual hace más sencilla la relación de personas con necesidades especiales, y tiene como objetivo la creación de un sistema de diálogo. Este sistema de diálogo permite la comunicación mediante texto y pictogramas, e incluye una herramienta de predicción que facilita su uso. El sistema de diálogo se basa en una aplicación tipo chat que permite a los usuarios enviar mensajes al resto de personas conectadas. Los mensajes son enviados usando tanto texto como pictogramas, y pueden ser introducidos de las dos formas, bien usando un teclado para escribir texto o seleccionando los pictogramas de la propia aplicación. Estos pictogramas están ordenados por categorías gracias al uso de la base de datos de Arasaac (portal Aragonés de la Comunicación Aumentativa y Alternativa). El sistema de predicción ordena las categorías presentando al usuario las que tienen un uso más probable en cada momento. Dentro de cada categoría, se pueden seleccionar los pictogramas (los cuales también aparecen ordenados por probabilidad de uso) añadiendo una nueva palabra al mensaje que se está escribiendo. El algoritmo de predicción también permite al usuario escoger sus frases más comunes al comienzo de la conversación, así como cada vez que envía o recibe un mensaje para poder seguir la conversación con más fluidez. Si el usuario se decide por redactar una nueva frase, el sistema de predicción le ayudará para completarla, actualizándose cada vez que se introduzca una nueva palabra y/o pictograma. En cuanto al sistema de predicción, éste se basa en modelos de lenguaje de N-gramas. Los N-gramas son secuencias de palabras de longitud N y la predicción se realiza a través de la probabilidad de cada N-grama en el modelo de lenguaje. Además, el algoritmo utiliza un modelo de lenguaje global y otro modelo de lenguaje caché, basado en el uso qué cada usuario hace del sistema, personalizándolo

An interactive classroom timetable for children with high functioning autism : development and qualitative evaluation of a computer-based timetable

Teachers report that advanced knowledge of the day's activities can considerably lessen anxiety for children with autism, especially if this information is highly visual in its presentation. Teaching children with autism often follows a highly structured approach including use of visual planning aids, such as symbolic timetables (Gillberg & Coleman, 2000; Mesibov, Browder, & Kirkland, 2002). Children with autism have been noted to be skilled at using computers and it has been suggested that the computer presents an ideal resource for both educational and recreational use (Attwood, 1998; Higgins & Boone, 1996; D. Moore, 1998; Murray, 1997). However, in spite of widespread use of visual timetables in schools by children with autism, there is little evidence of research being conducted in the area of computerised timetabling for this population. The overall aim of this particular study was to develop an interactive, computer-based timetable and to explore and assess the practical value for children with autism. The research was of an ethnographic nature and involved a case study approach. The development of the interactive timetable followed a user-centred design. Participating children attended a language and communication class attached to a mainstream primary school in Scotland. Seven males aged between six and nine years of age were involved, with four of the children able to participate in the final evaluations of the interactive timetable. Other participants included special needs teachers, nursery nurses, speech and language therapists, and parents of participating children. Examination of a symbolic timetable identified several important issues relevant to the design of an interactive timetable, whilst consideration of parental and educationalist views, gathered through interview, highlighted both benefits and concerns. By extending use to the home environment, the interactive timetable was considered to facilitate communication between child and parent. Concerns regarding access, size and position, user ability, and availability of resources were identified. Although findings were inconclusive, it was possible to communicate timetable information in a computer-based form and furthermore, the children appeared to enjoy using the timetable. Whilst the setting for this study was a single language and communication class attached to a mainstream primary school, this research has implications for a wider debate. The study found generally that the prospect of using information and communication technology to display an interactive timetable which was accessible both at school and at home, offered numerous possibilities for children with autism and their families.sub_psunpub77_ethesesunpu

Using Semantic Authoring for Blissymbols Communication Boards

Natural language generation (NLG) refers to the process of producing text in a spoken language, starting from an internal knowledge representation structure. Augmentative and Alternative Communication (AAC) deals with the development of devices and tools to enable basic conversation for language-impaired people. We present an applied prototype of an AAC-NLG system generating written output in English and Hebrew from a sequence of Bliss symbols. The system does not “translate ” the symbols sequence, but instead, it dynamically changes the communication board as the choice of symbols proceeds according to the syntactic and semantic content of selected symbols, generating utterances in natural language through a process of semantic authoring.