The Role of Emotional and Facial Expression in Synthesised Sign Language Avatars

This thesis explores the role that underlying emotional facial expressions might have in regards to understandability in sign language avatars. Focusing specifically on Irish Sign Language (ISL), we examine the Deaf community’s requirement for a visual-gestural language as well as some linguistic attributes of ISL which we consider fundamental to this research. Unlike spoken language, visual-gestural languages such as ISL have no standard written representation. Given this, we compare current methods of written representation for signed languages as we consider: which, if any, is the most suitable transcription method for the medical receptionist dialogue corpus. A growing body of work is emerging from the field of sign language avatar synthesis. These works are now at a point where they can benefit greatly from introducing methods currently used in the field of humanoid animation and, more specifically, the application of morphs to represent facial expression. The hypothesis underpinning this research is: augmenting an existing avatar (eSIGN) with various combinations of the 7 widely accepted universal emotions identified by Ekman (1999) to deliver underlying facial expressions, will make that avatar more human-like. This research accepts as true that this is a factor in improving usability and understandability for ISL users. Using human evaluation methods (Huenerfauth, et al., 2008) the research compares an augmented set of avatar utterances against a baseline set with regards to 2 key areas: comprehension and naturalness of facial configuration. We outline our approach to the evaluation including our choice of ISL participants, interview environment, and evaluation methodology. Remarkably, the results of this manual evaluation show that there was very little difference between the comprehension scores of the baseline avatars and those augmented withEFEs. However, after comparing the comprehension results for the synthetic human avatar “Anna” against the caricature type avatar “Luna”, the synthetic human avatar Anna was the clear winner. The qualitative feedback allowed us an insight into why comprehension scores were not higher in each avatar and we feel that this feedback will be invaluable to the research community in the future development of sign language avatars. Other questions asked in the evaluation focused on sign language avatar technology in a more general manner. Significantly, participant feedback in regard to these questions indicates a rise in the level of literacy amongst Deaf adults as a result of mobile technology

A Cognitive Linguistic Study of Visual Language as a Reflection of Culture: Violence Against Women in Iraq

The present work is qualitative descriptive. It aims to examine the idiosyncratic schema when deciphering the selected violence-based panel from Nasser Ibrahim’s caricatures. The researchers accordingly adopted part of Sharifian’s (2011) Cultural Schema model, particularly that part that is concerned with the examining the micro/idiosyncratic level of understanding. The study has revealed that the participants have not only differed among themselves regarding the way a figure is being denotatively conceptualized, they also highlighted different exact conceptualizations for the same figure, such as: using various adjectives that reflect various levels of intensity, emphasizing the behavioral aspect or the appearance of the figure, adopting different patterns of thinking like: metaphoric, metronomic, generic, specific, comprehensive or linear. Besides, based on their various points of attention and cognitive schema, they varied in conceptualizing the type of violence being depicted in this image as: psychological, verbal, physical, domestic violence or as the combination of more than one type. Finally, knowing about cultural schema does not involve to exemplify them in one’s behavior.

Animating evaluation: A multimodal appraisal analysis of story-time animation

This dissertation presents the multimodal appraisal analysis of story-time animation videos; a form of computer-mediated motion picture entertainment that are conventionally uploaded to the social media platform YouTube. By means of the multimodal appraisal analysis, this dissertation aims to elucidate the potential of story-time animation to construe emotions and opinions. To achieve this goal, the dissertation establishes a multimodal appraisal methodology which facilitates the identification, description and analysis of emotions and opinions in texts which implement several communicative modes. The dissertation first outlines several linguistic frameworks constructed to identify emotions and opinions - such as Martin and White’s Appraisal Framework (2005) - and proceeds to explore how such frameworks have been adapted to identify emotions and opinions in multimodal texts. It then establishes the multimodal appraisal methodology established in this dissertation, designed specifically with animated media in mind. The dissertation then applies this multimodal appraisal framework to various scenes of three story-time animation videos produced by the YouTube content creator ‘Jaiden Animations’ in order to elucidate the evaluative potential of story-time animations. The multimodal appraisal analysis of this dissertation ascertained that story-time animations possess great potential to construe emotion and opinions however each of the communicative modes that it utilises commits to these emotions and opinions to varying degrees. It also observed that these communicative modes would complement and contradict one another in order to upscale and downscale emotions and opinions and to construe a wide range of emotion hybrids

The Narratology of Comic Art

By placing comics in a lively dialogue with contemporary narrative theory, The Narratology of Comic Art builds a systematic theory of narrative comics, going beyond the typical focus on the Anglophone tradition. This involves not just the exploration of those properties in comics that can be meaningfully investigated with existing narrative theory, but an interpretive study of the potential in narratological concepts and analytical procedures that has hitherto been overlooked. This research monograph is, then, not an application of narratology in the medium and art of comics, but a revision of narratological concepts and approaches through the study of narrative comics. Thus, while narratology is brought to bear on comics, equally comics are brought to bear on narratology.Peer reviewe

A PSYCHOLINGUISTIC ANALYSIS OF DYSLEXIA IN BACKWARDS: THE RIDDLE OF DYSLEXIA

This study is under a psychopragmatics umbrella. The aim of this study is to analyze communication impairments phenomena of asperger syndrome suffered by Morton and its illocutionary acts, the main character in Mozart and the Whale. This study has two objectives: (1) to figure out the communication impairments experienced by the main character in Mozart and the Whale; and (2) to describe the illocutionary acts used by Morton in the movie. This study employed a descriptive qualitative method since it gave an emphasis on describing the phenomena of asperger syndrome in Mozart and the Whale. Moreover, the findings were presented in narrative or textual description. However, number was also used to support the analysis of the data. Some steps in analyzing the data were: identifying the raw data, classifying each datum into the categorization, analyzing each datum, interpreting each datum based on its contexts, reporting the findings, and drawing the conclusion. Finally, the data findings were triangulated by two linguistics students who were keen on psycholinguistics and pragmatics. This study reveals two findings. First, only seven types occur in Mozart and the Whale. There is a limited use of gestures, clumsy body language, inappropriate facial expression, limited facial expression, superficially perfect expressive language, impairments of comprehension including misinterpretation of literal or implied meaning and peculiar stiff gaze. Meanwhile, the absent types are odd prosody, peculiar voice characteristics, formal pedantic language, and delayed speech development. Last, types of illocutionary act which occur in the movie are representative and directive. Representative consists of informing and stating. Directive consists of ordering. The most occurences go to informing, then stating and the last is ordering. Meanwhile, the three types that Morton unable to perform are expressive, commissive, and declarative. The ability to employ the two types provides evidences that Morton is able to state what he believes to be the case or not and is able to ask someone to do something. His inability to employ the other types proves that he is unable to express his psychological state, commit himself to an action, change the world of others and to perform speech act which has the function related to the activity of enaging social interaction with others, but he is able to perform the speech act which has the function related to his personal interest, in this case calculating numbers

To Affinity and Beyond: Interactive Digital Humans as a Human Computer Interface

The field of human computer interaction is increasingly exploring the use of more natural, human-like user interfaces to build intelligent agents to aid in everyday life. This is coupled with a move to people using ever more realistic avatars to represent themselves in their digital lives. As the ability to produce emotionally engaging digital human representations is only just now becoming technically possible, there is little research into how to approach such tasks. This is due to both technical complexity and operational implementation cost. This is now changing as we are at a nexus point with new approaches, faster graphics processing and enabling new technologies in machine learning and computer vision becoming available. I articulate the issues required for such digital humans to be considered successfully located on the other side of the phenomenon known as the Uncanny Valley. My results show that a complex mix of perceived and contextual aspects affect the sense making on digital humans and highlights previously undocumented effects of interactivity on the affinity. Users are willing to accept digital humans as a new form of user interface and they react to them emotionally in previously unanticipated ways. My research shows that it is possible to build an effective interactive digital human that crosses the Uncanny Valley. I directly explore what is required to build a visually realistic digital human as a primary research question and I explore if such a realistic face provides sufficient benefit to justify the challenges involved in building it. I conducted a Delphi study to inform the research approaches and then produced a complex digital human character based on these insights. This interactive and realistic digital human avatar represents a major technical undertaking involving multiple teams around the world. Finally, I explored a framework for examining the ethical implications and signpost future research areas

A critical examination of race in Business English coursebooks

This thesis examines ways in which uses of images and words contribute towards constructions of race in published Business English language coursebooks, by exploring coursebook writers’ perspectives on compiling their Business English coursebooks and analysing Business English coursebook materials. The study drew on critical race theory, critical discourse analysis, and systemic functional linguistics to investigate relations between the ways coursebook writers construct race in selecting and organising materials for their Business English coursebooks and the ways race is constructed in Business English coursebooks. The data included seven published Business English coursebooks and interviews with writers of four of the coursebooks. The coursebooks were Global links 2 (Blackwell, 2001), Market leader intermediate (Cotton, Falvey, & Kent, 2005), Quick work pre-intermediate (Hollett, 2000), New international business English (Jones & Alexander, 2003), In company intermediate (Powell, 2002), First insights into business (Robbins, 2000), and International express pre-intermediate (Taylor, 2004). I argue that constructions of race in the coursebooks connect three notions: international business people, corporate ethics and responsibilities, and intercultural business communication. Patterns in the location and composition of the language learning materials, and expressions of opinions and emotions in illustrative extracts from the materials were found to contribute to these constructs