Eyetracking and Applied Linguistics

Eyetracking has become a powerful tool in scientific research and has finally found its way into disciplines such as applied linguistics and translation studies, paving the way for new insights and challenges in these fields. The aim of the first International Conference on Eyetracking and Applied Linguistics (ICEAL) was to bring together researchers who use eyetracking to empirically answer their research questions. It was intended to bridge the gaps between applied linguistics, translation studies, cognitive science and computational linguistics on the one hand and to further encourage innovative research methodologies and data triangulation on the other hand. These challenges are also addressed in this proceedings volume: While the studies described in the volume deal with a wide range of topics, they all agree on eyetracking as an appropriate methodology in empirical research

Calibrated Authenticity : Multimodal Analysis of Authenticity Construction in the Instagram Feed of @noelledowning

This thesis examines the multimodal construction of authenticity in the Instagram feed content of lifestyle social media influencers. The value of authenticity and ’realness’ has been made apparent in previous examinations of social media influencers where the portrayal of ’calibrated amateurism’, through establishing relatability, and authenticity have been identified as key components of content creation for social media influencers on Instagram (Abidin 2018). The need for authenticity is juxtaposed against an Instagram platform vernacular of conventional styles of content creation that are focused on highly curated and aesthetically pleasing representations. The vocational needs of the social media influencers in creating commercially laden advertorials further complicate the multimodal construction of authenticity. These realities of social media influencer content creation combine with the semiotic technology affordances of the Instagram platform to create an interesting environment in which different methods of constructing and communicating authenticity can be examined. The study examines this environment through a focus on the Instagram account of a New York - based lifestyle influencer Noelle Downing (@noelledowning). A textual analysis of the visual and written content in the Instagram feed posts was employed on a corpus of 275 posts and 369 photos connected to the posts. The theoretical framework of the thesis is built on the theory of representation of visual social actors by Theo van Leeuwen and on the theory of mediated authenticity by Gunn Enli. The theory of mediated authenticity was expanded successfully in this thesis through the category of engagement to further its applicability to social media content. The ideational, interpersonal, and textual discursive metafunctions devised by M.A.K. Halliday further guided the examination of authenticity construction in the synthesis of written and visual content. The results of the study indicate that authenticity is constructed through relatively uniform visual representations that communicate relatability, focus on creating an interpersonal connection with the followers, and through representing predictable and conventional lifestyle influencer topics and styles, whereas captions are used to provide a more detailed look into the private life, thoughts, and feelings of the influencer. Through a multimodal and multidisciplinary approach to social media content creation, the study establishes the lifestyle social media influencer authenticity construction as a form of calibrated authenticity where the mundane, relatable, and personal are invoked to balance the apparent commercialism of advertorial content. Influencers are found striving to establish an equitable relationship between themselves and their followers to ensure authenticity and relatability

How captions help people learn languages: A working-memory, eye-tracking study

Captions provide a useful aid to language learners for comprehending videos and learning new vocabulary, aligning with theories of multimedia learning. Multimedia learning predicts that a learner’s working memory (WM) influences the usefulness of captions. In this study, we present two eye-tracking experiments investigating the role of WM in captioned video viewing behavior and comprehension. In Experiment 1, Spanish-as-a-foreign-language learners differed in caption use according to their level of comprehension and to a lesser extent, their WM capacities. WM did not impact comprehension. In Experiment 2, English-as-a-second-language learners differed in comprehension according to their WM capacities. Those with high comprehension and high WM used captions less on a second viewing. These findings highlight the effects of potential individual differences and have implications for the integration of multimedia with captions in instructed language learning. We discuss how captions may help neutralize some of working memory’s limiting effects on learning

“MacGyver-Meets-Dr. Ruth”: Science Journalism and the Material Positioning of Dr. Carla Pugh

This article examines the rhetorical consequences of foregrounding female scientists\u27 materials through an analysis of seven news articles on Dr. Carla Pugh, a surgeon who designs medical patient simulators. Journalists foreground Pugh\u27s materials by positioning her as both “MacGyver,” creatively assembling simulators from everyday objects, and “Dr. Ruth,” willingly discussing intimate parts. These positions avoid focusing on Pugh\u27s personal life or body but still ultimately gender her and her work. The MacGyver position associates Pugh with gendered activities, objects, and spaces while undermining her affiliation with the technical aspects of design. Meanwhile, the Dr. Ruth position implies Pugh\u27s knowledge comes from inherent bodily expertise, making certain scientific fields appear more natural for women

Yearbooks as a Genre: A Case Study

ABSTRACT In the United States, high school and college yearbooks are extraordinarily well known as a genre, yet they are largely unstudied. Yearbooks preserve images, stories, and facts from each year for one specific group of people, linked by age and geographic community. Yearbook production is a significant commercial enterprise, yet it involves novice writers, editors, and designers. Blending elements of craft, tradition, business, and media, yearbooks as a distinctive genre bear closer rhetorical study and application of professional communication theories. This historical case study of production practices for a particular college yearbook positions yearbooks rhetorically as texts and sites of communication practices. The literature review examines the scholarship of yearbooks, rhetorical studies of genre and activity theory, and, because yearbooks are a visual genre, rhetorical studies of design. The research incorporates 1) a rich description of yearbook production from 2003-05, and 2) a rhetorical analysis of the spreads and images of the two college yearbooks produced during that period. The description of production relies on materials used in generating the yearbooks as well as a personal interview with a publishing representative and retrospective description of personal experience. It applies genre, activity theory, and genre ecology theory as a framework for analyzing yearbooks. The visual composition analysis applies the concepts of Kress and van Leeuwen. The results show that these two yearbooks were produced by a complex and interconnected activity system involving many different people, documents, technologies, and actions. One change to the system affects all other aspects and influences the entire dynamic of production. An analysis of the images in the two yearbooks revealed that persons were depicted predominantly as making an offer in gaze, at medium social distance, and at eye level with the viewer. What emerged from an analysis of the composition of yearbook layouts was an understanding of information value and the power of yearbook creators in determining the order of importance in the spread. The conclusion develops avenues for further study including reception, workplace communication, feminist studies, technology, and ideology. Ultimately, additional research might address this genre in terms of ideological critique, investigating Anis Bawarshi\u27s formulation about genres as \u27sites for cultural critique and change.\u27 Yearbooks, with their longevity and adherence to tradition, tend to present positive images and rarely confront questions of who or what is left out of their covers. Cultural critiques of yearbooks might educate future advisers and even publisher\u27s representatives

Can integrated titles improve the viewing experience? Investigating the impact of subtitling on the reception and enjoyment of film using eye tracking and questionnaire data

Historically a dubbing country, Germany is not well-known for subtitled productions. But while dubbing is predominant in Germany, more and more German viewers prefer original and subtitled versions of their favourite shows and films. Conventional subtitling, however, can be seen as a strong intrusion into the original image that can not only disrupt but also destroy the director’s intended shot composition and focus points. Long eye movements between focus points and subtitles decrease the viewer’s information intake, and especially German audiences, who are often not used to subtitles, seem to prefer to wait for the next subtitle instead of looking back up again. Furthermore, not only the placement, but also the overall design of conventional subtitles can disturb the image composition – for instance titles with a weak contrast, inappropriate typeface or irritating colour system. So should it not, despite the translation process, be possible to preserve both image and sound as far as possible? Especially given today’s numerous artistic and technical possibilities and the huge amount of work that goes into the visual aspects of a film, taking into account not only special effects, but also typefaces, opening credits and text-image compositions. A further development of existing subtitling guidelines would not only express respect towards the original film version but also the translator’s work.   The presented study shows how integrated titles can increase information intake while maintaining the intended image composition and focus points as well as the aesthetics of the shot compositions. During a three-stage experiment, the specifically for this purpose created integrated titles in the documentary “Joining the Dots” by director Pablo Romero-Fresco were analysed with the help of eye movement data from more than 45 participants. Titles were placed based on the gaze behaviour of English native speakers and then rated by German viewers dependant on a German translation. The results show that a reduction of the distance between intended focus points and titles allow the viewers more time to explore the image and connect the titles to the plot. The integrated titles were rated as more aesthetically pleasing and reading durations were shorter than with conventional subtitles. Based on the analysis of graphic design and filmmaking rules as well as conventional subtitling standards, a first workflow and set of placement strategies for integrated titles were created in order to allow a more respectful handling of film material as well as the preservation of the original image composition and typographic film identity

Telops for language learning: Japanese language learners’ perceptions of authentic Japanese variety shows and implications for their use in the classroom

Research on the use of leisure-oriented media products in foreign language learning is not a novelty. Building further on insights into the effects of audiovisual input on learners, recent studies have started to explore online learning behaviour. This research employed an exploratory design to examine the perceptions of a Japanese variety show with intralingual text, known as telops, by Japanese Language Learners (JLLs) and native Japanese speakers through a multimodal transcript, eye-tracking technology, questionnaires, and field notes. Two main objectives underlie this study: (1) to gain insights into participants’ multimodal perceptions and attitudes towards the use of such authentic material for language learning, and (2) to gain a better understanding of the distribution of participants’ visual attention between stimuli. Data from 43 JLLs and five native Japanese speakers were analysed. The JLLs were organised into a pre-exchange, exchange and post-exchange group while the native Japanese speakers functioned as the reference group. A thematic analysis was conducted on the open-ended questionnaire responses and Areas Of Interest (AOIs) were grouped to generate fixation data. The themes suggest that all learner groups feel that telops help them link the stimuli in the television programme although some difficulty was experienced with the amount and pace of telops in the pre-exchange and exchange groups. The eye-tracking results show that faces and telops gather the most visual attention from all participant groups. Less clear-cut trends in visual attention are detected when AOIs on telops are grouped according to the degree in which they resemble the corresponding dialogue. This thesis concludes with suggestions as to how such authentic material can complement Japanese language learning

Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools

Through Google-Colored Glass(es): Design, Emotion, Class, and Wearables as Commodity and Control

This chapter discusses the implications of wearable technologies like Google Glass that function as a tool for occupying, commodifying, and profiting from the bio- logical, psychological, and emotional data of its wearers and those who fall within its gaze. We argue that Google Glass privileges an imaginary of unbridled exploration and intrusion into the physical and emotional space of others. Glass’s recognizable esthetic and outward-facing camera has elicited intense emotional response, partic- ularly when “exploration” has taken place in areas of San Francisco occupied by residents who were finding themselves priced out or evicted from their homes to make way for the techno-elite. We find that very few trade and popular press articles have focused on the failure of Glass along these dimensions, while the surveillance and class-based aspects of Google Glass are fundamental to an accurate rendering of the product’s trajectory and the public’s emotional response to this product. The goal of this chapter is to foreground dimensions of surveillance and economics, class and resistance, in the face of unending rollouts of new wearable products designed to integrate seamlessly with everyday life—for those, of course, who can afford them. Ultimately, we believe more nuanced, intersectional analyses of power along race, class, and gender must be at the forefront of future research on wearable technologies. Our goal is to raise important critiques of the commodification of emotions, and the expansion of the surveillance state vis-à-vis Google’s increasing and unrivaled information empire, the longstanding social costs of which have yet to be fully articulated