Rethinking world language teacher education TPACK for integration of digital literacies in the classroom

Studies indicate that many language teachers have a tendency to view language as an abstract linguistic system and are, therefore, hesitant to acknowledge new dimensions of literacy and that learning a language in the digital age involves new communicative competencies including the ability to construct knowledge collaboratively and create and interpret texts that combine various resources made available by digital technologies. The main purpose of this thesis was to investigate the Technological Pedagogical Content Knowledge (TPACK) of language teachers engaged in the digital literacy practice of producing a multimodal ensemble with machinima with a view to proposing an updated TPACK model for integration of digital literacies into language teacher education. To this end, language teachers participated in a course specifically designed to train them to make machinima videos as well as prompt them to reflect on the affordances of the tool and their transformative effect on the concepts of language and literacy. Findings show that while participating teachers express traditional views of literacy, they demonstrate profound knowledge of multimodal composition by collaboratively constructing complex mode relationships during the machinima production process. Findings also suggest that if digital literacies are seen as encompassing the ability to adapt affordances and constraints of digital technologies to particular circumstances, then, teachers possess digital literacies as they enact the affordances and overcome the constraints of digital technologies through synaesthesia, spontaneous improvising and coaction. This thesis proposes a reconceptualisation of the Content Knowledge domain to include ecological perspectives on language and language learning and teaching and a metalanguage that would enable teachers to discuss and explain the creation of various mode relationships enabled by digital tools. The TPACK model proposed in this thesis allows for the consideration of concepts such as multimodal meaning-making, synaesthesia and coaction which are deemed to be relevant to a discussion of digital literacies within language teacher education programmes

Performative Authoring: Nurturing Children’s Creativity and Creative Self-Efficacy through Digitally-Augmented Enactment-Based Storytelling

Psychological research, especially by Paul Torrance, has shown that the child’s engagement in creative activities tends to drop precipitously at around the 3rd- to 4th-grade period (8 to 11 years old). This phenomenon, called the ‘Fourth-Grade Slump’, occurs possibly because of an increase in social awareness and critical self-evaluation of competence in the child during this period. Increasing awareness of the complexity of the world’s problems, new paradigms of design focusing on the user, and advances in technology has led to rapid developments in the design and development of tools to support children’s creativity. Research in creativity support tools has generally focused on augmenting creative performance within specific tasks, using strategies such as facilitating access to information, or exposing the user to a space of possible combinations. Much less studied however, is how tools may help to develop positive attitudes towards creativity in individuals. This is important, especially in systems designed for children where the focus on the development of the person, during critical periods of growth such as the period of the Fourth-Grade Slump, may be said to be of equivalent or greater importance than the support of process or the generation of product. In the domain of storytelling or narrative construction, work in child development, educational pedagogy, social psychology, and performance studies have looked at how to tap into the power of children’s imagination during pretend play to nurture their storytelling abilities and their sense of self-efficacy or confidence. These interventions typically take the form of drama workshops or classroom roleplaying exercises. While results appear to provide good evidence that drama interventions and theater-based methods have some positive effects on children’s development of narratives, studies have shown mixed results in terms of the effects on children’s self-efficacy. I refer here to self-efficacy in the sense of a child’s perception of her creative abilities, in other words, her belief that she can produce creative outcomes. This creativity-oriented sense of self-efficacy has been called ‘creative self-efficacy’. This dissertation investigates how pretend play can be harnessed into the design of an interface to support children’s creativity in storytelling and their sense of creative self-efficacy. This overarching question was explored through four phases of research: Exploration, Design, Evaluation, and Integration. The Exploration phase consisted of two studies: a) a set of interviews with elementary school teachers, and b) an experimental study of how the interface or medium may affect children’s creative storytelling process; The Design phase consisted of two experimental studies, and design and development: a) the first study investigated how the physicality of props may support children’s enactment-based storytelling, and b) the second study explored the influence of the presentation of digital contextual/environmental cues on children’s enactment-based storytelling, c) design and development consisted of an exercise using the NEVO methodology to embody design knowledge gained from the Design phase into a concrete usable system, called DiME; The Evaluation phase consisted of two studies: a) the first was a pilot study that tested the usability of DiME and protocol of use with children, and b) the second was an experimental study across two school districts with different profiles investigating the effects of digitally-augmented enactment-based storytelling using DiME, on children’s creativity, story writing and creative self-efficacy; The Integration phase of the research consisted of a workshop with elementary school teachers, which initiated an exploration into how such a story authoring approach may be used in an elementary school curriculum and setting. The body of work that this dissertation presents elucidates (i) a physical enactment-based method for the authoring of stories by children, and (ii) how a digitally-augmented space may move beyond simple drama methods to positively influence the child’s creativity and imagination during storytelling, as well as her self-belief and motivation to engage in creation. The digitally-augmented enactment-based storytelling environment, that I term performative authoring, allows the child to collaboratively create a story through pretend play with a partner, while her enactments are reflected in real-time in the form of animated cartoon characters and objects on a large screen display through the use of motion tracking technologies. I have found that performative authoring has positive effects not only on the child’s creative self-efficacy, especially for the less extraverted children, but also on the richness of the child’s retelling or written narrative of her story. The significance of the results of the studies is with respect to the various domains and subareas represented (child-computer interaction, interactive storytelling, education and educational psychology, creativity and cognition). There is great potential to extend the concept of exploiting digitally-augmented enactment to support and scaffold higher-level cognition, beyond physical enactment. Extensions of this work include making use of more epistemic forms of enactment, instead of full-blown enactment, to support children’s creative story brainstorming, or to make use of digitally-augmented enactment to support other forms of higher thought apart from creativity and imagination. In the domain of storytelling or narrative construction, work in child development, educational pedagogy, social psychology, and performance studies have looked at how to tap into the power of children’s imagination during pretend play to nurture their storytelling abilities and their sense of self-efficacy or confidence. These interventions typically take the form of drama workshops or classroom roleplaying exercises. While results appear to provide good evidence that drama interventions and theater-based methods have some positive effects on children’s development of narratives, studies have shown mixed results in terms of the effects on children’s self-efficacy. I refer here to self-efficacy in the sense of a child’s perception of her creative abilities, in other words, her belief that she can produce creative outcomes. This creativity-oriented sense of self-efficacy has been called ‘creative self-efficacy’ (Beghetto, 2006, 2007). This dissertation investigates how pretend play can be harnessed into the design of an interface to support children’s creativity in storytelling and their sense of creative self-efficacy. This overarching question was explored through four phases of research: I. Exploration, II. Design, III. Evaluation, and IV. Integration. Phase 1 Exploration consisted of two studies: 1A) a set of interviews with elementary school teachers, and 1B) an experimental study of how the interface or medium may affect children’s creative storytelling process; Phase 2 Design consisted of two experimental studies, and design and development: 2A) the first study investigates how the physicality of props may support children’s enactment-based storytelling, and 2B) the second study explores the influence of the presentation of digital contextual/environmental cues on children’s enactment-based storytelling, 2C) design and development consisted of an exercise using the NEVO methodology to embody design knowledge gained from Phase 2 into a concrete usable system, called DiME; Phase 3 Evaluation consisted of two studies: 3A) the first was a pilot study that tested the usability of DiME and protocol of use with children, and 3B) the second was an experimental study across two school districts with different profiles investigating the effects of digitally-augmented enactment-based storytelling using DiME, on children’s creativity, story writing and creative self-efficacy; and Phase 4 Integration consisted of a workshop with elementary school teachers, which initiated an exploration into how such a story authoring approach may be used in an elementary school curriculum and setting. The body of work that this dissertation presents elucidates (i) a physical enactment-based method for the authoring of stories by children, and (ii) how a digitally-augmented space may move beyond simple drama methods to positively influence the child’s creativity and imagination during storytelling, as well as her self-belief and motivation to engage in creation. The digitally-augmented enactment-based storytelling environment, termed performative authoring in this document, allows the child to collaboratively create a story through pretend play with a partner, while her enactments are reflected in real-time in the form of animated cartoon characters and objects on a large screen display through the use of motion tracking technologies. I have found that performative authoring has positive effects not only on the child’s creative self- efficacy, especially for the less extraverted children, but also on the richness of the child’s retelling or written narrative of her story. This dissertation concludes by discussing the significance of the results of our studies with respect to the various domains and subareas represented (child-computer interaction, interactive storytelling, education and educational psychology, creativity and cognition) and extends the concept of exploiting digitally-augmented enactment to support and scaffold higher-level cognition, beyond physical enactment. Extensions of this work include making use of more epistemic forms of enactment, instead of full-blown enactment, to support children’s creative story brainstorming, or to make use of digitally-augmented enactment to support other forms of higher thought apart from creativity and imagination

To Affinity and Beyond: Interactive Digital Humans as a Human Computer Interface

The field of human computer interaction is increasingly exploring the use of more natural, human-like user interfaces to build intelligent agents to aid in everyday life. This is coupled with a move to people using ever more realistic avatars to represent themselves in their digital lives. As the ability to produce emotionally engaging digital human representations is only just now becoming technically possible, there is little research into how to approach such tasks. This is due to both technical complexity and operational implementation cost. This is now changing as we are at a nexus point with new approaches, faster graphics processing and enabling new technologies in machine learning and computer vision becoming available. I articulate the issues required for such digital humans to be considered successfully located on the other side of the phenomenon known as the Uncanny Valley. My results show that a complex mix of perceived and contextual aspects affect the sense making on digital humans and highlights previously undocumented effects of interactivity on the affinity. Users are willing to accept digital humans as a new form of user interface and they react to them emotionally in previously unanticipated ways. My research shows that it is possible to build an effective interactive digital human that crosses the Uncanny Valley. I directly explore what is required to build a visually realistic digital human as a primary research question and I explore if such a realistic face provides sufficient benefit to justify the challenges involved in building it. I conducted a Delphi study to inform the research approaches and then produced a complex digital human character based on these insights. This interactive and realistic digital human avatar represents a major technical undertaking involving multiple teams around the world. Finally, I explored a framework for examining the ethical implications and signpost future research areas

Not Without Joy: A Reflective Practice Journey To Encouraging Literacy Through The Arts

ABSTRACT Not Without Joy: A Reflective Practice Journey To Encouraging Literacy Through The Arts This thesis chronicles my journey as a kindergarten teacher-researcher as I apply the arts to inform early literacy skills. It is the teacher’s role to drive the curriculum and it is his or her past experiences that will determine its delivery. After researching into my past, and gaining insight into my teaching philosophy, I discovered my passion for educating through the arts. To improve my teaching practice, and uncover the impact of my art driven curriculum, I recorded my actions, wrote daily field notes and collected the works of my students in order to discover the impact my lessons had on their self-perceptions as they became readers. Using the method of reflective practice, I question how teaching through the arts can support my students in visualizing themselves as readers while learning to read. The dissertation focuses on my daily reflections in regards to art activities intended to teach early literacy skills according to the International Reading Associations’ three reading standards. Reflections from these lessons are examined in parallel to the bi-monthly reading-self-portraits created by the students. Themes such as home and school emerge and, as the school year progresses, personal connections to reading are made as the children mature. Their reading-self-portraits evolve to illustrate lessons learned in class and conclusively depict independent reading. By expanding the definition of “literacy” to include new literacies and multimodality, and by validating the views of a reflective practitioner, I connect my teaching methods to my students’ improvement. Through reflective practice, I discover the key ingredient of a successful academic curriculum – joy

Exploring emotions and multimodality in digitally augmented puppeteering

Recently, multimodal and affective technologies have been adopted to support expressive and engaging interaction, bringing up a plethora of new research questions. Among the challenges, two essential topics are 1) how to devise truly multimodal systems that can be used seamlessly for customized performance and content generation, and 2) how to utilize the tracking of emotional cues and respond to them in order to create affective interaction loops. We present PuppetWall, a multi-user, multimodal system intended for digitally augmented puppeteering. This application allows natural interaction to control puppets and manipulate playgrounds comprising background, props, and puppets. PuppetWall utilizes hand movement tracking, a multi-touch display and emotion speech recognition input for interfacing. Here we document the technical features of the system and an initial evaluation. The evaluation involved two professional actors and also aimed at exploring naturally emerging expressive speech categories. We conclude by summarizing challenges in tracking emotional cues from acoustic features and their relevance for the design of affective interactive systems

Reoriented Illustration: Towards the Networked Image

In recent years there has emerged an increasing theoretical and contextual impetus from within the discipline of illustration that would seek to define the practice by authorial approaches to the production and distribution of illustrated content. The priority of this investigation is the attempt to imagine a theoretical landscape or environment in which an ‘authorial turn’ within the discipline might emerge and anchor itself to strategies outside of persisting colloquial or industrial notions of illustration practice. Specifically, this paper aims to tie such thinking to existing practices and concepts relevant to the contemporary construction, distribution and exchange of networked images