Creating stories for reflection from multimodal lifelog content: An initial investigation

Using lifelogging tools, digital artifacts can be collected continuously and passively throughout our day. These may include a stream of images recorded passively using tools such as the Microsoft SenseCam; documents, emails and webpages accessed; texts messages and mobile activity; and context sensing to uncover the current location and proximal individuals. The wealth of information such an archive contains on our personal life history provides us with the opportunity to review, reflect and reminisce upon our past experience. However, the complexity, volume and multimodal nature of such collections creates a barrier to such activities. We are currently exploring the potential of digital narratives formed from these collections as a means to overcome these challenges. By successfully reducing the content to that most appropriate to the story, and by then presenting it in a coherent and usable manner, we can hope to better enable reflection. The means by which content reduction and presentation should occur is investigated through card sorting activities and probe sessions which nine participants engaged in. The initial results are discussed, as well as the opportunity, as seen in these sessions, for lifelog-based stories to provide utility in personal reflection and reminiscence

Machine Understanding of Human Behavior

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior

Exploiting multimedia in creating and analysing multimedia Web archives

The data contained on the web and the social web are inherently multimedia and consist of a mixture of textual, visual and audio modalities. Community memories embodied on the web and social web contain a rich mixture of data from these modalities. In many ways, the web is the greatest resource ever created by human-kind. However, due to the dynamic and distributed nature of the web, its content changes, appears and disappears on a daily basis. Web archiving provides a way of capturing snapshots of (parts of) the web for preservation and future analysis. This paper provides an overview of techniques we have developed within the context of the EU funded ARCOMEM (ARchiving COmmunity MEMories) project to allow multimedia web content to be leveraged during the archival process and for post-archival analysis. Through a set of use cases, we explore several practical applications of multimedia analytics within the realm of web archiving, web archive analysis and multimedia data on the web in general

Using Visual Journals as a Reflective Worldview Window into Educator Identity

This ethnographic case study research and content analysis presents the conclusion of a three-year study involving 37 teacher candidate participants across a three-year study within a two year (2 semester program) Bachelor of Education program at a university in Ontario, Canada. Each academic year participants were intentionally given time over two semesters of literacy courses to engage in literacy practices and knowledge of self through the use of multimodal visual journals. Candidates reflect on their conceptions of literacy, teaching, identity and worldview within an institution grounded in the Christian faith. Findings, philosophical ponderings and content analysis suggest that the identity of the teacher candidate filters learning through visual and multimodal ways. The findings raise questions about the place of multimodal learning, self-reflection, faith and worldview in the learning process, and in identity formation of educators. We suggest that this study may inform current multimodal and visual literacy research while generating enriching discussions on how multimodal forms of literacy instruction may assist in acknowledgement of worldview recognition and self-identity awareness. Keywords: Multiliteracies, visual journals, self-knowledge, worldview, identity, visual literacy, multimodal literacy, teacher educatio

Doing new things with texts : multimodality and Mao’s Last Dancer

This paper describes a senior, multimodal task developed by Shauna O’Connor and the English staff at Brigidine College after consultation in the form of media workshops with Anita Jetnikoff. Gunther Kress (2006) suggested recently that due to the affordances of media platforms such as Web 2.0, “we need to be doing new things with texts”. The year 11 unit’s Finding a Voice parent text was the memoir, Mao’s last Dancer. The summative assessment task morphed over time from an ‘identity portrait’, into ‘a multimodal, first person narrative’

An information assistant system for the prevention of tunnel vision in crisis management

In the crisis management environment, tunnel vision is a set of bias in decision makers’ cognitive process which often leads to incorrect understanding of the real crisis situation, biased perception of information, and improper decisions. The tunnel vision phenomenon is a consequence of both the challenges in the task and the natural limitation in a human being’s cognitive process. An information assistant system is proposed with the purpose of preventing tunnel vision. The system serves as a platform for monitoring the on-going crisis event. All information goes through the system before arrives at the user. The system enhances the data quality, reduces the data quantity and presents the crisis information in a manner that prevents or repairs the user’s cognitive overload. While working with such a system, the users (crisis managers) are expected to be more likely to stay aware of the actual situation, stay open minded to possibilities, and make proper decisions

When Memories Make a Difference: Multimodal Literacy Narratives for Preservice ELA Methods Students

This article examines multimodal literacy narrative projects designed by students in a methods of teaching course for secondary preservice English Language Arts teachers. For the multimodal project, preservice teachers infused written, audio, and visual text using a variety of creative mediums. Through combined theoretical frames, the researcher explores semiotics and preservice teachers’ use of multiliteracies as they shift their conceptions of what it means to compose. Finally, this article explores how the act of reflection through the literacy narrative influences preservice teachers’ notions of teaching composition through a variety of mediums

Affective Interaction in Smart Environments

AbstractWe present a concept where the smart environments of the future will be able to provide ubiquitous affective communication. All the surfaces will become interactive and the furniture will display emotions. In particular, we present a first prototype that allows people to share their emotional states in a natural way. The input will be given through facial expressions and the output will be displayed in a context-aware multimodal way. Two novel output modalities are presented: a robotic painting that applies the concept of affective communication to the informative art and an RGB lamp that represents the emotions remaining in the user's peripheral attention. An observation study has been conducted during an interactive event and we report our preliminary findings in this paper