Interactive searching and browsing of video archives: using text and using image matching

Over the last number of decades much research work has been done in the general area of video and audio analysis. Initially the applications driving this included capturing video in digital form and then being able to store, transmit and render it, which involved a large effort to develop compression and encoding standards. The technology needed to do all this is now easily available and cheap, with applications of digital video processing now commonplace, ranging from CCTV (Closed Circuit TV) for security, to home capture of broadcast TV on home DVRs for personal viewing. One consequence of the development in technology for creating, storing and distributing digital video is that there has been a huge increase in the volume of digital video, and this in turn has created a need for techniques to allow effective management of this video, and by that we mean content management. In the BBC, for example, the archives department receives approximately 500,000 queries per year and has over 350,000 hours of content in its library. Having huge archives of video information is hardly any benefit if we have no effective means of being able to locate video clips which are of relevance to whatever our information needs may be. In this chapter we report our work on developing two specific retrieval and browsing tools for digital video information. Both of these are based on an analysis of the captured video for the purpose of automatically structuring into shots or higher level semantic units like TV news stories. Some also include analysis of the video for the automatic detection of features such as the presence or absence of faces. Both include some elements of searching, where a user specifies a query or information need, and browsing, where a user is allowed to browse through sets of retrieved video shots. We support the presentation of these tools with illustrations of actual video retrieval systems developed and working on hundreds of hours of video content

Investigating design features of a computer-mediated communication system

Computer-mediated communication (CMC) is increasingly used in higher education, but it is not without problems. The effectiveness of CMC depends on many factors, including the characteristics of CMC systems themselves. The research reported here therefore aimed to investigate how an educational CMC system might be improved, in order to support learning more effectively. The main context for the research was distance learning at the UK Open University (OU). A two-stage, mixed methods research approach was adopted. In the first stage, interviews and observations were carried out to explore the benefits and problems experienced by users. This revealed two major issues: information overload and lack of social presence. Information overload relates to users’ problems dealing with large numbers of messages. Social presence relates to the need for users to feel connected with each other. The second stage investigated system features aimed at addressing these issues, implemented in a prototype computer conferencing system. Features to address overload included branched message threading and user recommendations. Features to address social presence were ‘résumés’ and instant messaging. These features were evaluated using questionnaires, with several cohorts of students in an OU course. Students expressed approval of the features, although some features were not widely used. Students preferred branched threading to chronological threading because branching helped them to follow ‘conversations’. Students were uncomfortable recommending messages, feeling that the value of a message would vary between people. They were also uncomfortable using instant messaging to contact others whom they did not know. However, the awareness aspect of instant messaging provided a sense of solidarity. The research demonstrated that the problems of overload and lack of social presence are significant, and each has social aspects which must be addressed. Students’ relationships with each other affect whether and how they use the features of CMC systems. We can conclude that particular attention must be paid to the social aspects of online communication, both when designing educational CMC systems and when considering how they are used. To maximise the benefits for learning, students need to feel comfortable with each other online, and there are few short cuts to achieving this

Multimodal News Summarization, Tracking and Annotation Incorporating Tensor Analysis of Memes

We demonstrate four novel multimodal methods for efficient video summarization and comprehensive cross-cultural news video understanding. First, For video quick browsing, we demonstrate a multimedia event recounting system. Based on nine people-oriented design principles, it summarizes YouTube-like videos into short visual segments (812sec) and textual words (less than 10 terms). In the 2013 Trecvid Multimedia Event Recounting competition, this system placed first in recognition time efficiency, while remaining above average in description accuracy. Secondly, we demonstrate the summarization of large amounts of online international news videos. In order to understand an international event such as Ebola virus, AirAsia Flight 8501 and Zika virus comprehensively, we present a novel and efficient constrained tensor factorization algorithm that first represents a video archive of multimedia news stories concerning a news event as a sparse tensor of order 4. The dimensions correspond to extracted visual memes, verbal tags, time periods, and cultures. The iterative algorithm approximately but accurately extracts coherent quad-clusters, each of which represents a significant summary of an important independent aspect of the news event. We give examples of quad-clusters extracted from tensors with at least 108 entries derived from international news coverage. We show the method is fast, can be tuned to give preferences to any subset of its four dimensions, and exceeds three existing methods in performance. Thirdly, noting that the co-occurrence of visual memes and tags in our summarization result is sparse, we show how to model cross-cultural visual meme influence based on normalized PageRank, which more accurately captures the rates at which visual memes are reposted in a specified time period in a specified culture. Lastly, we establish the correspondences of videos and text descriptions in different cultures by reliable visual cues, detect culture-specific tags for visual memes and then annotate videos in a cultural settings. Starting with any video with less text or no text in one culture (say, US), we select candidate annotations in the text of another culture (say, China) to annotate US video. Through analyzing the similarity of images annotated by those candidates, we can derive a set of proper tags from the viewpoints of another culture (China). We illustrate cultural-based annotation examples by segments of international news. We evaluate the generated tags by cross-cultural tag frequency, tag precision, and user studies

A Structured Approach to Online Discussion

Online message boards have transformed public discussion, allowing anyone with internet access to share their thoughts on a broad range of topics. While they present users with a wealth of information, online message boards currently do not provide an effective way to make sense of it. These systems are typically designed as unstructured lists of comments with no overview. This lack of structure provides little incentive for users to interact in ways that benefit the collective, resulting in poor contributions and behaviours that lower the overall quality of discussion. These problems are expected to worsen as the activity on online message boards increase.This dissertation aims to investigate how online message boards can be designed to structure and facilitate online discussion. To start, a novel structured discussion flow is conceptualised. Each step of the discussion flow increases the affordances and information available to the user. This discussion flow is then implemented into Potluck, a working online message board. Potluck is designed to (1) have users actively participate in the discussion and express their views without social influence; (2) help users make sense of the discussion by automatically collecting and summarising similar viewpoints; and (3) add structure and encourage reflection of different viewpoints by having users recursively answer and ask questions.This work is evaluated through three field deployments of Potluck in professional, educational, and event-based settings. Results suggest that the proposed discussion flow and system provides support for different forms of engagement; gives users a relevant overview of the discussion; encourages normatively desirable user participation; and is usable by different user groups. The studies also reveal potential applications of the system beyond online discussion to be explored in future work. Ultimately, design considerations are presented for system designers and engineers to build more effective online message boards than are currently available

The Construction of Shared Knowledge through Asynchronous Dialogue

This thesis investigates how groups of learners use asynchronous dialogue to build shared knowledge together over time. To do this, it takes a sociocultural approach, with a situated focus on learners' social and temporal settings as well as on the tools they employ. It utilises concepts developed to support understanding of knowledge co-construction in face- to-face environments, particularly the social modes of thinking identified by Mercer and his colleagues (Mercer, 1995, 2000, 2002; Mercer & Littleton, 2007; Mercer & Wegerif, 1999) and the improvable objects described by Wells (1999). Analysis shows that, over short periods of time, groups of learners construct shared vocabulary, history and understanding slowly through the use of a series of discursive devices including those identified here as 'constructive synthesis', the 'proposal pattern' and 'powerful synthesis'. Over longer periods they may engage in 'attached dialogue', a form of asynchronous dialogue that is mediated by improvable objects. The development of these improvable objects involves learners engaging in exploratory dialogue that builds into progressive discourse, a coordinated form of co-reasoning in language. While doing this, they actively work to avoid unproductive interaction by consistently shifting responsibility from the individual to the group. Previous studies have suggested that asynchronous dialogue may act to limit learners to cumulative exchanges (Littleton & Whitelock, 2005; Wegerif, 1998). The analysis over time presented here shows that asynchronous exchanges are enriched by the use of textual affordances that are not available in speech. In the case of attached dialogue, groups of learners are prompted to share knowledge, challenge ideas, justify opinions, evaluate evidence and consider options in a reasoned and equitable way. They do this more successfully when their co-construction of knowledge is not solely task-focused but also focuses on tool use and on the development of social knowledge about the group

ECSCW 2013 Adjunct Proceedings The 13th European Conference on Computer Supported Cooperative Work 21 - 25. September 2013, Paphos, Cyprus

This volume presents the adjunct proceedings of ECSCW 2013.While the proceedings published by Springer Verlag contains the core of the technical program, namely the full papers, the adjunct proceedings includes contributions on work in progress, workshops and master classes, demos and videos, the doctoral colloquium, and keynotes, thus indicating what our field may become in the future

Haunted/Haunting Digital Archives of the Fukushima Nuclear Disaster: Weaving Ghost Stories around the Ongoing Disaster in the Past, Present and Future

This thesis examines the digital archives of the Fukushima nuclear disaster that took place on 11 March, 2011. I propose key thesis questions regarding the roles of the digital archive in articulating the memory and knowledge about the disaster, in relation to its capacity of storytelling. I specifically focus on the production of “ghost stories,” the stories concerning exclusions and invisibilities produced in the digital archive as a flexible, transformative vehicle of ephemeral data. This research draws on interdisciplinary discussions in the fields of media studies, sociology and archival studies, as well as the contributions of feminism and queer theory to delineating the struggles to engage with lost histories and submerged narratives. My contribution is both theoretical and methodological, in developing hauntology as a way of intervening to temporal and narrative modalities of the practices of digital archiving. In formulating hauntological methods, I attend to the creation of “haunted data” and the contingent dis/appearance of digital traces, which have allowed me to employ archival imaginaries to take into account gaps, absences and erasures as a constitutive part of archival storytelling. I also aim to demonstrate a multivalence of haunting at work in the mutual construction of the archive and the archived, with the Fukushima disaster as both haunted and haunting object of inquiry. The digital archives I analyse in the empirical chapters are: two archival repositories on the website of the Tokyo Electric Power Company (TEPCO) that owns the damaged plant; the Japan Disasters Digital Archive (JDA); SimplyInfo.org and Nukewatch.org; Teach311.org. They are “moving” repositories that keep archival objects in motion, and I ask how they articulate and bring together the fragments of the disaster, by intervening to, and generating the intricate web of connections between the past, present and future. Throughout the thesis, I argue that the constant and contingent retelling of the Fukushima disaster in the practices of digital archiving calls attention to narrative possibilities afforded by digital technologies. This research explores how the production of the digital archive entails the conflation of fact and fiction, of multiple temporalities that register different facets of haunting, and myriad regimes of remembering and forgetting, which would shape our understandings of the ongoing disaster with no definitive beginnings and ends

Design Thinking as Heterogeneous Engineering: Emerging Design Methods in Meme Warfare

The shift of production of material artefacts to digital and online making has been greatly disruptive to material culture. Design has typically concerned itself with studying material cultures in order to develop a better understanding of the ways people go about shaping the world around them. This thesis contributes to this space by looking at an emerging form of artefact generation in digital and online making, namely, visual communication design in online information warfare. Developing understanding of participation in this space reveals possible trajectory of working with material culture as it increasingly becomes digital and online. Marshall McLuhan wrote in 1970 that “World War 3 is a guerrilla information war with no division between military and civilian participation” (p. 66), anticipating ubiquitous symmetrical capacity of users as both producers and consumers of information through communication technology. This space has emerged as our digital and online environment, and prominent in this environment are images with characteristics of visual communication design. It appears that the trajectory of visual communication design from the late 19t h century is moving toward ubiquitous making and exchanging of visual communication, as anyone with a smartphone can make an internet meme with worldwide reach and influence