Designing novel applications for multimedia

Current R&D in Multimedia technology is advancing in a fierce rate and will sure to become part of our important regular items in a 'conventional' technology inventory in near future. While the R&D nature of this technology means its accuracy, reliability and robustness are not sufficient enough to be used in real world yet, we want to envision NOW the near-future where this technology will have matured and used in real applications in order to explore and start shaping many possible new ways this novel technology could be utilised. In this talk, some of this effort in designing novel applications that incorporate Multimedia technology as their backend will be presented. Looking at possible future applications of the outcomes of the Multimedia research will be useful for us in re-visiting our own initial ideas and visions when such technologies were first conceived. In addition, with some representative examples of such novel applications the problems and issues will be raised in interaction design research for novel applications and how somewhat different Human-Computer Interaction approach is required in practice

Issues in designing novel applications for multimedia technologies

Emerging computational multimedia tools and techniques promise powerful ways to organise, search and browse our ever-increasing multimedia contents by automating annotation and indexing, augmenting meta-data, understanding media contents, linking related pieces of information amongst them, and providing intriguing visualisation and exploration front-ends. Identifying real-world scenarios and designing interactive applications that leverage these developing multimedia technology is certainly an important research topic in itself but poses a number of challenges. In this talk, I will discuss and highlight some of these challenges in designing these novel applications by reflecting on my own design practice with a number of design examples

Searching the Físchlár-NEWS archive on a mobile device

The Físchlár-NEWS system provides web-based access to an archive of digitally recorded TV News broadcasts over several months, and has been operational for over a year. Users can browse keyframes, search teletext and have streamed video playback of segments of news broadcasts to their desktops. This paper reports on the development of mFíschlár-NEWS, a version of Físchlár-NEWS which operates on a mobile PDA over a wireless LAN connection. In the design and development of mFíschlár-NEWS we have realised that mobile access to a digital library of video materials is more than just the desktop system on a smaller screen, and the functionality and role that information retrieval techniques play in the mFíschlár-NEWS system are very different to what is present in the desktop system. The paper describes the design, interface, functionality and operational status of this mobile access to a video library

Interaction platform-orientated perspective in designing novel applications

The lack of HCI offerings in the invention of novel software applications and the bias of design knowledge towards desktop GUI make it difficult for us to design for novel scenarios and applications that leverage emerging computational technologies. These include new media platforms such as mobiles, interactive TV, tabletops and large multi-touch walls on which many of our future applications will operate. We argue that novel application design should come not from user-centred requirements engineering as in developing a conventional application, but from understanding the interaction characteristics of the new platforms. Ensuring general usability for a particular interaction platform without rigorously specifying envisaged usage contexts helps us to design an artifact that does not restrict the possible application contexts and yet is usable enough to help brainstorm its more exact place for future exploitation

Establishing the design knowledge for emerging interaction platforms

While awaiting a variety of innovative interactive products and services to appear in the market in the near future such as interactive tabletops, interactive TVs, public multi-touch walls, and other embedded appliances, this paper calls for preparation for the arrival of such interactive platforms based on their interactivity. We advocate studying, understanding and establishing the foundation for interaction characteristics and affordances and design implications for these platforms which we know will soon emerge and penetrate our everyday lives. We review some of the archetypal interaction platform categories of the future and highlight the current status of the design knowledge-base accumulated to date and the current rate of growth for each of these. We use example designs illustrating design issues and considerations based on the authors’ 12-year experience in pioneering novel applications in various forms and styles

Designing novel applications inspired by emerging media technologies

The field of Human-Computer Interaction provides a number of useful tools and methods for obtaining information on end-users and their usage context to inform the design of computer systems, yet relatively little is known on how to go about designing for a completely novel application where there is no user base, no existing practice of use available at the start. The success of the currently available HCI methodology that focuses on understanding users’ needs and establishing requirements is well-deserved in making computing applications usable in terms of fitting them to end-users’ usage contexts. However, too much emphasis on identifying user needs tends to stifle other more exploratory design activities where new types of applications are invented in order to discover or create new activities currently not practiced. In this paper, we argue that a great starting point of novel application design is not the problem space (trying to rigorously define the user requirements) but the solution space (trying to leverage emerging computational technologies and growing design knowledge for various interaction platforms), and we build a foundation for a pragmatic design methodology supported by the authors’ extensive experience in designing novel applications inspired by emerging media technologies

Designing novel applications inspired by emerging media technologies

The lack of HCI provisions in the invention of novel software applications and the bias of design knowledge towards desktop GUI make it difficult for us to design for novel scenarios and applications that leverage emerging computational technologies. These include new media platforms such as mobiles, interactive TV, tabletops and large multi-touch walls on which many of our future applications will operate. In this presentation I will discuss that novel application design should come not from user-centred requirements engineering as in developing a conventional application, but from understanding the interaction characteristics of the new platforms. Ensuring general usability for a particular interaction platform without rigorously specifying envisaged usage contexts helps us to design an artifact that does not restrict the possible application contexts and yet is usable enough to help brainstorm its more exact place for future exploitation. With the themes on Multitouch, I will give examples of design effort currently underway with mobile and Multitouch Wall applications, and also raise (possibly) controversial issues in designing such novel applications when there is no user-base or work practice we can base our knowledge on

iForgot: a model of forgetting in robotic memories

Much effort has focused in recent years on developing more life-like robots. In this paper we propose a model of memory for robots, based on human digital memories, though our model incorporates an element of forgetting to ensure that the robotic memory appears more human and therefore can address some of the challenges for human-robot interaction

Interactive searching and browsing of video archives: using text and using image matching

Over the last number of decades much research work has been done in the general area of video and audio analysis. Initially the applications driving this included capturing video in digital form and then being able to store, transmit and render it, which involved a large effort to develop compression and encoding standards. The technology needed to do all this is now easily available and cheap, with applications of digital video processing now commonplace, ranging from CCTV (Closed Circuit TV) for security, to home capture of broadcast TV on home DVRs for personal viewing. One consequence of the development in technology for creating, storing and distributing digital video is that there has been a huge increase in the volume of digital video, and this in turn has created a need for techniques to allow effective management of this video, and by that we mean content management. In the BBC, for example, the archives department receives approximately 500,000 queries per year and has over 350,000 hours of content in its library. Having huge archives of video information is hardly any benefit if we have no effective means of being able to locate video clips which are of relevance to whatever our information needs may be. In this chapter we report our work on developing two specific retrieval and browsing tools for digital video information. Both of these are based on an analysis of the captured video for the purpose of automatically structuring into shots or higher level semantic units like TV news stories. Some also include analysis of the video for the automatic detection of features such as the presence or absence of faces. Both include some elements of searching, where a user specifies a query or information need, and browsing, where a user is allowed to browse through sets of retrieved video shots. We support the presentation of these tools with illustrations of actual video retrieval systems developed and working on hundreds of hours of video content

User evaluation outside the lab: the trial of Físchlár-News

A user study of Físchlár-News system was conducted in Spring 2004 with 16 users, each user using the system for a 1-month period. Físchlár-News is an experimental online news archive that incorporates various automatic content-based video indexing techniques and a news story recommender algorithm to process and index the daily 9 o’clock broadcast news from TV and allows its users to browse, search, be recommended, and play news stories on a conventional web browser. Pre and post-trial questionnaires, interaction logging and incident diary methods collected both qualitative and quantitative usage data during the trial period. While the details of the findings from this evaluation is reported elsewhere, in this paper we report the details of the methodology taken and our experience of conducting this evaluation