ELVIS: Entertainment-led video summaries

© ACM, 2010. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Multimedia Computing, Communications, and Applications, 6(3): Article no. 17 (2010) http://doi.acm.org/10.1145/1823746.1823751Video summaries present the user with a condensed and succinct representation of the content of a video stream. Usually this is achieved by attaching degrees of importance to low-level image, audio and text features. However, video content elicits strong and measurable physiological responses in the user, which are potentially rich indicators of what video content is memorable to or emotionally engaging for an individual user. This article proposes a technique that exploits such physiological responses to a given video stream by a given user to produce Entertainment-Led VIdeo Summaries (ELVIS). ELVIS is made up of five analysis phases which correspond to the analyses of five physiological response measures: electro-dermal response (EDR), heart rate (HR), blood volume pulse (BVP), respiration rate (RR), and respiration amplitude (RA). Through these analyses, the temporal locations of the most entertaining video subsegments, as they occur within the video stream as a whole, are automatically identified. The effectiveness of the ELVIS technique is verified through a statistical analysis of data collected during a set of user trials. Our results show that ELVIS is more consistent than RANDOM, EDR, HR, BVP, RR and RA selections in identifying the most entertaining video subsegments for content in the comedy, horror/comedy, and horror genres. Subjective user reports also reveal that ELVIS video summaries are comparatively easy to understand, enjoyable, and informative

Interactive Film Recombination

In this paper we discuss an innovative media entertainment application called Interactive Movietelling. As an offspring of Interactive Storytelling applied to movies, we propose to integrate narrative generation through AI planning with video processing and modeling to construct filmic variants starting from the baseline content. The integration is possible thanks to content description using semantic attributes pertaining to intermediate-level concepts shared between video processing and planning levels. The output is a recombination of segments taken from the input movie performed so as to convey an alternative plot. User tests on the prototype proved how promising Interactive Movietelling might be, even if it was designed at a proof of concept level. Possible improvements that are suggested here lead to many challenging research issues

Using Web Archives to Enrich the Live Web Experience Through Storytelling

Much of our cultural discourse occurs primarily on the Web. Thus, Web preservation is a fundamental precondition for multiple disciplines. Archiving Web pages into themed collections is a method for ensuring these resources are available for posterity. Services such as Archive-It exists to allow institutions to develop, curate, and preserve collections of Web resources. Understanding the contents and boundaries of these archived collections is a challenge for most people, resulting in the paradox of the larger the collection, the harder it is to understand. Meanwhile, as the sheer volume of data grows on the Web, storytelling is becoming a popular technique in social media for selecting Web resources to support a particular narrative or story . In this dissertation, we address the problem of understanding the archived collections through proposing the Dark and Stormy Archive (DSA) framework, in which we integrate storytelling social media and Web archives. In the DSA framework, we identify, evaluate, and select candidate Web pages from archived collections that summarize the holdings of these collections, arrange them in chronological order, and then visualize these pages using tools that users already are familiar with, such as Storify. To inform our work of generating stories from archived collections, we start by building a baseline for the structural characteristics of popular (i.e., receiving the most views) human-generated stories through investigating stories from Storify. Furthermore, we checked the entire population of Archive-It collections for better understanding the characteristics of the collections we intend to summarize. We then filter off-topic pages from the collections the using different methods to detect when an archived page in a collection has gone off-topic. We created a gold standard dataset from three Archive-It collections to evaluate the proposed methods at different thresholds. From the gold standard dataset, we identified five behaviors for the TimeMaps (a list of archived copies of a page) based on the page’s aboutness. Based on a dynamic slicing algorithm, we divide the collection and cluster the pages in each slice. We then select the best representative page from each cluster based on different quality metrics (e.g., the replay quality, and the quality of the generated snippet from the page). At the end, we put the selected pages in chronological order and visualize them using Storify. For evaluating the DSA framework, we obtained a ground truth dataset of hand-crafted stories from Archive-It collections generated by expert archivists. We used Amazon’s Mechanical Turk to evaluate the automatically generated stories against the stories that were created by domain experts. The results show that the automatically generated stories by the DSA are indistinguishable from those created by human subject domain experts, while at the same time both kinds of stories (automatic and human) are easily distinguished from randomly generated storie

Toward media collection-based storytelling

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (p. 113-118).Life is filled with stories. Modern technologies enable us to document and share life events with various kinds of media, such as photos, videos, etc. But people still find it time-consuming to select and arrange media fragments to create coherent and engaging narratives. This thesis proposes a novel storytelling system called Storied Navigation, which lets users assemble a sequence of video clips based on their roles in telling a story, rather than solely by explicit start and end times. Storied Navigation uses textual annotations expressed in unconstrained natural language, using parsing and Commonsense reasoning to deduce possible connections between the narrative intent of the storyteller, and descriptions of events and characters in the video. It helps users increase their familiarity with a documentary video corpus. It helps them develop story threads by prompting them with recommendations of alternatives as well as possible continuations for each selected video clip. We view it as a promising first step towards transforming today's fragmented media production experience into an enjoyable, integrated storytelling activity.Edward Yu-Te Chen.S.M

Reusing A Compound-based Infrastructure For Searching Video Stories

The fast evolution of technology has led to a growing demand for multimedia data, increasing the amount of research into efficient systems to manage those materials. A lot of research has being done by the Content-Based Image Retrieval (CBIR) community in the field of images. Nowadays, they play a key role in digital applications. Thus, contextual integration of images with different sources is vital. It involves reusing and aggregating a large amount of information with other media types. In particular, if we consider video data, images can be used to summarize videos into storyboards, providing an easy way to navigate and to browse large video collections. This has been the goal of a quickly evolving research area known as video summarization. In this paper, we present a novel approach to reuse the CBIR infrastructure for searching video stories, taking advantage of the compound object (CO) concept to integrate resources. Our approach relies on a specific component technology to encapsulate the CBIR related tasks and integrate them with video summarization techniques, known as Digital Content Component (DCC). Such a strategy provides an effective way to reuse, compose, and aggregate both content and processing software. © 2011 IEEE.222227 IEEE Systems, Man and Cybernetics Society (SMC),Society for Information Reuse and Integration (SIRI)Achananuparp, P., McCain, K.W., Allen, R.B., Supporting student collaboration for image indexing (2007) Int. Conf. Asia-Pacific Digital Libraries (ICADL'07), pp. 24-34Almeida, J., Minetto, R., Almeida, T.A., Torres, R.S., Leite, N.J., Robust estimation of camera motion using optical flow models (2009) Int. Symp. Advances VisualComput. (ISVC'09), pp. 435-446Almeida, J., Minetto, R., Almeida, T.A., Torres, R.S., Leite, N.J., Estimation of camera parameters in video sequences with a large amount of scene motion (2010) IEEE Int. Conf. Syst. Signals and Image Proc. (IWSSIP'10), pp. 348-351Almeida, J., Pinto-Ćaceres, S.M., Torres, R.S., Leite, N.J., Intuitive video browsing along hierarchical trees (2011) Technical Report IC-11-06, , Institute ofComputing, University of CampinasAlmeida, J., Torres, R.S., Leite, N.J., Rapid video summarization oncompressed video (2010) IEEE Int. Symp. Multimedia (ISM'10), pp. 113-120Avila, S.E.F., Lopes, A.P.B., Luz Jr., A., Aráujo, A.A., VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method (2011) Pattern Recognition Letters, 32 (1), pp. 56-68Benini, S., Migliorati, P., Leonardi, R., Retrieval of video story units by markov entropy rate (2008) IEEE Int. Workshop Content-Based Multimedia Indexing (CBMI'08), pp. 41-45Torres, R.S., Falc̃ao, A.X., Content-based image retrieval: Theory and applications (2006) Revista de Inforḿatica Téorica e Aplicada, 13 (2), pp. 161-185Torres, R.D.S., Medeiros, C.B., Gonccalves, M.A., Fox, E.A., A digital library framework for biodiversity information systems (2006) International Journal on Digital Libraries, 6 (1), pp. 3-17. , DOI 10.1007/s00799-005-0124-1Furini, M., Geraci, F., Montangero, M., Pellegrini, M., STIMO: STIll and MOving video storyboard for the web scenario (2010) Multimedia Tools Appl, 46 (1), pp. 47-69Jansen, M., Heeren, W., Dijk, B., Videotrees: Improving video surrogate presentation using hierarchy (2008) IEEE Int. Workhop Content-Based Multimedia Indexing (CBMI'08), pp. 560-567Jung, B., Song, J., Lee, Y.-J., A narrative-based abstraction framework for story-oriented video (2007) ACM Trans. MultimediaComput.Commun. Appl, 3 (2), pp. 1-28Junior, G.Z.P., (2008) Managing the Lifecycle of Sensor Data: From Production to Consumption, , PhD thesis, Institute ofComputing, University of CampinasKozievitch, N.P., Complex objects in digital libraries. ACM/IEEE-CS Joint Conf. Digital Libraries (JCDL'09) (2009) Doctoral ConsortiumKozievitch, N.P., Codio, S., Francois, J.A., Fox, E.A., Torres, R.S., Exploring CBIR concepts in the CTRnet project (2010) Technical Report IC-10-32, , Institute ofComputing, University of CampinasKozievitch, N.P., Torres, R.S., Park, S.H., Fox, E.A., Short, N., Abbott, L., Misra, S., Hsiao, M., Rethinking fingerprint evidence through integration of very large digital libraries (2010) VLDL Workshop at Euro. Conf. on Research and Advanced Techn. for Digital Libraries (ECDL'10), pp. 1-8Lagoze, C., De Sompel, H.V., Compound information objects: The OAI-ORE perspective (2007) Open Archives Initiative Object Reuse and Exchange White Paper, , http://www.openarchives.org/ore/documentsMundur, P., Rao, Y., Yesha, Y., Keyframe-based video summarization using Delaunay clustering (2006) International Journal on Digital Libraries, 6 (2), pp. 219-232. , DOI 10.1007/s00799-005-0129-9Murthy, U., Fox, E.A., Chen, Y., Hallerman, E., Torres, R.S., Ramos, E.J., Falc̃ao, T.R.C., Superimposed image description and retrieval for fish species identification (2009) Euro. Conf. Research Advanced Techn. Digital Libraries (ECDL'09), pp. 285-296Murthy, U., Kozievitch, N.P., Leidig, J., Torres, R.S., Yang, S., Gonçalves, M., Delcambre, L., Fox, E.A., Extending the 5s framework of digital libraries to supportcomplex objects, superimposed information, and content-based image retrieval services (2010) Technical Report TR-10-05, , Virginia Tech, Department ofComputer ScienceNelson, M.L., Van De Sompel, H., IJDL special issue on complex digital objects: Guest editors' introduction (2006) International Journal on Digital Libraries, 6 (2), pp. 113-114. , DOI 10.1007/s00799-005-0127-yPenatti, O.A.B., Torres, R.S., Eva: An evaluation tool forcomparing descriptors in content-based image retrieval tasks (2010) Multimedia Information Retrieval (MIR'10), pp. 413-416Santanche, A., Medeiros, C.B., A component model and infrastructure for a fluid Web (2007) IEEE Transactions on Knowledge and Data Engineering, 19 (2), pp. 324-341. , DOI 10.1109/TKDE.2007.16Santanch̀e, A., Medeiros, C.B., Pastorello Jr., G.Z., Userauthor centered multimedia building blocks (2007) Multimedia Syst, 12 (4), pp. 403-421Shipman, F.M., Girgensohn, A., Wilcox, L., Authoring, viewing, and generating hypervideo: An overview of Hyper-Hitchcock (2008) ACM Trans. MultimediaComput.Commun. Appl, 5 (2), pp. 1-19Stehling, R.O., Nascimento, M.A., Falc̃ao, A.X., Acompact and efficient image retrieval approach based on border/interior pixel classification. in (2002) ACM Int. Conf. Inf. Knowl. Management (CIKM'02), pp. 102-109Truong, B.T., Venkatesh, S., Video abstraction: A systematic review and classification (2007) ACM Trans. MultimediaComput.Commun. Appl, 3 (1), pp. 1-37Veerasamy, A., Belkin, N.J., Evaluation of a tool for visualization of information retrieval results (1996) ACM Int. Conf. Research Develop. Inf. Retrieval (SIGIR'96), pp. 85-9