Information Seeking Support System for E-Learning

[[abstract]]How to provide a multimedia information seeking support system (ISSS) is becoming more and more imperative. In this paper, we provide a multimedia ISSS for e-learning. This system requires flexible support for the modeling of multimedia content models and also supports possible interactivity, transfer of streams multimedia data such as audio, video, text and annotations using network facilities. However, we investigated the existing standards and applications for multimedia documents models such as HTML, MHEG, SMIL, and HyTime etc. We propose a new approach for the modeling of reusable and adaptable multimedia content. We also developed a comprehensive system for advanced multimedia content production: support for recording the presentation, retrieving the content, summarizing the presentation, weaving the presentation and customizing the representation. This approach significantly impacts and supports the multimedia presentation authoring processes in terms of methodology and commercial aspects.[[sponsorship]]IEEE Taipei Section; National Science Council; Ministry of Education; Tamkang University; Asia University; Providence University; The University of Aizu; Lanzhou University[[conferencetype]]國際[[conferencetkucampus]]淡水校園[[conferencedate]]20091203~20091205[[booktype]]紙本[[iscallforpapers]]Y[[conferencelocation]]Tamsui, Taipei, Taiwa

InterActive Reading: Understanding Strategies Learners Use to Study Multimedia Content in Tablet-Based Textbooks

poster abstractActive reading of educational textbooks is a complex meta-cognitive process. The traditional framework for active reading is conceptualized as the combination of three types of actions: annotation (e.g. highlighting and note taking), reorganization (e.g. outlining and summarizing) and browsing (e.g. studying annotations and outlines to prepare for future recall). However, as the traditional textbook paradigm evolves to include interactive, multimedia tablet-based products, dramatic changes are on the horizon for the ways in which educational content is delivered and consumed. Tablet devices allow textbook authors, publishers and developers to integrate multimedia content, such as video, audio, animations and interactive visualizations, with traditional expository text, designed as a browse-able book. However, existing tablet devices (i.e., iPad; Kindle Fire) only offer tools that support traditional active reading learning for text-based content. This research project reports findings of an exploratory qualitative study that examines what new active reading strategies emerge when learners engage with tablet-based multimedia textbooks. Participants were presented with one of two tablet textbooks developed using Apple’s iBook Author. The texts included a number of content forms, including traditional expository text, videos & animations, clickable keywords, image galleries, and interactive information graphics. Concept mapping tests were conducted to determine what students learned during their tablet study sessions, and semi-structured interviews were conducted to determine how easy or difficult it was for participants to actively study videos and animations. Early results suggest that the active learning tools developed for the tablet–namely, highlighting and bookmarking–are not sufficient for multimedia content and new tools must be developed to better support such activities. Future research and development are discussed

Discourse oriented summarization

The meaning of text appears to be tightly related to intentions and circumstances. Context sensitivity of meaning is addressed by theories of discourse structure. Few attempts have been made to exploit text organization in summarization. This thesis is an exploration of what knowledge of discourse structure can do for content selection as a subtask of automatic summarization, and query-based summarization in particular. Query-based summarization is the task of answering an arbitrary user query or question by using content from potentially relevant sources. This thesis presents a general framework for discourse oriented summarization, relying on graphs to represent semantic relations in discourse, and redundancy as a special type of semantic relation. Semantic relations occur on several levels of text analysis (query-relevance, coherence, layout, etc.), and a broad range of textual features may be required to detect them. The graph-based framework facilitates combining multiple features into an integrated semantic model of the documents to summarize. Recognizing redundancy and entailment relations between text passages is particularly important when a summary is generated of multiple documents, e.g. to avoid including redundant content in a summary. For this reason, I pay particular attention to recognizing textual entailment. Within this framework, a three-fold evaluation is performed to evaluate different aspects of discourse oriented summarization. The first is a user study, measuring the effect on user appreciation of using a particular type of knowledge for query-based summarization. In this study, three presentation strategies are compared: summarization using the rhetorical structure of the source, a baseline summarization method which uses the layout of the source, and a baseline presentation method which uses no summarization but just a concise answer to the query. Results show that knowledge of the rhetorical structure not only helps to provide the necessary context for the user to verify that the summary addresses the query adequately, but also to increase the amount of relevant content. The second evaluation is a comparison of implementations of the graph-based framework which are capable of fully automatic summarization. The two variables in the experiment are the set of textual features used to model the source and the algorithm used to search a graph for relevant content. The features are based on cosine similarity, and are realized as graph representations of the source. The graph search algorithms are inspired by existing algorithms in summarization. The quality of summaries is measured using the Rouge evaluation toolkit. The best performer would have ranked first (Rouge-2) or second (Rouge-SU4) if it had participated in the DUC 2005 query-based summarization challenge. The third study is an evaluation in the context of the DUC 2006 summarization challenge, which includes readability measurements as well as various content-based evaluation metrics. The evaluated automatic discourse oriented summarization system is similar to the one described above, but uses additional features, i.e. layout and textual entailment. The system performed well on readability at the cost of content-based scores which were well below the scores of the highest ranking DUC 2006 participant. This indicates a trade-off between readable, coherent content and useful content, an issue yet to be explored. Previous research implies that theories of text organization generalize well to multimedia. This suggests that the discourse oriented summarization framework applies to summarizing multimedia as well, provided sufficient knowledge of the organization of the (multimedia) source documents is available. The last study in this thesis is an investigation of the applicability of structural relations in multimedia for generating picture-illustrated summaries, by relating summary content to picture-associated text (i.e. captions or surrounding paragraphs). Results suggest that captions are the more suitable annotation for selecting appropriate pictures. Compared to manual illustration, results of automatic pictures are similar if the manual picture is mainly decorative

Movies Tags Extraction Using Deep Learning

Retrieving information from movies is becoming increasingly demanding due to the enormous amount of multimedia data generated each day. Not only it helps in efficient search, archiving and classification of movies, but is also instrumental in content censorship and recommendation systems. Extracting key information from a movie and summarizing it in a few tags which best describe the movie presents a dedicated challenge and requires an intelligent approach to automatically analyze the movie. In this paper, we formulate movies tags extraction problem as a machine learning classification problem and train a Convolution Neural Network (CNN) on a carefully constructed tag vocabulary. Our proposed technique first extracts key frames from a movie and applies the trained classifier on the key frames. The predictions from the classifier are assigned scores and are filtered based on their relative strengths to generate a compact set of most relevant key tags. We performed a rigorous subjective evaluation of our proposed technique for a wide variety of movies with different experiments. The evaluation results presented in this paper demonstrate that our proposed approach can efficiently extract the key tags of a movie with a good accuracy

Video summarisation: A conceptual framework and survey of the state of the art

This is the post-print (final draft post-refereeing) version of the article. Copyright @ 2007 Elsevier Inc.Video summaries provide condensed and succinct representations of the content of a video stream through a combination of still images, video segments, graphical representations and textual descriptors. This paper presents a conceptual framework for video summarisation derived from the research literature and used as a means for surveying the research literature. The framework distinguishes between video summarisation techniques (the methods used to process content from a source video stream to achieve a summarisation of that stream) and video summaries (outputs of video summarisation techniques). Video summarisation techniques are considered within three broad categories: internal (analyse information sourced directly from the video stream), external (analyse information not sourced directly from the video stream) and hybrid (analyse a combination of internal and external information). Video summaries are considered as a function of the type of content they are derived from (object, event, perception or feature based) and the functionality offered to the user for their consumption (interactive or static, personalised or generic). It is argued that video summarisation would benefit from greater incorporation of external information, particularly user based information that is unobtrusively sourced, in order to overcome longstanding challenges such as the semantic gap and providing video summaries that have greater relevance to individual users