Annotation of multimedia learning materials for semantic search

Multimedia is the main source for online learning materials, such as videos, slides and textbooks, and its size is growing with the popularity of online programs offered by Universities and Massive Open Online Courses (MOOCs). The increasing amount of multimedia learning resources available online makes it very challenging to browse through the materials or find where a specific concept of interest is covered. To enable semantic search on the lecture materials, their content must be annotated and indexed. Manual annotation of learning materials such as videos is tedious and cannot be envisioned for the growing quantity of online materials. One of the most commonly used methods for learning video annotation is to index the video, based on the transcript obtained from translating the audio track of the video into text. Existing speech to text translators require extensive training especially for non-native English speakers and are known to have low accuracy. This dissertation proposes to index the slides, based on the keywords. The keywords extracted from the textbook index and the presentation slides are the basis of the indexing scheme. Two types of lecture videos are generally used (i.e., classroom recording using a regular camera or slide presentation screen captures using specific software) and their quality varies widely. The screen capture videos, have generally a good quality and sometimes come with metadata. But often, metadata is not reliable and hence image processing techniques are used to segment the videos. Since the learning videos have a static background of slide, it is challenging to detect the shot boundaries. Comparative analysis of the state of the art techniques to determine best feature descriptors suitable for detecting transitions in a learning video is presented in this dissertation. The videos are indexed with keywords obtained from slides and a correspondence is established by segmenting the video temporally using feature descriptors to match and align the video segments with the presentation slides converted into images. The classroom recordings using regular video cameras often have poor illumination with objects partially or totally occluded. For such videos, slide localization techniques based on segmentation and heuristics is presented to improve the accuracy of the transition detection. A region prioritized ranking mechanism is proposed that integrates the keyword location in the presentation into the ranking of the slides when searching for a slide that covers a given keyword. This helps in getting the most relevant results first. With the increasing size of course materials gathered online, a user looking to understand a given concept can get overwhelmed. The standard way of learning and the concept of “one size fits all” is no longer the best way to learn for millennials. Personalized concept recommendation is presented according to the user’s background knowledge. Finally, the contributions of this dissertation have been integrated into the Ultimate Course Search (UCS), a tool for an effective search of course materials. UCS integrates presentation, lecture videos and textbook content into a single platform with topic based search capabilities and easy navigation of lecture materials

Exploring Design Options for interactive Video with the Mnemovie hypervideo system

Mnemovie is an investigative hypervideo system for exploring design options for interactivity with digital motion picture files (video). The custom-designed software toolset is used to build a series of experimental interactive models from which three models were subsequently developed for initial user experience testing and evaluation. We compared interaction with each of the models across three groups of video file users, from expert to non-expert. Understanding participants preference for each model helps define the different dimensions of the actual user experience. We discuss how these findings and the subsequent development of persona scenarios can inform the design of hypervideo systems and the implications this has for interaction design

Framework for Knowledge Discovery in Educational Video Repositories

The ease of creating digital content coupled with technological advancements allows institutions and organizations to further embrace distance learning. Teaching materials also receive attention, because it is difficult for the student to obtain adequate didactic material, being necessary a high effort and knowledge about the material and the repository. This work presents a framework that enables the automatic metadata generation for materials available in educational video repositories. Each module of the framework works autonomously and can be used in isolation, complemented by another technique or replaced by a more appropriate approach to the field of use, such as repositories with other types of media or other content

Reasoning & Querying – State of the Art

Various query languages for Web and Semantic Web data, both for practical use and as an area of research in the scientific community, have emerged in recent years. At the same time, the broad adoption of the internet where keyword search is used in many applications, e.g. search engines, has familiarized casual users with using keyword queries to retrieve information on the internet. Unlike this easy-to-use querying, traditional query languages require knowledge of the language itself as well as of the data to be queried. Keyword-based query languages for XML and RDF bridge the gap between the two, aiming at enabling simple querying of semi-structured data, which is relevant e.g. in the context of the emerging Semantic Web. This article presents an overview of the field of keyword querying for XML and RDF

A survey of exploratory search systems based on LOD resources

The fact that the existing Web allows people to effortlessly share data over the Internet has resulted in the accumulation of vast amounts of information available on the Web.Therefore, a powerful search technology that will allow retrieval of relevant information is one of the main requirements for the success of the Web which is complicated further due to use of many different formats for storing information. Semantic Web technology plays a major role in resolving this problem by permitting the search engines to retrieve meaningful information. Exploratory search system, a special information seeking and exploration approach, supports users who are unfamiliar with a topic or whose search goals are vague and unfocused to learn and investigate a topic through a set of activities. In order to achieve exploratory search goals Linked Open Data (LOD) can be used to help search systems in retrieving related data, so the investigation task runs smoothly.This paper provides an overview of the Semantic Web Technology, Linked Data and search strategies, followed by a survey of the state of the art Exploratory Search Systems based on LOD.Finally the systems are compared in various aspects such as algorithms, result rankings and explanations