Indexing and retrieval of multimodal lecture recordings from open repositories for personalized access in modern learning settings

An increasing number of lecture recordings are available to complement face-to face and the more conventional content-based e-learning approaches. These recordings provide additional channels for remote students and time-independent access to the lectures. Many universities offer even complete series of recordings of hundreds of courses which are available for public access and this service provides added value for users outside the university. The lecture recordings show the use of a great variety of media or modalities (such as video, audiom lecture media, presentation behaviour) and formats. Insofar, none of the existing systems and services have sufficient retrieval functionality or support appropriate interfaces to enable searching for lecture recordings over several repositories. This situation has motivated us to initiate research on a lecture recording indexing and retrieval system for knowledge transfer and learning activities in various settings. This system is built on our former experiences and prototypes developed within the MISTRAL research project. In this paper we outline requirements for an enhanced lecture recording retrieval system, introduce our solution and prototype, and discuss the initial results and findings

Asynchronously Replicated Shared Workspaces for a Multi-Media Annotation Service over Internet

This paper describes a world wide collaboration system through multimedia Post-its (user generated annotations). DIANE is a service to create multimedia annotations to every application output on the computer, as well as to existing multimedia annotations. Users collaborate by registering multimedia documents and user generated annotation in shared workspaces. However, DIANE only allows effective participation in a shared workspace over a high performance network (ATM, fast Ethernet) since it deals with large multimedia object. When only slow or unreliable connections are available between a DIANE terminal and server, useful work becomes impossible. To overcome these restrictions we need to replicate DIANE servers so that users do not suffer degradation in the quality of service. We use the asynchronous replication service ODIN to replicate the shared workspaces to every interested site in a transparent way to users. ODIN provides a cost-effective object replication by building a dynamic virtual network over Internet. The topology of this virtual network optimizes the use of network resources while it satisfies the changing requirements of the users

Mobile and web tools for participative learning

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa, para a obtenção do grau de Mestre em Engenharia InformáticaThe combination of different media formats has been a crucial aspect on teaching and learning processes. The recent developments of multimedia technologies over the Internet and using mobile devices can improve the communication between professors and students, and allow students to study anywhere and anytime, allowing each student progress at its own pace. The usage of these new platforms and the increase of multimedia sharing applied to educational environments allow a more participative learning, and make the study of interfaces a relevant aspect of existing multimedia learning systems. The work done in this dissertation explores interfaces and tools for participative learning,using multimedia educational systems over Internet broadband and mobile devices. In this work, aWeb-based learning system was developed, which enables to store, transmit, search and share the contents of courses captured in video and its extension to support Tablet PCs. The Web system, developed as part of the VideoStore project, explores video interfaces and video annotations, which encourage the participative work. The usage of Tablet PCs, through the mEmLearn project, has the aim to encourage the participative work, allowing the students to augment the course materials and to share them with other students or instructors

Fine-grained Recording and Streaming Lectures

There are several lecture recording systems (commercial and non-commercial) based on VOD technology. These systems record only slides or WWW images statically, so the fine-grained events are missed. For example, scrolling and/or mouse focusing are not recorded. We developed a fine-grained lecture recording system which records mouse movement during lectures, and also records lectures using Microsoft PowerPoint® and WWW.ArticleProceedings of the International Conference on Computers in Education (ICCE’04). 1957-1962 (2004)conference pape

Automatic Lecture Recording

Lecture recording has become a very common tool to provide students with additional media for their examination preparations. While its effort has to stay reasonable, only a very basic way of recording is done in many cases. Therefore, watching the resulting videos can get very boring completely independent of how interesting the original topic or session was. This thesis proposes a new approach to lecture recordings by letting distributed computers emulate the work of a human camera team, which is the natural way of creating attractive recordings. This thesis is structured in six chapters, starting with the examination of the current situation, and taking its constraints into account. The first chapter concludes with a reflection on related work. Chapter two is about the design of our prototype system. It is deduced from a human camera team in the real world which gets transferred into the virtual world. Finally, a detailed overview about all parts necessary for our prototype and their planned functionality is given. In chapter three, the implementation of all parts and tasks and the incidents occurring during implementation are described in detail. Chapter four describes the technical experiences made with the different parts during development, testing and evaluation with a view to functionality, performance, and an proposal towards future work. The evaluation of the whole system with students is presented and discussed in the fifth chapter. Chapter six concludes this thesis by summing up the facts and gives an outlook on future work

Application sharing in teaching context with wireless networks

The success of teaching is depending on a couple of factors: on how far students are involved into lectures, on the material, its completeness and on co-learning of students. Involvement of students into lectures means, being able to follow the thoughts of the teacher, ask questions and make comments. The material must be presented in a suitable form and essential parts of it have to be available during the whole learning process, for preparing participation in lectures and exercises as well as for exams. For more effective learning and training of social abilities, working in groups of co-learners has to be encouraged. Mobile and ubiquitous computing offer new possibilities to achieve these goals by increasing the awareness in class and supporting an active participation of students. By promoting existing concepts and enabling new ways of application sharing, the project SASCIA (System architecture supporting cooperative and interactive applications) aims at developing a framework for multiple applications to support teaching in collocated, remote and hybrid scenarios. Its core is composed of components to capture and distribute context information about sessions, participants and those applications that are used during a lecture or encounter among students. A configurable floor control was designed to cope with a wide spectrum of applications and learning situations. For some cases, even a control for semantic consistency can be necessary. In combination with a suitable user and session management, a whiteboard for annotations and a recording facility to support latecomers as well as subsequent replay, these components are providing the required functionality. As a consequence, SASCIA offers remote control and viewing facilities to all participants during lectures and co-learning sessions

Multiple Media Correlation: Theory and Applications

This thesis introduces multiple media correlation, a new technology for the automatic alignment of multiple media objects such as text, audio, and video. This research began with the question: what can be learned when multiple multimedia components are analyzed simultaneously? Most ongoing research in computational multimedia has focused on queries, indexing, and retrieval within a single media type. Video is compressed and searched independently of audio, text is indexed without regard to temporal relationships it may have to other media data. Multiple media correlation provides a framework for locating and exploiting correlations between multiple, potentially heterogeneous, media streams. The goal is computed synchronization, the determination of temporal and spatial alignments that optimize a correlation function and indicate commonality and synchronization between media objects. The model also provides a basis for comparison of media in unrelated domains. There are many real-world applications for this technology, including speaker localization, musical score alignment, and degraded media realignment. Two applications, text-to-speech alignment and parallel text alignment, are described in detail with experimental validation. Text-to-speech alignment computes the alignment between a textual transcript and speech-based audio. The presented solutions are effective for a wide variety of content and are useful not only for retrieval of content, but in support of automatic captioning of movies and video. Parallel text alignment provides a tool for the comparison of alternative translations of the same document that is particularly useful to the classics scholar interested in comparing translation techniques or styles. The results presented in this thesis include (a) new media models more useful in analysis applications, (b) a theoretical model for multiple media correlation, (c) two practical application solutions that have wide-spread applicability, and (d) Xtrieve, a multimedia database retrieval system that demonstrates this new technology and demonstrates application of multiple media correlation to information retrieval. This thesis demonstrates that computed alignment of media objects is practical and can provide immediate solutions to many information retrieval and content presentation problems. It also introduces a new area for research in media data analysis

Consistency Algorithms and Protocols for Distributed Interactive Applications

The Internet has a major impact not only on how people retrieve information but also on how they communicate. Distributed interactive applications support the communication and collaboration of people through the sharing and manipulation of rich multimedia content via the Internet. Aside from shared text editors, meeting support systems, and distributed virtual environments, shared whiteboards are a prominent example of distributed interactive applications. They allow the presentation and joint editing of documents in video conferencing scenarios. The design of such a shared whiteboard application, the multimedia lecture board (mlb), is a main contribution of this thesis. Like many other distributed interactive applications, the mlb has a replicated architecture where each user runs an instance of the application. This has the distinct advantage that the application can be deployed in a lightweight fashion, without relying on a supporting server infrastructure. But at the same time, this peer-to-peer architecture raises a number of challenging problems: First, application data needs to be distributed among all instances. For this purpose, we present the network protocol RTP/I for the standardized communication of distributed interactive applications, and a novel application-level multicast protocol that realizes efficient group communication while taking application-level knowledge into account. Second, consistency control mechanisms are required to keep the replicated application data synchronized. We present the consistency control algorithms “local lag”, “Timewarp”, and “state request”, show how they can be combined, and discuss how to provide visual feedback so that the session members are able to handle conflicting actions. Finally, late-joining participants need to be initialized with the current application state before they are able to participate in a collaborative session. We propose a novel late-join algorithm, which is both flexible and scalable. All algorithms and protocols presented in this dissertation solve the aforementioned problems in a generic way. We demonstrate how they can be employed for the mlb as well as for other distributed interactive applications

Highly Interactive Web-Based Courseware

Zukünftige Lehr-/Lernprogramme sollen als vernetzte Systeme die Lernenden befähigen, Lerninhalte zu erforschen und zu konstruieren, sowie Verständnisschwierigkeiten und Gedanken in der Lehr-/Lerngemeinschaft zu kommunizieren. Lehrmaterial soll dabei in digitale Lernobjekte übergeführt, kollaborativ von Programmierern, Pädagogen und Designern entwickelt und in einer Datenbank archiviert werden, um von Lehrern und Lernenden eingesetzt, angepasst und weiterentwickelt zu werden. Den ersten Schritt in diese Richtung machte die Lerntechnologie, indem sie Wiederverwendbarkeit und Kompabilität für hypermediale Kurse spezifizierte. Ein größeres Maß an Interaktivität wird bisher allerdings noch nicht in Betracht gezogen. Jedes interaktive Lernobjekt wird als autonome Hypermedia-Einheit angesehen, aufwändig in der Erstellung, und weder mehrstufig verschränk- noch anpassbar, oder gar adäquat spezifizierbar. Dynamische Eigenschaften, Aussehen und Verhalten sind fest vorgegeben. Die vorgestellte Arbeit konzipiert und realisiert Lerntechnologie für hypermediale Kurse unter besonderer Berücksichtigung hochgradig interaktiver Lernobjekte. Innovativ ist dabei zunächst die mehrstufige, komponenten-basierte Technologie, die verschiedenste strukturelle Abstufungen von kompletten Lernobjekten und Werkzeugsätzen bis hin zu Basiskomponenten und Skripten, einzelnen Programmanweisungen, erlaubt. Zweitens erweitert die vorgeschlagene Methodik Kollaboration und individuelle Anpassung seitens der Teilnehmer eines hypermedialen Kurses auf die Software-Ebene. Komponenten werden zu verknüpfbaren Hypermedia-Objekten, die in der Kursdatenbank verwaltet und von allen Kursteilnehmern bewertet, mit Anmerkungen versehen und modifiziert werden. Neben einer detaillierten Beschreibung der Lerntechnologie und Entwurfsmuster für interaktive Lernobjekte sowie verwandte hypermediale Kurse wird der Begriff der Interaktivität verdeutlicht, indem eine kombinierte technologische und symbolische Definition von Interaktionsgraden vorgestellt und daraus ein visuelles Skriptschema abgeleitet wird, welches Funktionalität übertragbar macht. Weiterhin wird die Evolution von Hypermedia und Lehr-/Lernprogrammen besprochen, um wesentliche Techniken für interaktive, hypermediale Kurse auszuwählen. Die vorgeschlagene Architektur unterstützt mehrsprachige, alternative Inhalte, bietet konsistente Referenzen und ist leicht zu pflegen, und besitzt selbst für interaktive Inhalte Online-Assistenten. Der Einsatz hochgradiger Interaktivität in Lehr-/Lernprogrammen wird mit hypermedialen Kursen im Bereich der Computergraphik illustriert.The grand vision of educational software is that of a networked system enabling the learner to explore, discover, and construct subject matters and communicate problems and ideas with other community members. Educational material is transformed into reusable learning objects, created collaboratively by developers, educators, and designers, preserved in a digital library, and utilized, adapted, and evolved by educators and learners. Recent advances in learning technology specified reusability and interoperability in Web-based courseware. However, great interactivity is not yet considered. Each interactive learning object represents an autonomous hypermedia entity, laborious to create, impossible to interlink and to adapt in a graduated manner, and hard to specify. Dynamic attributes, the look and feel, and functionality are predefined. This work designs and realizes learning technology for Web-based courseware with special regard to highly interactive learning objects. The innovative aspect initially lies in the multi-level, component-based technology providing a graduated structuring. Components range from complex learning objects to toolkits to primitive components and scripts. Secondly, the proposed methodologies extend community support in Web-based courseware – collaboration and personalization – to the software layer. Components become linkable hypermedia objects and part of the courseware repository, rated, annotated, and modified by all community members. In addition to a detailed description of technology and design patterns for interactive learning objects and matching Web-based courseware, the thesis clarifies the denotation of interactivity in educational software formulating combined levels of technological and symbolical interactivity, and deduces a visual scripting metaphor for transporting functionality. Further, it reviews the evolution of hypermedia and educational software to extract substantial techniques for interactive Web-based courseware. The proposed framework supports multilingual, alternative content, provides link consistency and easy maintenance, and includes state-driven online wizards also for interactive content. The impact of great interactivity in educational software is illustrated with courseware in the Computer Graphics domain