An affect-based video retrieval system with open vocabulary querying

Content-based video retrieval systems (CBVR) are creating new search and browse capabilities using metadata describing significant features of the data. An often overlooked aspect of human interpretation of multimedia data is the affective dimension. Incorporating affective information into multimedia metadata can potentially enable search using this alternative interpretation of multimedia content. Recent work has described methods to automatically assign affective labels to multimedia data using various approaches. However, the subjective and imprecise nature of affective labels makes it difficult to bridge the semantic gap between system-detected labels and user expression of information requirements in multimedia retrieval. We present a novel affect-based video retrieval system incorporating an open-vocabulary query stage based on WordNet enabling search using an unrestricted query vocabulary. The system performs automatic annotation of video data with labels of well defined affective terms. In retrieval annotated documents are ranked using the standard Okapi retrieval model based on open-vocabulary text queries. We present experimental results examining the behaviour of the system for retrieval of a collection of automatically annotated feature films of different genres. Our results indicate that affective annotation can potentially provide useful augmentation to more traditional objective content description in multimedia retrieval

A Novel Approach to Multimedia Ontology Engineering for Automated Reasoning over Audiovisual LOD Datasets

Multimedia reasoning, which is suitable for, among others, multimedia content analysis and high-level video scene interpretation, relies on the formal and comprehensive conceptualization of the represented knowledge domain. However, most multimedia ontologies are not exhaustive in terms of role definitions, and do not incorporate complex role inclusions and role interdependencies. In fact, most multimedia ontologies do not have a role box at all, and implement only a basic subset of the available logical constructors. Consequently, their application in multimedia reasoning is limited. To address the above issues, VidOnt, the very first multimedia ontology with SROIQ(D) expressivity and a DL-safe ruleset has been introduced for next-generation multimedia reasoning. In contrast to the common practice, the formal grounding has been set in one of the most expressive description logics, and the ontology validated with industry-leading reasoners, namely HermiT and FaCT++. This paper also presents best practices for developing multimedia ontologies, based on my ontology engineering approach

Automated annotation of multimedia audio data with affective labels for information management

The emergence of digital multimedia systems is creating many new opportunities for rapid access to huge content archives. In order to fully exploit these information sources, the content must be annotated with significant features. An important aspect of human interpretation of multimedia data, which is often overlooked, is the affective dimension. Such information is a potentially useful component for content-based classification and retrieval. Much of the affective information of multimedia content is contained within the audio data stream. Emotional features can be defined in terms of arousal and valence levels. In this study low-level audio features are extracted to calculate arousal and valence levels of multimedia audio streams. These are then mapped onto a set of keywords with predetermined emotional interpretations. Experimental results illustrate the use of this system to assign affective annotation to multimedia data

Advanced content-based semantic scene analysis and information retrieval: the SCHEMA project

The aim of the SCHEMA Network of Excellence is to bring together a critical mass of universities, research centers, industrial partners and end users, in order to design a reference system for content-based semantic scene analysis, interpretation and understanding. Relevant research areas include: content-based multimedia analysis and automatic annotation of semantic multimedia content, combined textual and multimedia information retrieval, semantic -web, MPEG-7 and MPEG-21 standards, user interfaces and human factors. In this paper, recent advances in content-based analysis, indexing and retrieval of digital media within the SCHEMA Network are presented. These advances will be integrated in the SCHEMA module-based, expandable reference system

Data mining technology for the evaluation of learning content interaction

Interactivity is central for the success of learning. In e-learning and other educational multimedia environments, the evaluation of interaction and behaviour is particularly crucial. Data mining – a non-intrusive, objective analysis technology – shall be proposed as the central evaluation technology for the analysis of the usage of computer-based educational environments and in particular of the interaction with educational content. Basic mining techniques are reviewed and their application in a Web-based third-level course environment is illustrated. Analytic models capturing interaction aspects from the application domain (learning) and the software infrastructure (interactive multimedia) are required for the meaningful interpretation of mining results

A GIS model-based assessment of the environmental distribution of g-hexachlorocyclohexane in European soils and waters

The MAPPE GIS based multimedia model is used to produce a quantitative description of the behaviour of γ-hexachlorocyclohexane (γ-HCH) in Europe, with emphasis on continental surface waters. The model is found to reasonably reproduce γ-HCH distributions and variations along the years in atmosphere and soil; for continental surface waters, concentrations were reasonably well predicted for year 1995, when lindane was still used in agriculture, while for 2005, assuming severe restrictions in use, yields to substantial underestimation. Much better results were yielded when same mode of release as in 1995 was considered, supporting the conjecture that for γ-HCH, emission data rather that model structure and parameterization can be responsible for wrong estimation of concentrations. Future research should be directed to improve the quality of emission data. Joint interpretation of monitoring and modelling results, highlights that lindane emissions in Europe, despite the marked decreasing trend, persist beyond the provisions of existing legislation. An spatially-explicit multimedia modelling strategy was applied to describe the historical distribution of γ-HCH in European soils and surface waters

PENGEMBANGAN MULTIMEDIA INTERAKTIF BERBASIS REALISTIC MATEMATICS EDUCATION PADA MATERI PERSAMAAN DAN PERTIDAKSAMAAN NILAI MUTLAK

ABSTRAK Pengembangan multimedia interaktif berbasis Realistic Mathematics Education (RME) pada materi Persamaan dan Pertidaksamaan Nilai Mutlak Satu Variabel bertujuan menghasilkan produk yang layak dan praktis. Penelitian dan pengembangan ini menggunakan Four D Model atau 4-D yang terdiri dari Define, Design, Develop dan Disseminate. Instrumen dalam penelitian pengembangan menggunakan angket validasi ahli materi dan ahli media untuk mengetahui kriteria kelayakan, angket respon peserta didik dan angket wawancara untuk mengetahui kriteria kepraktisan. Analisis data terbagi menjadi 2 yaitu data kuantitatif berupa persentase dan kualitatatif berupa deskripsi. Persentase kelayakan materi sebesar 72% dan 76,8% dengan interpretasi kuat, kelayakan media sebesar 71,2% dan 76% dengan interpretasi kuat sedangkan kepraktisan sebesar 80,16% dengan interpretasi sangat kuat. Berdasarkan hasil penelitian dan pengembangan dapat disimpulkan bahwa produk berupa multimedia interaktif berbasis Realistic Mathematics Education (RME) pada materi Persamaan dan Pertidaksamaan Nilai Mutlak layak dan sangat praktis. Sehingga penggunaan multimedia interaktif dapat mempermudah peserta didik dalam memahami materi pembelajaran dan menunjang tercapainya tujuan pembelajaran. Kata kunci: multimedia interaktif; nilai mutlak; pengembangan; RME   ABSTRACT The development of Realistic Mathematics Education (RME) based interactive multimedia on the Equations and Inequality of Absolute One Variable Value aims to produce instructional products that are feasible and practical. This research and development uses the Four D Model or 4-D which consists of Define, Design, Develop, and Disseminate. The instruments used are material and instructional media validation questionnaires from experts to know the eligibility criteria. Besides, it uses a questionnaire of students' responses and interviews to determine practicality criteria. The data analysis is divided into two: quantitative data in the form of percentages, and qualitative in the form of descriptions. The percentage of material feasibility is 72% and 76,8% with strong interpretation. On the other hand, the feasibility of media is 71,2%, and 76% with strong interpretation. While the practicality is 80,16% with very strong interpretation. Based on the results of research and development, it can be concluded that the product in the form of Realistic Mathematics Education (RME) based interactive multimedia on the Equations and Inequality of Absolute One Variable Value is feasible and very practical. So that the use of interactive multimedia can make students easier to understand learning materials and support the achievement of learning objectives. Keywords: absolute value; development; interactive multimedia; RM

A Framework to Enable the Semantic Inferencing and Querying of Multimedia Content

Cultural institutions, broadcasting companies, academic, scientific and defence organisations are producing vast quantities of digital multimedia content. With this growth in audiovisual material comes the need for standardised representations encapsulating the rich semantic meaning required to enable the automatic filtering, machine processing, interpretation and assimilation of multimedia resources. Additionally generating high-level descriptions is difficult and manual creation is expensive although significant progress has been made in recent years on automatic segmentation and low-level feature recognition for multimedia. Within this paper we describe the application of semantic web technologies to enable the generation of high-level, domain-specific, semantic descriptions of multimedia content from low-level, automatically-extracted features. By applying the knowledge reasoning capabilities provided by ontologies and inferencing rules to large, multimedia data sets generated by scientific research communities, we hope to expedite solutions to the complex scientific problems they face