Semantics of Multimedia in MPEG-7

In this paper, we present the tools standardized by MPEG-7 for describing the semantics of multimedia. In particular, we focus on the abstraction model, entities, attributes and relations of MPEG-7 semantic descriptions. MPEG-7 tools can describe the semantics of specific instances of multimedia such as one image or one video segment but can also generalize these descriptions either to multiple instances of multimedia or to a set of semantic descriptions. The key components of MPEG-7 semantic descriptions are semantic entities such as objects and events, attributes of these entities such as labels and properties, and, finally, relations of these entities such as an object being the patient of an event. The descriptive power and usability of these tools has been demonstrated in numerous experiments and applications, these make them key candidates to enable intelligent applications that deal with multimedia at human levels

VAMP: semantic validation for MPEG-7 profile descriptions

MPEG-7 can be used to create complex and comprehensive metadata descriptions of multimedia content. Since MPEG-7 is defined in terms of an XML schema, the semantics of its elements has no formal grounding. In addition, certain features can be described in multiple ways. MPEG-7 profiles are subsets of the standard that apply to specific application areas and that aim to reduce this syntactic variability, but they still lack formal semantics. We propose an approach for expressing the semantics explicitly

Are we Ready to Embrace the Semantic Web?

The aim of the semantic web is to describe resources available on the web using metadata elements that can be processed or interpreted by machines. MPEG-7 is the result of a standardisation effort to annotate multimedia documents, and it offers a rich suite of metadata descriptors for describing these documents at various levels of abstraction from low-level features to high-level semantics. Owing to the proliferation of multimedia content in the internet, there is now a lot of interest in the semantic web community in multimedia metadata standards in general, and MPEG-7 in particular. Despite the fact that the semantic web initiatives could benefit a lot from MPEG-7 for the annotation of multimedia documents, recent studies have underlined the limitations of MPEG-7 in describing the semantics of highly structured domains like sports or medicine. This has led to an upsurge of interest in adopting an integrated approach to the design of multimedia ontologies. In our work, we describe a systematic approach to the design of multimedia ontologies in which we use MPEG-7 to model only the structural and the low-level aspects of multimedia documents. High-level semantics are described using domain-specific vocabularies. A retrieval engine based on this framework will then be able to process high-level text-based semantic queries. Whilst a lot of research has been done in the design of multimedia ontologies, a plaguing issue is the automatic annotation of multimedia content at a semantic level. It is possible to derive low-level descriptors using state-of-the-art techniques in multimedia content analysis, but the same does not hold true when it comes to analysing multimedia content at a high level of abstraction. We discuss various approaches that have been recently proposed to accomplish this task. An interesting line of discussion is the automatic population and enrichment of multimedia ontologies that offers a lot of challenges and stresses the need for efficient approaches for the semantic analysis of multimedia documents

Enriching MPEG-7 user models with content metadata

MPEG-7 is an XML-based standard that provides tools for creating rich and structured multimedia content metadata. However, only an extremely limited range of preferences can be specified for user models and multimedia content metadata created by other parts of the standard cannot be fully exploited. This results in a very incomplete mapping of user models to content models. We present an approach to address the problem by representing user models by means of existing MPEG-7 content description tools

COSMOS-7: Video-oriented MPEG-7 scheme for modelling and filtering of semantic content

MPEG-7 prescribes a format for semantic content models for multimedia to ensure interoperability across a multitude of platforms and application domains. However, the standard leaves it open as to how the models should be used and how their content should be filtered. Filtering is a technique used to retrieve only content relevant to user requirements, thereby reducing the necessary content-sifting effort of the user. This paper proposes an MPEG-7 scheme that can be deployed for semantic content modelling and filtering of digital video. The proposed scheme, COSMOS-7, produces rich and multi-faceted semantic content models and supports a content-based filtering approach that only analyses content relating directly to the preferred content requirements of the user

An MPEG-7 scheme for semantic content modelling and filtering of digital video

Abstract Part 5 of the MPEG-7 standard specifies Multimedia Description Schemes (MDS); that is, the format multimedia content models should conform to in order to ensure interoperability across multiple platforms and applications. However, the standard does not specify how the content or the associated model may be filtered. This paper proposes an MPEG-7 scheme which can be deployed for digital video content modelling and filtering. The proposed scheme, COSMOS-7, produces rich and multi-faceted semantic content models and supports a content-based filtering approach that only analyses content relating directly to the preferred content requirements of the user. We present details of the scheme, front-end systems used for content modelling and filtering and experiences with a number of users

Video semantic content analysis framework based on ontology combined MPEG-7

The rapid increase in the available amount of video data is creating a growing demand for efficient methods for understanding and managing it at the semantic level. New multimedia standard, MPEG-7, provides the rich functionalities to enable the generation of audiovisual descriptions and is expressed solely in XML Schema which provides little support for expressing semantic knowledge. In this paper, a video semantic content analysis framework based on ontology combined MPEG-7 is presented. Domain ontology is used to define high level semantic concepts and their relations in the context of the examined domain. MPEG-7 metadata terms of audiovisual descriptions and video content analysis algorithms are expressed in this ontology to enrich video semantic analysis. OWL is used for the ontology description. Rules in Description Logic are defined to describe how low-level features and algorithms for video analysis should be applied according to different perception content. Temporal Description Logic is used to describe the semantic events, and a reasoning algorithm is proposed for events detection. The proposed framework is demonstrated in sports video domain and shows promising results