Content vs. context for multimedia semantics: the case of SenseCam image structuring

Much of the current work on determining multimedia semantics from multimedia artifacts is based around using either context, or using content. When leveraged thoroughly these can independently provide content description which is used in building content-based applications. However, there are few cases where multimedia semantics are determined based on an integrated analysis of content and context. In this keynote talk we present one such example system in which we use an integrated combination of the two to automatically structure large collections of images taken by a SenseCam, a device from Microsoft Research which passively records a person’s daily activities. This paper describes the post-processing we perform on SenseCam images in order to present a structured, organised visualisation of the highlights of each of the wearer’s days

A systemic approach to automatic metadata extraction from multimedia content

There is a need for automatic processing and extracting of meaningful metadata from multimedia information, especially in the audiovisual industry. This higher level information is used in a variety of practices, such as enriching multimedia content with external links, clickable objects and useful related information in general. This paper presents a system for efficient multimedia content analysis and automatic annotation within a multimedia processing and publishing framework. This system is comprised of three modules: the first provides detection of faces and recognition of known persons; the second provides generic object detection, based on a deep convolutional neural network topology; the third provides automated location estimation and landmark recognition based on state-of-the-art technologies. The results are exported in meaningful metadata that can be utilized in various ways. The system has been successfully tested in the framework of the EC Horizon 2020 Mecanex project, targeting advertising and production markets

ON - LINE BIOMECHANICAL ANALYSIS

INTRODUCTION : Biomechanical analysis is that branch of science which involves a description of the position and movement of the body and its parts and the forces acting on them. This analysis may include the energy released or consumed by these parts. On-line analysis means the technique of collecting electronically measured date from an experiment and fed directly into a computer for immediate date processing and result sorting. The idea of an on-line computer system for recording biomechanical date was discussed for 25 years by the Biomechanical Laboratory or Pennsylvania State University [1]. This work and others showed three advantages for the on-line system; fast and accurate measurement and collection of biomechanical data, drastic reduction of processing time for a large number of parameters and finally the possible immediate feedback from the on-line system operator to the subject or the player under analysis. Recent advances in multimedia and the simplicity of handling multimedia packages permit the trainer and coach to efficiently operate on-line biomechanical analysis systems

Dynamic Generation of Intelligent Multimedia Presentations Through Semantic Inferencing

This paper first proposes a high-level architecture for semi-automatically generating multimedia presentations by combining semantic inferencing with multimedia presentation generation tools. It then describes a system, based on this architecture, which was developed as a service to run over OAI archives - but is applicable to any repositories containing mixed-media resources described using Dublin Core. By applying an iterative sequence of searches across the Dublin Core metadata, published by the OAI data providers, semantic relationships can be inferred between the mixed-media objects which are retrieved. Using predefined mapping rules, these semantic relationships are then mapped to spatial and temporal relationships between the objects. The spatial and temporal relationships are expressed within SMIL files which can be replayed as multimedia presentations. Our underlying hypothesis is that by using automated computer processing of metadata to organize and combine semantically-related objects within multimedia presentations, the system may be able to generate new knowledge by exposing previously unrecognized connections. In addition, the use of multilayered information-rich multimedia to present the results, enables faster and easier information browsing, analysis, interpretation and deduction by the end-user

Machine learning paradigms for modeling spatial and temporal information in multimedia data mining

Multimedia data mining and knowledge discovery is a fast emerging interdisciplinary applied research area. There is tremendous potential for effective use of multimedia data mining (MDM) through intelligent analysis. Diverse application areas are increasingly relying on multimedia under-standing systems. Advances in multimedia understanding are related directly to advances in signal processing, computer vision, machine learning, pattern recognition, multimedia databases, and smart sensors. The main mission of this special issue is to identify state-of-the-art machine learning paradigms that are particularly powerful and effective for modeling and combining temporal and spatial media cues such as audio, visual, and face information and for accomplishing tasks of multimedia data mining and knowledge discovery. These models should be able to bridge the gap between low-level audiovisual features which require signal processing and high-level semantics. A number of papers have been submitted to the special issue in the areas of imaging, artificial intelligence; and pattern recognition and five contributions have been selected covering state-of-the-art algorithms and advanced related topics. The first contribution by D. Xiang et al. “Evaluation of data quality and drought monitoring capability of FY-3A MERSI data” describes some basic parameters and major technical indicators of the FY-3A, and evaluates data quality and drought monitoring capability of the Medium-Resolution Imager (MERSI) onboard the FY-3A. The second contribution by A. Belatreche et al. “Computing with biologically inspired neural oscillators: application to color image segmentation” investigates the computing capabilities and potential applications of neural oscillators, a biologically inspired neural model, to gray scale and color image segmentation, an important task in image understanding and object recognition. The major contribution of this paper is the ability to use neural oscillators as a learning scheme for solving real world engineering problems. The third paper by A. Dargazany et al. entitled “Multibandwidth Kernel-based object tracking” explores new methods for object tracking using the mean shift (MS). A bandwidth-handling MS technique is deployed in which the tracker reach the global mode of the density function not requiring a specific staring point. It has been proven via experiments that the Gradual Multibandwidth Mean Shift tracking algorithm can converge faster than the conventional kernel-based object tracking (known as the mean shift). The fourth contribution by S. Alzu’bi et al. entitled “3D medical volume segmentation using hybrid multi-resolution statistical approaches” studies new 3D volume segmentation using multiresolution statistical approaches based on discrete wavelet transform and hidden Markov models. This system commonly reduced the percentage error achieved using the traditional 2D segmentation techniques by several percent. Furthermore, a contribution by G. Cabanes et al. entitled “Unsupervised topographic learning for spatiotemporal data mining” proposes a new unsupervised algorithm, suitable for the analysis of noisy spatiotemporal Radio Frequency Identification (RFID) data. The new unsupervised algorithm depicted in this article is an efficient data mining tool for behavioral studies based on RFID technology. It has the ability to discover and compare stable patterns in a RFID signal, and is appropriate for continuous learning. Finally, we would like to thank all those who helped to make this special issue possible, especially the authors and the reviewers of the articles. Our thanks go to the Hindawi staff and personnel, the journal Manager in bringing about the issue and giving us the opportunity to edit this special issue

SISTEM REKOMENDASI KEAHLIAN MULTIMEDIA BERDASARKAN MINAT BAKAT SISWA SMPN 42 BANDUNG MENGGUNAKAN METODE AHP

Penelitian ini bertujuan untuk memberikan alternatif pilihan dan rekomendasi kepada siswa di SMPN 42 Bandung dalam memilih jurusan kelanjutan sekolah di bidang keahlian Multimedia. Luaran yang dihasilkan berupa sistem informasi Penjurusan Multimedia berbasis web. Penelitian ini dilaksanakan menggunakan metode System Development Life Cycle (SDLC) yang terdiri dari enam tahap yaitu, Analisis Sistem, Desain Sistem, Development, Pengujian Sistem, Implementasi dan Maintenance. Tahap analisis sistem mencakup analisis kebutuhan sistem dan analisis kebutuhan perancangan. Setelah tahap analisis, dilanjutkan dengan tahap desain sistem yang mencakup use case diagram, data flow diagram, dan desin antarmuka sistem. Selanjutnya tahap development mencakup tahap pengkodean menggunakan bahasa pemrograman PHP, framework CodeIgniter, dan database MySQL. Dilanjutkan dengan tahap pengujian sistem mencakup pengumpulan dan pengolahan data, dan pengujian validitas menggunakan black-box¬ testing meliputi perhitungan fungsionalitas sistem dan database, kinerja sistem, desain antarmuka sistem, dan pengaruh implementasi sistem. Tahap terakhir yaitu implementasi dan maintenance mencakup pengumpulan dan pengolahan data, implementasi penjurusan multimedia, sistem perhitungan AHP, dan hasil penjurusan. ----- This study aims to provide alternative choices and recommendations to students at SMPN 42 Bandung in choosing a school continuation major in the field of Multimedia expertise. The resulting output is a web-based Multimedia Majoring information system. This research was carried out using the System Development Life Cycle (SDLC) method which consisted of six stages, namely System Analysis, System Design, Development, System Testing, Implementation and Maintenance. The system analysis phase includes system requirements analysis and design requirements analysis. After the analysis phase, it is continued with the system design stage which includes use case diagrams, data flow diagrams, and system interface design. Furthermore, the development stage includes the coding stage using the PHP programming language, the CodeIgniter framework, and the MySQL database. Followed by the system testing phase which includes data collection and processing, and validity testing using black-box testing including calculating system and database functionality, system performance, system interface design, and the effect of system implementation. The last stage is implementation and maintenance which includes data collection and processing, implementation of multimedia majors, AHP calculation system, and major results

Multimedia on the Mountaintop: Using public snow images to improve water systems operation

This paper merges multimedia and environmental research to verify the utility of public web images for improving water management in periods of water scarcity, an increasingly critical event due to climate change. A multimedia processing pipeline fetches mountain images from multiple sources and extracts virtual snow indexes correlated to the amount of water accumulated in the snow pack. Such indexes are used to predict water availability and design the operating policy of Lake Como, Italy. The performance of this informed policy is contrasted, via simulation, with the current operation, which depends only on lake water level and day of the year, and with a policy that exploits official Snow Water Equivalent (SWE) estimated from ground stations data and satellite imagery. Virtual snow indexes allow improving the system performance by 11.6% w.r.t. The baseline operation, and yield further improvement when coupled with official SWE information, showing that the two data sources are complementary. The proposed approach exemplifies the opportunities and challenges of applying multimedia content analysis methods to complex environmental problems

AVEC 2016 – Depression, mood, and emotion recognition workshop and challenge

The Audio/Visual Emotion Challenge and Workshop (AVEC 2016) "Depression, Mood and Emotion" will be the sixth competition event aimed at comparison of multimedia processing and machine learning methods for automatic audio, visual and physiological depression and emotion analysis, with all participants competing under strictly the same conditions. The goal of the Challenge is to provide a common benchmark test set for multi-modal information processing and to bring together the depression and emotion recognition communities, as well as the audio, video and physiological processing communities, to compare the relative merits of the various approaches to depression and emotion recognition under well-defined and strictly comparable conditions and establish to what extent fusion of the approaches is possible and beneficial. This paper presents the challenge guidelines, the common data used, and the performance of the baseline system on the two tasks

Web-Based Benchmark for Keystroke Dynamics Biometric Systems: A Statistical Analysis

Most keystroke dynamics studies have been evaluated using a specific kind of dataset in which users type an imposed login and password. Moreover, these studies are optimistics since most of them use different acquisition protocols, private datasets, controlled environment, etc. In order to enhance the accuracy of keystroke dynamics' performance, the main contribution of this paper is twofold. First, we provide a new kind of dataset in which users have typed both an imposed and a chosen pairs of logins and passwords. In addition, the keystroke dynamics samples are collected in a web-based uncontrolled environment (OS, keyboards, browser, etc.). Such kind of dataset is important since it provides us more realistic results of keystroke dynamics' performance in comparison to the literature (controlled environment, etc.). Second, we present a statistical analysis of well known assertions such as the relationship between performance and password size, impact of fusion schemes on system overall performance, and others such as the relationship between performance and entropy. We put into obviousness in this paper some new results on keystroke dynamics in realistic conditions.Comment: The Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIHMSP 2012), Piraeus : Greece (2012

Astronomical Image Compression Techniques Based on ACC and KLT Coder

This paper deals with a compression of image data in applications in astronomy. Astronomical images have typical specific properties — high grayscale bit depth, size, noise occurrence and special processing algorithms. They belong to the class of scientific images. Their processing and compression is quite different from the classical approach of multimedia image processing. The database of images from BOOTES (Burst Observer and Optical Transient Exploring System) has been chosen as a source of the testing signal. BOOTES is a Czech-Spanish robotic telescope for observing AGN (active galactic nuclei) and the optical transient of GRB (gamma ray bursts) searching. This paper discusses an approach based on an analysis of statistical properties of image data. A comparison of two irrelevancy reduction methods is presented from a scientific (astrometric and photometric) point of view. The first method is based on a statistical approach, using the Karhunen-Loeve transform (KLT) with uniform quantization in the spectral domain. The second technique is derived from wavelet decomposition with adaptive selection of used prediction coefficients. Finally, the comparison of three redundancy reduction methods is discussed. Multimedia format JPEG2000 and HCOMPRESS, designed especially for astronomical images, are compared with the new Astronomical Context Coder (ACC) coder based on adaptive median regression