14,055 research outputs found
Perceived synchronization of mulsemedia services
Multimedia synchronization involves a temporal relationship between audio and visual media components. The presentation of "in-sync" data streams is essential to achieve a natural impression, as "out-of-sync" effects are often associated with user quality of experience (QoE) decrease. Recently, multi-sensory media (mulsemedia) has been demonstrated to provide a highly immersive experience for its users. Unlike traditional multimedia, mulsemedia consists of other media types (i.e., haptic, olfaction, taste, etc.) in addition to audio and visual content. Therefore, the goal of achieving high quality mulsemedia transmission is to present no or little synchronization errors between the multiple media components. In order to achieve this ideal synchronization, there is a need for comprehensive knowledge of the synchronization requirements at the user interface. This paper presents the results of a subjective study carried out to explore the temporal boundaries within which haptic and air-flow media objects can be successfully synchronized with video media. Results show that skews between sensorial media and multimedia might still give the effect that the mulsemedia sequence is "in-sync" and provide certain constraints under which synchronization errors might be tolerated. The outcomes of the paper are used to provide recommendations for mulsemedia service providers in order for their services to be associated with acceptable user experience levels, e.g. haptic media could be presented with a delay of up to 1 s behind video content, while air-flow media could be released either 5 s ahead of or 3 s behind video content
Perceived synchronization of olfactory multimedia
This is the post-print version of this Article. The official published version can be accessed from the link below - Copyright @ 2010 IEEEThe concept of synchronization is of fundamental importance in multimedia systems and applications. The focus of this this paper is on olfaction-enhanced multimedia, which concerns itself with associating computer-generated smell with other media. However, the lingering nature of smell, as opposed to the transitory nature of other media objects that multimedia applications are accustomed to, such as video and audio, means that specific attention needs to be given when synchronizing other media content with olfactory data. Consequently, this paper presents the results of an experimental study carried out to explore and investigate the temporal boundaries within which olfactory-data output in an olfaction-enhanced multimedia application can be successfully synchronized with other media objects from an end-user perspective. Results show the presence of two main synchronization regions, and that olfaction ahead of audiovisual content is more tolerable than olfaction behind content
Beyond multimedia adaptation: Quality of experience-aware multi-sensorial media delivery
Multiple sensorial media (mulsemedia) combines multiple media elements which engage three or more of human senses, and as most other media content, requires support for delivery over the existing networks. This paper proposes an adaptive mulsemedia framework (ADAMS) for delivering scalable video and sensorial data to users. Unlike existing two-dimensional joint source-channel adaptation solutions for video streaming, the ADAMS framework includes three joint adaptation dimensions: video source, sensorial source, and network optimization. Using an MPEG-7 description scheme, ADAMS recommends the integration of multiple sensorial effects (i.e., haptic, olfaction, air motion, etc.) as metadata into multimedia streams. ADAMS design includes both coarse- and fine-grained adaptation modules on the server side: mulsemedia flow adaptation and packet priority scheduling. Feedback from subjective quality evaluation and network conditions is used to develop the two modules. Subjective evaluation investigated users' enjoyment levels when exposed to mulsemedia and multimedia sequences, respectively and to study users' preference levels of some sensorial effects in the context of mulsemedia sequences with video components at different quality levels. Results of the subjective study inform guidelines for an adaptive strategy that selects the optimal combination for video segments and sensorial data for a given bandwidth constraint and user requirement. User perceptual tests show how ADAMS outperforms existing multimedia delivery solutions in terms of both user perceived quality and user enjoyment during adaptive streaming of various mulsemedia content. In doing so, it highlights the case for tailored, adaptive mulsemedia delivery over traditional multimedia adaptive transport mechanisms
Recommended from our members
Synchronization of Olfaction-enhanced multimedia
This paper presents the results of an experimental study carried out to explore, from an end user perspective, the temporal boundaries within which olfactory data can be used to enhance multimedia applications. Results show the presence of two main synchronization regions, and that olfaction ahead of audiovisual content is more tolerable than olfaction behind content
A framework for realistic 3D tele-immersion
Meeting, socializing and conversing online with a group of people using teleconferencing systems is still quite differ- ent from the experience of meeting face to face. We are abruptly aware that we are online and that the people we are engaging with are not in close proximity. Analogous to how talking on the telephone does not replicate the experi- ence of talking in person. Several causes for these differences have been identified and we propose inspiring and innova- tive solutions to these hurdles in attempt to provide a more realistic, believable and engaging online conversational expe- rience. We present the distributed and scalable framework REVERIE that provides a balanced mix of these solutions. Applications build on top of the REVERIE framework will be able to provide interactive, immersive, photo-realistic ex- periences to a multitude of users that for them will feel much more similar to having face to face meetings than the expe- rience offered by conventional teleconferencing systems
Automatic Synchronization of Multi-User Photo Galleries
In this paper we address the issue of photo galleries synchronization, where
pictures related to the same event are collected by different users. Existing
solutions to address the problem are usually based on unrealistic assumptions,
like time consistency across photo galleries, and often heavily rely on
heuristics, limiting therefore the applicability to real-world scenarios. We
propose a solution that achieves better generalization performance for the
synchronization task compared to the available literature. The method is
characterized by three stages: at first, deep convolutional neural network
features are used to assess the visual similarity among the photos; then, pairs
of similar photos are detected across different galleries and used to construct
a graph; eventually, a probabilistic graphical model is used to estimate the
temporal offset of each pair of galleries, by traversing the minimum spanning
tree extracted from this graph. The experimental evaluation is conducted on
four publicly available datasets covering different types of events,
demonstrating the strength of our proposed method. A thorough discussion of the
obtained results is provided for a critical assessment of the quality in
synchronization.Comment: ACCEPTED to IEEE Transactions on Multimedi
User quality of experience of mulsemedia applications
User Quality of Experience (QoE) is of fundamental importance in multimedia applications and has been extensively studied for decades. However, user QoE in the context of the emerging multiple-sensorial media (mulsemedia) services, which involve different media components than the traditional multimedia applications, have not been comprehensively studied. This article presents the results of subjective tests which have investigated user perception of mulsemedia content. In particular, the impact of intensity of certain mulsemedia components including haptic and airflow on user-perceived experience are studied. Results demonstrate that by making use of mulsemedia the overall user enjoyment levels increased by up to 77%
Recommended from our members
A dynamic petri net model for iterative and interactive distributed multimedia presentation
Object Composition Petri Nets (OCPN), Priority Petri Nets (P-Net), Dynamic OCPN (DOCPN) and Enhanced P-Nets (EP-Net) have extended the original Petri Net to achieve the modeling of media synchronization and asynchronous user interactions during multimedia playback. Dynamic Petri Net (DPN) has been conceptualized to tackle existing problems in these two areas of modeling distributed multimedia systems. DPN features dynamic modeling elements which allows iteration and hence is able to reduce graph sizes of synchronous playback models while allowing greater details to be shown. DPN also introduces asynchronous event handling techniques that are powerful and effective. DPN was used in the design and modeling of a multimedia orchestration tool which is a typical representation of an application that works in a distributed multimedia system
Recommended from our members
Mobile Audiovisual Terminal: System Design and Subjective Testing in DECT and UMTS networks
It is anticipated that there will shortly be a requirement
for multimedia terminals that operate via mobile
communications systems. This paper presents a functional specification
for such a terminal operating at 32 kb/s in a digital
European cordless telecommunications (DECT) and universal
mobile telecommunications system (UMTS) radio network. A terminal
has been built, based on a PC with digital signal processor
(DSP) boards for audio and video coding and decoding. Speech
coding is by a phonetically driven code-excited linear prediction
(CELP) speech coder and video coding by a block-oriented hybrid
discrete cosine transform (DCT) coder. Separate channel coding
is provided for the audio and video data. The paper describes the
techniques used for audio and video coding, channel coding, and
synchronization. Methods of subjective testing in a DECT network
and in a UMTS network are also described. These consisted of
subjective tests of first impressions of the mobile audio–visual
terminal (MAVT) quality, interactive tests, and the completion
of an exit questionnaire. The test results showed that the quality
of the audio was sufficiently good for comprehension and the
video was sufficiently good for following and repeating simple
mechanical tasks. However, the quality of the MAVT was not
good enough for general use where high-quality audio and video
was needed, especially when transmission was in a noisy radio
environment
Operating-system support for distributed multimedia
Multimedia applications place new demands upon processors, networks and operating systems. While some network designers, through ATM for example, have considered revolutionary approaches to supporting multimedia, the same cannot be said for operating systems designers. Most work is evolutionary in nature, attempting to identify additional features that can be added to existing systems to support multimedia. Here we describe the Pegasus project's attempt to build an integrated hardware and operating system environment from\ud
the ground up specifically targeted towards multimedia
- …