User quality of experience of mulsemedia applications

User Quality of Experience (QoE) is of fundamental importance in multimedia applications and has been extensively studied for decades. However, user QoE in the context of the emerging multiple-sensorial media (mulsemedia) services, which involve different media components than the traditional multimedia applications, have not been comprehensively studied. This article presents the results of subjective tests which have investigated user perception of mulsemedia content. In particular, the impact of intensity of certain mulsemedia components including haptic and airflow on user-perceived experience are studied. Results demonstrate that by making use of mulsemedia the overall user enjoyment levels increased by up to 77%

Mulsemedia: State of the art, perspectives, and challenges

Mulsemedia-multiple sensorial media-captures a wide variety of research efforts and applications. This article presents a historic perspective on mulsemedia work and reviews current developments in the area. These take place across the traditional multimedia spectrum-from virtual reality applications to computer games-as well as efforts in the arts, gastronomy, and therapy, to mention a few. We also describe standardization efforts, via the MPEG-V standard, and identify future developments and exciting challenges the community needs to overcome

Audio masking effect on inter-component skews in olfaction-enhanced multimedia presentations

Media-rich content plays a vital role in consumer applications today, as these applications try to find new and interesting ways to engage their users. Video, audio, and the more traditional forms of media content continue to dominate with respect to the use of media content to enhance the user experience. Tactile interactivity has also now become widely popular in modern computing applications, while our olfactory and gustatory senses continue to have a limited role. However, in recent times, there have been significant advancements regarding the use of olfactory media content (i.e., smell), and there are a variety of devices now available to enable its computer-controlled emission. This paper explores the impact of the audio stream on user perception of olfactory-enhanced video content in the presence of skews between the olfactory and video media. This research uses the results from two experimental studies of user-perceived quality of olfactory-enhanced multimedia, where audio was present and absent, respectively. Specifically, the paper shows that the user Quality of Experience (QoE) is generally higher in the absence of audio for nearly perfect synchronized olfactory-enhanced multimedia presentations (i.e., an olfactory media skew of between {−10,+10s}); however, for greater olfactory media skews (ranging between {−30s;−10s} and {+10s, +30s}) user QoE is higher when the audio stream is present. It can be concluded that the presence of the audio has the ability to mask larger synchronization skews between the other media components in olfaction-enhanced multimedia presentations

Do I smell coffee? The tale of a 360º Mulsemedia experience

One of the main challenges in current multimedia networking environments is to find solutions to help accommodate the next generation of mobile application classes with stringent Quality of Service (QoS) requirements whilst enabling Quality of Experience (QoE) provisioning for users. One such application class, featured in this paper, is 360º mulsemedia—multiple sensorial media—which enriches 360º video by adding sensory effects that stimulate human senses beyond those of sight and hearing, such as the tactile and olfactory ones. In this paper, we present a conceptual framework for 360º mulsemedia delivery and a 360º mulsemedia-based prototype that enables users to experience 360º mulsemedia content. User evaluations revealed that higher video resolutions do not necessarily lead to the highest QoE levels in our experimental setup. Therefore, bandwidth savings can be leveraged with no detrimental impact on QoE

A tutorial for olfaction-based multisensorial media application design and evaluation

© ACM, 2017. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in PUBLICATION, {VOL50, ISS5, September 2017} https://doi.org/10.1145/310824

A DASH-based mulsemedia adaptive delivery solution

European Union’s Horizon 2020 Research and Innovation programme via Grant Agreement no. 688503 for NEWTON (http://newtonproject.eu); China Postdoctoral Science Foundatio

Is Multimedia Multisensorial? - A Review of Mulsemedia Systems

© 2018 Copyright held by the owner/author(s). Mulsemedia - multiple sensorial media - makes possible the inclusion of layered sensory stimulation and interaction through multiple sensory channels. e recent upsurge in technology and wearables provides mulsemedia researchers a vehicle for potentially boundless choice. However, in order to build systems that integrate various senses, there are still some issues that need to be addressed. is review deals with mulsemedia topics remained insu ciently explored by previous work, with a focus on multi-multi (multiple media - multiple senses) perspective, where multiple types of media engage multiple senses. Moreover, it addresses the evolution of previously identi ed challenges in this area and formulates new exploration directions.This article was funded by the European Union’s Horizon 2020 Research and Innovation program under Grant Agreement no. 688503

Guidelines for evaluating wearables’ quality of experience in a mulsemedia context

Copyright © The Author(s) 2022. Quality of Experience (QoE) is inextricably linked to the user experience of multimedia computing and, although QoE has been explored in relation to other types of multimedia devices, thus far its applicability to wearables has remained largely ignored. Given the proliferation of wearable devices and their growing use to augment and complement the multimedia user experience, the need for a set of QoE guidelines becomes imperative. This study meets that need and puts forward a set of guidelines tailored exclusively towards wearables’ QoE. Accordingly, an extensive experimental investigation has been undertaken to see how wearables impact users’ QoE in multiple sensorial media (mulsemedia) context. Based on the exploratory study, the findings have shown that the haptic vest (KOR-FX) enhanced user QoE to a certain extent. In terms of adoption, participants reported they would generally incorporate the heart rate (HR) monitor wristband (Mio Go) into their daily lives as opposed to the haptic vest. Other findings revealed that human factors play a part in user’s attitudes towards wearables and predominantly age was the major influencing factor. Moreover, the participants’ HR varied throughout the experiments, suggesting an enhanced level of engagement whilst viewing the multimedia video clips. Furthermore, the results suggest that there is a potential future for wearables, if the QoE is a positive one and if the design of such devices are appealing as well as unobtrusive

Mulsemedia in Telecommunication and Networking Education: A Novel Teaching Approach that Improves the Learning Process

The advent and increased use of new technologies, such as innovative mulsemedia and multi-modal content distribution mechanisms, have brought new challenges and diverse opportunities for technology enhanced learning (TEL). NEWTON is a Horizon 2020 European project that revolutionizes the educational process through innovative TEL methodologies and tools, integrated in a pan-European STEM-related learning network platform. This article focuses on one of these novel TEL methodologies (i.e., mulsemedia) and presents how NEWTON enables mulsemedia- enhanced teaching and learning of STEM subjects, with a particular focus on telecommunication and networking related modules. The article also discusses the very promising results of NEWTON case studies carried out with engineering students across two different universities in Spain and Ireland, respectively. The case studies focused on analyzing the impact on the learning process of the mulsemedia-enhanced teaching in the context of telecommunication and networking modules. The main conclusion of the article is that mulsemedia-enhanced education significantly increases students' learning experience and improves their knowledge gain

360° Mulsemedia: A Way to Improve Subjective QoE in 360° Videos

Previous research has shown that adding multisensory media-mulsemedia-to traditional audiovisual content has a positive effect on user Quality of Experience (QoE). However, the QoE impact of employing mulsemedia in 360° videos has remained unexplored. Accordingly, in this paper, a QoE study for watching a 360° video-with and without multisensory effects-in a full free-viewpoint VR setting is presented. The parametric space we considered to influence the QoE consists of the encoding quality and the motion level of the transmitted media. To achieve our research aim, we propose a wearable VR system that provides multisensory enhancement of 360° videos. Then, we utilise its capabilities to systematically evaluate the effects of multisensory stimulation on perceived quality degradation for videos with different motion levels and encoding qualities. Our results make a strong case for the inclusion of multisensory effects in 360° videos, as they reveal that both user-perceived quality, as well as enjoyment, are significantly higher when mulsemedia (as opposed to traditional multimedia) is employed in this context. Moreover, these observations hold true independent of the underlying 360° video encoding quality-thus QoE can be significantly enhanced with a minimal impact on networking resources