220 research outputs found
The influence of human factors on 360ā mulsemedia QoE
Quality of Experience (QoE) is indelibly linked to the human side of the multimedia experience. Surprisingly, however, there is a paucity of research which explores the impact that human factors has in determining QoE. Whilst this is true of multimedia, it is even more starkly so as far as mulsemedia - applications that involve media engaging three or more of human senses - is concerned. Hence, in the study reported in this paper, we focus on an exciting subset of mulsemedia applications - 360ā mulsemedia - particularly important given that the upcoming 5G technology is foreseen to be a key enabler for the proliferation of immersive Virtual Reality (VR) applications. Accordingly, we study the impact that human factors such as gender, age, prior computing experience, and smell sensitivity have on 360ā mulsemedia QoE. Results showed insight into the potential of 360ā mulsemedia to inspire and to enrich experiences for Generation Z - a generation empowered by rapidly advancing technology. Patterns of prior media usage and smell sensitivity play also an important role in influencing the QoE evaluation - users who have a preference for dynamic videos enjoy and find realistic the 360ā mulsemedia experiences
A CNN-based Framework for Enhancing 360Ā° VR Experiences with Multisensorial Effects
Improving user experience during the delivery of
immersive content is crucial for its success for both the content
creators and audience. Creators can express themselves better
with multisensory stimulation, while the audience can experience a higher level of involvement. The rapid development of
mulsemedia devices provides better access for stimuli such as
olfaction and haptics. Nevertheless, due to the required manual
annotation process of adding mulsemedia effects, the amount
of content available with sensorial effects is still limited. This
work introduces an innovative mulsemedia-enhancement solution
capable of automatically generating olfactory and haptic content
based on 360Ā° video content, with the use of neural networks. Two
parallel neural networks are responsible for automatically adding
scents to 360Ā° videos: a scene detection network (responsible
for static, global content) and an action detection network
(responsible for dynamic, local content). A 360Ā° video dataset with
scent labels is also created and used for evaluating the robustness
of the proposed solution. The solution achieves a 69.19% olfactory
accuracy and 72.26% haptics accuracy during evaluation using
two different datasets
Multisensory 360 videos under varying resolution levels enhance presence
Omnidirectional videos have become a leading multimedia format for Virtual Reality applications. While live 360ā¦ videos offer a unique immersive experience, streaming of omnidirectional content at high resolutions is not always feasible in bandwidth-limited networks. While in the case of flat videos, scaling to lower resolutions works well, 360ā¦ video quality is seriously degraded because of the viewing distances involved in head-mounted displays. Hence, in this paper, we investigate first how quality degradation impacts the sense of presence in immersive Virtual Reality applications. Then, we are pushing the boundaries of 360ā¦ technology through the enhancement with multisensory stimuli. 48 participants experimented both 360ā¦ scenarios (with and without multisensory content), while they were divided randomly between four conditions characterised by different encoding qualities (HD, FullHD, 2.5K, 4K). The results showed that presence is not mediated by streaming at a higher bitrate. The trend we identified revealed however that presence is positively and significantly impacted by the enhancement with multisensory content. This shows that multisensory technology is crucial in creating more immersive experiences
360Ā° mulsemedia experience over next generation wireless networks - a reinforcement learning approach
The next generation of wireless networks targets aspiring key performance indicators, like very low latency, higher data rates and more capacity, paving the way for new generations of video streaming technologies, such as 360Ā° or omnidirectional videos. One possible application that could revolutionize the streaming technology is the 360Ā° MULtiple SEnsorial MEDIA (MULSEMEDIA) which enriches the 360Ā° video content with other media objects like olfactory, haptic or even thermoceptic ones. However, the adoption of the 360Ā° Mulsemedia applications might be hindered by the strict Quality of Service (QoS) requirements, like very large bandwidth and low latency for fast responsiveness to the users, inputs that could impact their Quality of Experience (QoE). To this extent, this paper introduces the new concept of 360Ā° Mulsemedia as well as it proposes the use of Reinforcement Learning to enable QoS provisioning over the next generation wireless networks that influences the QoE of the end-users
360Ā° mulsemedia experience over next generation wireless networks - a reinforcement learning approach
The next generation of wireless networks targets aspiring key performance indicators, like very low latency, higher data rates and more capacity, paving the way for new generations of video streaming technologies, such as 360Ā° or omnidirectional videos. One possible application that could revolutionize the streaming technology is the 360Ā° MULtiple SEnsorial MEDIA (MULSEMEDIA) which enriches the 360Ā° video content with other media objects like olfactory, haptic or even thermoceptic ones. However, the adoption of the 360Ā° Mulsemedia applications might be hindered by the strict Quality of Service (QoS) requirements, like very large bandwidth and low latency for fast responsiveness to the users, inputs that could impact their Quality of Experience (QoE). To this extent, this paper introduces the new concept of 360Ā° Mulsemedia as well as it proposes the use of Reinforcement Learning to enable QoS provisioning over the next generation wireless networks that influences the QoE of the end-users
MediaSync: Handbook on Multimedia Synchronization
This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences
Macro-and Micro-Expressions Facial Datasets: A Survey
Automatic facial expression recognition is essential for many potential applications. Thus, having a clear overview on existing datasets that have been investigated within the framework of face expression recognition is of paramount importance in designing and evaluating effective solutions, notably for neural networks-based training. In this survey, we provide a review of more than eighty facial expression datasets, while taking into account both macro-and micro-expressions. The proposed study is mostly focused on spontaneous and in-the-wild datasets, given the common trend in the research is that of considering contexts where expressions are shown in a spontaneous way and in a real context. We have also provided instances of potential applications of the investigated datasets, while putting into evidence their pros and cons. The proposed survey can help researchers to have a better understanding of the characteristics of the existing datasets, thus facilitating the choice of the data that best suits the particular context of their application
Multisensory 360 videos under varying resolution levels enhance presence
Omnidirectional videos have become a leading multimedia format for Virtual Reality applications. While live 360ā¦ videos offer a unique immersive experience, streaming of omnidirectional content at high resolutions is not always feasible in bandwidth-limited networks. While in the case of flat videos, scaling to lower resolutions works well, 360ā¦ video quality is seriously degraded because of the viewing distances involved in head-mounted displays. Hence, in this paper, we investigate first how quality degradation impacts the sense of presence in immersive Virtual Reality applications. Then, we are pushing the boundaries of 360ā¦ technology through the enhancement with multisensory stimuli. 48 participants experimented both 360ā¦ scenarios (with and without multisensory content), while they were divided randomly between four conditions characterised by different encoding qualities (HD, FullHD, 2.5K, 4K). The results showed that presence is not mediated by streaming at a higher bitrate. The trend we identified revealed however that presence is positively and significantly impacted by the enhancement with multisensory content. This shows that multisensory technology is crucial in creating more immersive experiences
Olfaction Modulates Inter-Subject Correlation of Neural Responses
Odors can be powerful stimulants. It is well-established that odors provide strong cues for recall of locations, people and events. The effects of specific scents on other cognitive functions are less well-established. We hypothesized that scents with different odor qualities will have a different effect on attention. To assess attention, we used Inter-Subject Correlation of the EEG because this metric is strongly modulated by attentional engagement with natural audiovisual stimuli.We predicted that scents known to be āenergizingā would increase Inter-Subject Correlation during watching of videos as compared to ācalmingā scents. In a first experiment, we confirmed this for eucalyptol and linalool while participants watched animated autobiographical narratives. The result was replicated in a second experiment, but did not generalize to limonene, also considered an āenergizingā odorant. In a third, double-blind experiment, we tested a battery of scents including single molecules, as well as mixtures, as participants watched various short video clips. We found a varying effect of odor on Inter-Subject Correlation across the various scents. This study provides a basis for reliably and reproducibly assessing effects of odors on brain activity. Future research is needed to further explore the effect of scent-based up-modulation in engagement on learning and memory performance. Educators, product developers and fragrance brands might also benefit from such objective neurophysiological measures
Recognizing emotions induced by wearable haptic vibration using noninvasive electroencephalogram
The integration of haptic technology into affective computing has led to a new field known as affective haptics. Nonetheless, the mechanism underlying the interaction between haptics and emotions remains unclear. In this paper, we proposed a novel haptic pattern with adaptive vibration intensity and rhythm according to the volume, and applied it into the emotional experiment paradigm. To verify its superiority, the proposed haptic pattern was compared with an existing haptic pattern by combining them with conventional visualāauditory stimuli to induce emotions (joy, sadness, fear, and neutral), and the subjectsā EEG signals were collected simultaneously. The features of power spectral density (PSD), differential entropy (DE), differential asymmetry (DASM), and differential caudality (DCAU) were extracted, and the support vector machine (SVM) was utilized to recognize four target emotions. The results demonstrated that haptic stimuli enhanced the activity of the lateral temporal and prefrontal areas of the emotion-related brain regions. Moreover, the classification accuracy of the existing constant haptic pattern and the proposed adaptive haptic pattern increased by 7.71 and 8.60%, respectively. These findings indicate that flexible and varied haptic patterns can enhance immersion and fully stimulate target emotions, which are of great importance for wearable haptic interfaces and emotion communication through haptics
- ā¦