Immersive Authoring of 360 Degree Interactive Applications

Although there are proposals in the literature for authoring mulsemedia (mul tiple se nsorial media) applications with 2D content, there are no suitable solutions when it comes to 360° content. Moreover, little consensus on 360° mulsemedia authoring methodology exists. Aiming at filling this gap, we propose the concept of immersive authoring of 360° multisensory applications. Our proposal comprises an immersive 360° authoring environment to bring the author closer to the final user presentation environment. We implemented our proposal in AMUSEVR, a virtual-reality (VR) environment for authoring 360° mulsemedia applications. We see it as an alternative or a possible complement to available 2D mulsemedia authoring tools. AMUSEVR provides creation and editing of interactive multiple sensorial media scenes by directly arranging objects in a 3D space using VR technology. Also, the tool allows users to run their applications through AMUSEVR viewer mode. We used the Goal Question Metric (GQM) approach to plan our tests and a group of users evaluated the tool with the SUS and UEQ questionnaires, obtaining a SUS score of 82.25 and an excellent UEQ benchmark, which are very promising results.10.13039/501100002322-Brazilian National Council for Scientific and Technological Development (CNPq), Carlos Chagas Filho Foundation for Research Support in the State of Rio de Janeiro (FAPERJ), Coordination for the Improvement of Higher Education Personnel (CAPES), and Program for Institutional Internationalization (CAPES PrInt

Toward Content-Driven Intelligent Authoring of Mulsemedia Applications

Improvement of Higher Education Personnel—Brazil; 10.13039/501100004586-Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro; 10.13039/501100003593-Conselho Nacional de Desenvolvimento Científico e Tecnológico; CAPES PRINT

Semi-Automatic mulsemedia authoring analysis from the user's perspective

Mulsemedia (Multiple Sensorial Media) authoring is a complex task that requires the author to scan the media content to identify the moments to activate sensory effects. A novel proposal is to integrate content recognition algorithms into authoring tools to alleviate the authoring effort. Such algorithms could potentially replace the work of the human author when analyzing audiovisual content, by performing automatic extraction of sensory effects. Besides that, the semi-Automatic method proposes to maintain the author subjectivity, allowing the author to define which sensory effects should be automatically extracted. This paper presents an evaluation of the proposed semi-Automatic authoring considering the point of view of users. Experiments were done with the STEVE 2.0 mulsemedia authoring tool. Our work uses the GQM (Goal Question Metric) methodology, a questionnaire for collecting users' feedback, and analyzes the results. We conclude that users believe that the semi-Automatic authoring is a positive addition to the authoring method.CAPES, CAPES Print, CNPQ, INCT-MACC and FAPERJ

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Uma metodologia e um ambiente MDE para a verificação de aplicações hipermídia

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia de Automação e Sistemas, Florianópolis, 2016No desenvolvimento de aplicações hipermídia, o projetista pode erroneamente inserir comportamentos indesejados. Metodologias baseadas em teste ou análise de linha temporal para verificar a corretude de aplicações são limitadas, por não serem exaustivas e serem consumidoras de tempo. Outra alternativa é a utilização de metodologias baseadas em verificação formal, que permitem uma análise exaustiva e mais rápida da aplicação. A verificação formal requer que a aplicação e os comportamentos a serem verificados estejam representados em linguagens formais, de difícil aprendizagem por um projetista de aplicação hipermídia. O presente trabalho propõe uma metodologia baseada no uso de verificação formal por model-checking, a partir de uma representação da aplicação, das propriedades a serem verificadas e do diagnostico de eventuais erros, ambos em linguagens e representações de fácil uso e entendimento para o projetista destas aplicações. Essa metodologia é dividida em quatro fases: Modelagem, Transformação, Verificação e Diagnóstico/Correção. Inicialmente, o projetista codifica sua aplicação em alguma linguagem de domínio específico (por exemplo, NCL ou SMIL), e especifica os comportamentos desejados a serem verificados numa linguagem de descrição simples, proposta neste trabalho. A seguir, essas descrições das aplicações e comportamentos são transformadas, seguindo a abordagem MDE (Model Driven Engineering), nos modelos formais utilizados na verificação. Em caso de algum comportamento desejado não ser satisfeito, a ferramenta de model-checking oferece um contraexemplo que, após transformação, é apresentado na forma de uma linha de tempo, permitindo diagnosticar a origem do erro e fornecer informações para a sua correção. Para apoiar a metodologia proposta, foi construído um protótipo de um ambiente de desenvolvimento, no qual o projetista pode verificar o comportamento de sua aplicação. As avaliações da metodologia e de seu ambiente, realizadas em diversas aplicações hipermídia mostram suas potencialidades de uso para aplicações mais complexas e no caso de edição "ao vivo".Abstract : In the development of hypermedia applications, the designer can mistakenly insert undesirable behaviors. Methodologies based on tests or timeline analysis to verify the correctness of applications are limited because they are not exhaustive and are time consuming. Another alternative is the use of methodologies based on formal verification, allowing an exhaustive and more fast analysis of the application. Formal verification requires that the application and behavior to be verified are represented in formal languages, which are difficult to learn by a hypermedia application designer. This work proposes a methodology based on the use of formal verification by model-checking, from an application representation, the properties to be verified and the diagnosis of errors, both in languages and representations of easy use and understanding by designer of these applications. This methodology is divided into four phases: Modeling, Transformation, Verification and Diagnosis/Correction. Initially, the designer encodes his application in any domain specific language (eg, NCL or SMIL), and specifies the desired behaviors to be checked in a simple description language proposed in this work. Then these descriptions of applications and behaviors are transformed, following the MDE approach (Model Driven Engineering), in formal models used for verification. If some desired behavior is not satisfied, the model-checking tool provides a counterexample that, after processing, is presented as a timeline, allowing to diagnose the source of the error and provide information for its correction. To support the proposed methodology, a prototype development environment was built, in which the designer can verify the behavior of your application. Evaluations of the methodology and its environment, performed in several hypermedia applications, showed their potential of use for more complex applications and in the case of editing "live"

Understanding the challenges of immersive technology use in the architecture and construction industry: A systematic review

Despite the increasing scholarly attention being given to immersive technology applications in the architecture and construction industry, very few studies have explored the key challenges associated with their usage, with no aggregation of findings or knowledge. To bridge this gap and gain a better understanding of the state-of-the-art immersive technology application in the architecture and construction sector, this study reviews and synthesises the existing research evidence through a systematic review. Based on rigorous inclusion and exclusion criteria, 51 eligible articles published between 2010 and 2019 (inclusive) were selected for the final review. Predicted upon a wide range of scholarly journals, this study develops a generic taxonomy consisting of various dimensions. The results revealed nine (9) critical challenges which were further ranked in the following order: Infrastructure; Algorithm Development; Interoperability; General Health and Safety; Virtual Content Modelling; Cost; Skills Availability; Multi-Sensory Limitations; and Ethical Issues

Literacy for digital futures : Mind, body, text

The unprecedented rate of global, technological, and societal change calls for a radical, new understanding of literacy. This book offers a nuanced framework for making sense of literacy by addressing knowledge as contextualised, embodied, multimodal, and digitally mediated. In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes – Mind and Materiality; Body and Senses; and Texts and Digital Semiotics – to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice

Facilitating immersion, engagement and flow in multi-user virtual environments

Virtual worlds are providing myriad opportunities for the development of innovative curricula for tertiary educators. They provide a virtual meeting space for those students and lecturers who are geographically remote from one another, rendering distance irrelevant and facilitating the formation of community. This paper will look at those factors - physical, social, virtual and those related to pedagogy - which facilitate immersion in virtual worlds; that suspension of disbelief which generates the feeling of presence or 'being there', crucial to promoting student engagement and ultimately, flow

Development of interactive and distributed virtual environments for immersive communication in the furniture, fixture and equipment sector

Despite the significance of the furniture fixture and equipment (FFE) sector to the UK’s economy and the construction industry, this sector faces challenges that impede its productivity and performance, including an inability to meet end-user expectations in the delivery of its services. Lack of adoption of digitalisation and poor design communication between the stakeholders have been identified as one of the issues leading to challenges in the FFE Sector. In this context, visual representation offered by virtual reality (VR) can play a critical role in communicating the designs with the stakeholders effectively. However, evidence suggests that the current state of the VR application in the FFE sector lacks three critical advancements namely BIM-data interaction (interaction with the meta-data associated with the FFE elements), human-building interaction (interaction of stakeholders with FFE elements in the virtual environment) and human-human interactions (multi-user interaction). Therefore, the aim of this study was to bridge this gap through the development and testing of novel virtual environments for immersive communication between FFE and its construction project stakeholders. Furthermore, pre-conditions for the successful implementation of the developed VR applications were evaluated in this study through experimentation.A sequential, exploratory, mixed-method research design was adopted for this study in three phases. In phase one, an extensive literature investigation was carried out to acquire deeper knowledge of existing literature to understand the state-of-the-art developments of immersive technologies in the construction industry with a specific focus on the current challenges and benefits of implementing immersive technology. Phase two of the study involved the development and testing of immersive, distributed and interactive VR applications for various scenarios of the FFE communications for construction. Each application was developed by applying rapid application development methodologies and combining BIM, game engine and low latency cloud server development paradigm. The developments were tested through quasi-experiments and evaluation by stakeholders to ascertain usefulness and utility in the FFE sector context. The first development focused on interactive VR for FFE and was tested among (n = 12) stakeholders using a quasi-experiment in a single-group, pre-test-post-test design. The second development focused on distributed immersion for FFE design communication and was tested among construction stakeholders involved in FFE design decision-making (n = 26). The distributed VR application was further tested further among (n = 9) stakeholders in the context of FFE retail and showcasing of FFE products. The experimental approaches in the second phase adopted combined quantitative and qualitative evaluations to ascertain system usability which fed into further development and finetuning of applications. Finally, in the third phase of the study, the interactive and distributed VR applications were validated among wider group of construction stakeholders (n = 117) using a survey to ascertain industry-wide utility and usefulness as well as establish factors that influence their wider adoption in the sector. A combination of descriptive and inferential statistics was applied to establish findings including Kruskal-Wallis and ANOVA to measure variations in views across different segments of the population of respondents.Findings indicated that the interactive distributed immersive virtual FFE environment can enhance the productivity of the design team through a collaborative virtual workspace offering a synchronised networked design testing and review platform. Furthermore, it can reduce the time required for the stakeholders (Client/end-user, architect, FFE designer/contractor, FFE manufacturer) to comprehend and test the design options. In addition, the developed VR applications can enhance the design communication and quality of the design and encourage a collaborative culture in the industry and improve the design satisfaction of the stakeholders. It was also identified that the VR applications developed for this study can reduce the time required for design decision-making significantly when compared with traditional methods. In the retail and product showcasing context, the system was found to be a highly efficient and viable tool, which can deliver a compelling and richer experience similar to an FFE in-store experience. The testing also revealed that the proposed system not only improves the sense of presence but also brings in a new dimension of a sense of being together, which has a positive impact on decision making. Cumulative findings of this study revealed that distributed and interactive VR has become essential to digitalising the FFE sector’s design communication, with improved design communication being regarded as the most important benefit of its use. Conversely, the most critical challenge that inhibits the implementation of these two VR applications in the FFE sector is the perceived cost. This research proposes a step change in the way furniture design is communicated and coordinated through an immersive virtual experience, thus allowing informed decisions making and creating shared understanding before the commencement of the construction activity