Constructing sonified haptic line graphs for the blind student: first steps

Line graphs stand as an established information visualisation and analysis technique taught at various levels of difficulty according to standard Mathematics curricula. It has been argued that blind individuals cannot use line graphs as a visualisation and analytic tool because they currently primarily exist in the visual medium. The research described in this paper aims at making line graphs accessible to blind students through auditory and haptic media. We describe (1) our design space for representing line graphs, (2) the technology we use to develop our prototypes and (3) the insights from our preliminary work

Enhancing the broadcasted TV consumption experience with broadband omnidirectional video content

[EN] The current wide range of heterogeneous consumption devices and delivery technologies, offers the opportunity to provide related contents in order to enhance and enrich the TV consumption experience. This paper describes a solution to handle the delivery and synchronous consumption of traditional broadcast TV content and related broadband omnidirectional video content. The solution is intended to support both hybrid (broadcast/broadband) delivery technologies and has been designed to be compatible with the Hybrid Broadcast Broadband TV (HbbTV) standard. In particular, some specifications of HbbTV, such as the use of global timestamps or discovery mechanisms, have been adopted. However, additional functionalities have been designed to achieve accurate synchronization and to support the playout of omnidirectional video content in current consumption devices. In order to prove that commercial hybrid environments could be immediately enhanced with this type of content, the proposed solution has been included in a testbed, and objectively and subjectively evaluated. Regarding the omnidirectional video content, the two most common types of projections are supported: equirectangular and cube map. The results of the objective assessment show that the playout of broadband delivered omnidirectional video content in companion devices can be accurately synchronized with the playout on TV of traditional broadcast 2D content. The results of the subjective assessment show the high interest of users in this type of new enriched and immersive experience that contributes to enhance their Quality of Experience (QoE) and engagement.This work was supported by the Generalitat Valenciana, Investigacion Competitiva Proyectos, through the Research and Development Program Grants for Research Groups to be Consolidated, under Grant AICO/2017/059 and Grant AICO/2017Marfil-Reguero, D.; Boronat, F.; López, J.; Vidal Meló, A. (2019). Enhancing the broadcasted TV consumption experience with broadband omnidirectional video content. IEEE Access. 7:171864-171883. https://doi.org/10.1109/ACCESS.2019.2956084S171864171883

Image-Based Scene Representations for Head-Motion Parallax in 360° Panoramas

Creation and delivery of “RealVR” experiences essentially consists of the following four main steps: capture, processing, representation and rendering. In this chapter, we present, compare, and discuss two recent end-to-end approaches, Parallax360 by Luo et al. [9] and MegaParallax by Bertel et al. [3]. Both propose complete pipelines for RealVR content generation and novel-view synthesis with head-motion parallax for 360° environments.Parallax360 uses a robotic arm for capturing thousands of input views on the surface of a sphere. Based on precomputed disparity motion fields and pairwise optical flow, novel viewpoints are synthesized on the fly using flow-based blending of the nearest two to three input views which provides compelling head-motion parallax.MegaParallax proposes a pipeline for RealVR content generation and rendering that emphasizes casual, hand-held capturing. The approach introduces view-dependent flow-based blending to enable novel-view synthesis with head-motion parallax within a viewing area determined by the field of view of the input cameras and the capturing radius.We describe both methods and discuss their similarities and differences in corresponding steps in the RealVR pipeline and show selected results. The chapter ends by discussing advantages and disadvantages as well as outlining the most important limitations and future work.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 66599

マルチタスク学習を用いたシーン理解とデータ拡張による複合現実感の向上

早大学位記番号:新9140早稲田大

Camera positioning for 3D panoramic image rendering

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University London.Virtual camera realisation and the proposition of trapezoidal camera architecture are the two broad contributions of this thesis. Firstly, multiple camera and their arrangement constitute a critical component which affect the integrity of visual content acquisition for multi-view video. Currently, linear, convergence, and divergence arrays are the prominent camera topologies adopted. However, the large number of cameras required and their synchronisation are two of prominent challenges usually encountered. The use of virtual cameras can significantly reduce the number of physical cameras used with respect to any of the known camera structures, hence adequately reducing some of the other implementation issues. This thesis explores to use image-based rendering with and without geometry in the implementations leading to the realisation of virtual cameras. The virtual camera implementation was carried out from the perspective of depth map (geometry) and use of multiple image samples (no geometry). Prior to the virtual camera realisation, the generation of depth map was investigated using region match measures widely known for solving image point correspondence problem. The constructed depth maps have been compare with the ones generated using the dynamic programming approach. In both the geometry and no geometry approaches, the virtual cameras lead to the rendering of views from a textured depth map, construction of 3D panoramic image of a scene by stitching multiple image samples and performing superposition on them, and computation of virtual scene from a stereo pair of panoramic images. The quality of these rendered images were assessed through the use of either objective or subjective analysis in Imatest software. Further more, metric reconstruction of a scene was performed by re-projection of the pixel points from multiple image samples with a single centre of projection. This was done using sparse bundle adjustment algorithm. The statistical summary obtained after the application of this algorithm provides a gauge for the efficiency of the optimisation step. The optimised data was then visualised in Meshlab software environment, hence providing the reconstructed scene. Secondly, with any of the well-established camera arrangements, all cameras are usually constrained to the same horizontal plane. Therefore, occlusion becomes an extremely challenging problem, and a robust camera set-up is required in order to resolve strongly the hidden part of any scene objects. To adequately meet the visibility condition for scene objects and given that occlusion of the same scene objects can occur, a multi-plane camera structure is highly desirable. Therefore, this thesis also explore trapezoidal camera structure for image acquisition. The approach here is to assess the feasibility and potential of several physical cameras of the same model being sparsely arranged on the edge of an efficient trapezoid graph. This is implemented both Matlab and Maya. The quality of the depth maps rendered in Matlab are better in Quality

3D Computer Graphics and Virtual Reality

This chapter is dedicated to the description of 3D computer graphics used for the needs of virtual reality. Virtual reality (VR) is the use of computer technology to create a 3D virtual environment. The chapter presents some graphical features used in an environment as well as an explanation of good design practice. The chapter contains also the description of lighting settings, 3D objects/models and virtualization sequence, camera, and scenes where the wheelchair simulator is used as an example of the implementation environment

Heritage education through serious games. A web-based proposal for primary schools to cope with distance learning

[EN] In recent years a growing amount of research has shown interest in studying how virtual reality (VR) could be relevant in many fields. In this respect, VR has gained consideration throughout many applications such as education. Among other aims for its use in education, serious games based on VR were used to promote heritage and make students experience either far or inaccessible scenarios. Until now, VR-based applications have been mainly implemented using head mounted displays (HMD), which actually reduced their circulation. This gap is particularly remarkable in the current Sars-CoV19 pandemic because students, being at home or being at school without sharing equipment, cannot exploit educational programs based on this technology. The current paper proposes a web-based platform on which VR applications could be accessed on any device, either desktop- or mobile-based. The serious game was initially set up on a computer with a specialized software using a HMD, while the process of turning it into a web-based platform is described so that the used methodology could be available to those, who would like to follow it. This project is probably also able to cope with the general aim of making inaccessible objects available to students and, thus, to make the application useful even beyond the current pandemic emergency.The VAR.HEE. project – Virtual and Augmented Reality for Heritage and art Education in school and museum Experiences – was funded by the Free University of Bozen/Bolzano with a competitive call for proposals by the Central Research Commission in 2017. The project lasts three years, started in January 2018 and will end in December 2020 (June 2021, after Covid-19 health emergency).Luigini, A.; Fanini, B.; Basso, A.; Basso, D. (2020). Heritage education through serious games. A web-based proposal for primary schools to cope with distance learning. VITRUVIO - International Journal of Architectural Technology and Sustainability. 5(2):73-85. https://doi.org/10.4995/vitruvio-ijats.2020.14665OJS73855