22 research outputs found

    Dynamic Viewport-Adaptive Rendering in Distributed Interactive VR Streaming: Optimizing viewport resolution under latency and viewport orientation constraints

    Get PDF
    In streaming Virtual Reality to thin clients one of the main concerns is the massive bandwidth requirement of VR video. Additionally, streaming VR requires a low latency of less than 25ms to avoid cybersickness and provide a high Quality of Experience. Since a user is only viewing a portion of the VR content sphere at a time, researchers have leveraged this to increase the relative quality of the user viewport compared to peripheral areas. This way bandwidth can be saved, since the peripheral areas are streamed at a lower bitrate. In streaming 360°360\degree video this has resulted in the common strategy of tiling a video frame and delivering different quality tiles based on current available bandwidth and the user's viewport location. However, such an approach is not suitable for real-time Interactive VR streaming. Furthermore, streaming only the user's viewport results in the user observing unrendered or very low-quality areas at higher latency values. In order to provide a high viewport quality in Interactive VR, we propose the novel method of Dynamic Viewport-Adaptive Rendering. By rotating the frontal direction of the content sphere with the user gaze, we can dynamically render more or less of the peripheral area and thus increase the proportional resolution of the frontal direction in the video frame. We show that DVAR can successfully compensate for different system RTT values while offering a significantly higher viewport resolution than other implementations. We further discuss how DVAR can be easily extended by other optimization methods and discuss how we can incorporate head movement prediction to allow DVAR to optimally determine the amount of peripheral area to render, thus providing an optimal viewport resolution given the system constraints

    Visual Distortions in 360-degree Videos.

    Get PDF
    Omnidirectional (or 360°) images and videos are emergent signals being used in many areas, such as robotics and virtual/augmented reality. In particular, for virtual reality applications, they allow an immersive experience in which the user can interactively navigate through a scene with three degrees of freedom, wearing a head-mounted display. Current approaches for capturing, processing, delivering, and displaying 360° content, however, present many open technical challenges and introduce several types of distortions in the visual signal. Some of the distortions are specific to the nature of 360° images and often differ from those encountered in classical visual communication frameworks. This paper provides a first comprehensive review of the most common visual distortions that alter 360° signals going through the different processing elements of the visual communication pipeline. While their impact on viewers' visual perception and the immersive experience at large is still unknown-thus, it is an open research topic-this review serves the purpose of proposing a taxonomy of the visual distortions that can be encountered in 360° signals. Their underlying causes in the end-to-end 360° content distribution pipeline are identified. This taxonomy is essential as a basis for comparing different processing techniques, such as visual enhancement, encoding, and streaming strategies, and allowing the effective design of new algorithms and applications. It is also a useful resource for the design of psycho-visual studies aiming to characterize human perception of 360° content in interactive and immersive applications

    Streaming and User Behaviour in Omnidirectional Videos

    Get PDF
    Omnidirectional videos (ODVs) have gone beyond the passive paradigm of traditional video, offering higher degrees of immersion and interaction. The revolutionary novelty of this technology is the possibility for users to interact with the surrounding environment, and to feel a sense of engagement and presence in a virtual space. Users are clearly the main driving force of immersive applications and consequentially the services need to be properly tailored to them. In this context, this chapter highlights the importance of the new role of users in ODV streaming applications, and thus the need for understanding their behaviour while navigating within ODVs. A comprehensive overview of the research efforts aimed at advancing ODV streaming systems is also presented. In particular, the state-of-the-art solutions under examination in this chapter are distinguished in terms of system-centric and user-centric streaming approaches: the former approach comes from a quite straightforward extension of well-established solutions for the 2D video pipeline while the latter one takes the benefit of understanding users’ behaviour and enable more personalised ODV streaming

    Visual Distortions in 360-degree Videos

    Get PDF
    Omnidirectional (or 360-degree) images and videos are emergent signals in many areas such as robotics and virtual/augmented reality. In particular, for virtual reality, they allow an immersive experience in which the user is provided with a 360-degree field of view and can navigate throughout a scene, e.g., through the use of Head Mounted Displays. Since it represents the full 360-degree field of view from one point of the scene, omnidirectional content is naturally represented as spherical visual signals. Current approaches for capturing, processing, delivering, and displaying 360-degree content, however, present many open technical challenges and introduce several types of distortions in these visual signals. Some of the distortions are specific to the nature of 360-degree images, and often different from those encountered in the classical image communication framework. This paper provides a first comprehensive review of the most common visual distortions that alter 360-degree signals undergoing state of the art processing in common applications. While their impact on viewers' visual perception and on the immersive experience at large is still unknown ---thus, it stays an open research topic--- this review serves the purpose of identifying the main causes of visual distortions in the end-to-end 360-degree content distribution pipeline. It is essential as a basis for benchmarking different processing techniques, allowing the effective design of new algorithms and applications. It is also necessary to the deployment of proper psychovisual studies to characterise the human perception of these new images in interactive and immersive applications

    Mobile graphics: SIGGRAPH Asia 2017 course

    Get PDF
    Peer ReviewedPostprint (published version

    Deep learning and bidirectional optical flow based viewport predictions for 360° video coding

    Get PDF
    The rapid development of virtual reality applications continues to urge better compression of 360° videos owing to the large volume of content. These videos are typically converted to 2-D formats using various projection techniques in order to benefit from ad-hoc coding tools designed to support conventional 2-D video compression. Although recently emerged video coding standard, Versatile Video Coding (VVC) introduces 360° video specific coding tools, it fails to prioritize the user observed regions in 360° videos, represented by the rectilinear images called the viewports. This leads to the encoding of redundant regions in the video frames, escalating the bit rate cost of the videos. In response to this issue, this paper proposes a novel 360° video coding framework for VVC which exploits user observed viewport information to alleviate pixel redundancy in 360° videos. In this regard, bidirectional optical flow, Gaussian filter and Spherical Convolutional Neural Networks (Spherical CNN) are deployed to extract perceptual features and predict user observed viewports. By appropriately fusing the predicted viewports on the 2-D projected 360° video frames, a novel Regions of Interest (ROI) aware weightmap is developed which can be used to mask the source video and introduce adaptive changes to the Lagrange and quantization parameters in VVC. Comprehensive experiments conducted in the context of VVC Test Model (VTM) 7.0 show that the proposed framework can improve bitrate reduction, achieving an average bitrate saving of 5.85% and up to 17.15% at the same perceptual quality which is measured using Viewport Peak Signal-To-Noise Ratio (VPSNR)

    A REVIEW STUDY OF EUROPEAN R&D PROJECTS FOR SATELLITE COMMUNICATIONS IN 5G/6G ERA

    Get PDF
    Κατά τις τελευταίες δεκαετίες τα δορυφορικά συστήματα τηλεπικοινωνιών έχουν προσφέρει μια γκάμα από πολυμεσικές υπηρεσίες όπως δορυφορική τηλεόραση, δορυφορική τηλεφωνία και ευρυζωνική πρόσβαση στο διαδίκτυο. Οι μακροπρόθεσμες τεχνολογικές αναβαθμίσεις σε συνδυασμό με την προσθήκη νέων δορυφορικών συστημάτων γεωστατικής και ελλειπτικής τροχιάς και με την ενσωμάτωση τεχνολογιών πληροφορικής έχουν ωθήσει την αύξηση του μέγιστου εύρους των δορυφόρων στο 1Gbps σε μεμονωμένους δορυφόρους ενώ σε διάταξη αστερισμού μπορούν να ξεπεράσουν το 1 Tbps. Σε συνδυασμό με την μείωση του χρόνου απόκρισης σε ρυθμούς ανταγωνιστικούς με τις χερσαίες υποδομές ανοίγουν νέες ευκαιρίες και νέους ρόλους εντός ενός οικοσυστήματος ετερογενούς δικτύων 5ης γενιάς. Σε αυτήν την διατριβή, αξιολογούμε επιδοτούμενα επιστημονικά προγράμματα έρευνας και ανάπτυξης της Ευρωπαϊκής Επιτροπής Διαστήματος (ESA) και του προγράμματος επιδότησης Horizon 2020 της Ευρωπαϊκής Ένωσης, προκειμένου να εξηγήσουμε τις δυνατότητες των δορυφόρων εντός ενός ετερογενούς δικτύου 5ης γενιάς, αναφέρουμε συγκεκριμένα αυτά που αφορούν την εξέλιξη των δορυφορικών ψηφιακών συστημάτων και την ικανότητα ενσωμάτωσης τους σε τωρινές αλλά και μελλοντικές υποδομές χερσαίων τηλεπικοινωνιακών δικτύων μέσω της εμφάνισης νέων τεχνολογιών στις ηλεκτρονικές και οπτικές επικοινωνίες αέρος μαζί με την εμφάνιση τεχνολογιών πληροφορικής όπως της δικτύωσης βασισμένης στο λογισμικό και της εικονικοποίησης λειτουργιών δικτύου. Αναφερόμαστε στους στόχους του κάθε project ξεχωριστά και κατηγοριοποιημένα στους ακόλουθους τομείς έρευνας: -Συσσωμάτωση των δορυφόρων με τα επίγεια δίκτυα 5ης γενιάς με οργανωμένες μελέτες και στρατηγικές -Ενσωμάτωση των τεχνολογιών δικτύωσης βασισμένης στο λογισμικό και εικονικοποίησης λειτουργιών δικτύου στο δορυφορικών τμήμα των δικτύων 5ης γενιάς -Ο ρόλος των δορυφόρων σε εφαρμογές του διαδικτύου των πραγμάτων σε συνάφεια με τα χερσαία δίκτυα 5ης γενιάς -Ο ρόλος των δορυφόρων στην δίκτυα διανομής πολυμεσικού περιεχομένου & η επιρροή των πρωτοκόλλων διαδικτύου στην ποιότητα υπηρεσίας χρήστη κατά την διάρκεια μιας δορυφορικής σύνδεσης. -Μελλοντικές βελτιώσεις και εφαρμογές στα δορυφορικά συστήματα με έμφαση στα μελλοντικά πρότυπα του φυσικό επιπέδου Στο τέλος διαθέτουμε ένα παράρτημα που αφορά τεχνικές αναλύσεις στην εξέλιξη του φυσικού επιπέδου των δορυφορικών συστημάτων, συνοδευόμενο με την συσχετιζόμενη βιβλιογραφία για περαιτέρω μελέτη.Over the last decades satellite telecommunication systems offer many types of multimedia services like Satellite TV, telephony and broadband internet access. The long-term technological evolutions occurred into state-of-the-art satellite systems altogether with the addition of new high throughput geostatic and non-geostatic systems, individual satellites can now achieve a peak bandwidth of up to Gbps, and with possible extension into satellite constellation systems the total capacity can reach up to Tbps. Supplementary, with systems latency being comparable to terrestrial infrastructures and with integration of several computer science technologies, satellite systems can achieve new & more advanced roles inside a heterogeneous 5G network’s ecosystem. In this thesis, we have studied European Space Agency (ESA’s) and European Union’s (EU) Horizon 2020 Research and Development (R&D) funded projects in order to describe the satellite capabilities within a 5G heterogeneous network, mentioning the impact of the evolution of digital satellite communications and furthermore the integration with the state-of the art & future terrain telecommunication systems by new technologies occurred through the evolution of electronic & free space optical communications alongside with the integration of computer science’s technologies like Software Defined Networking (SDN) and Network Function Virtualization (NFV). In order to describe this evolution we have studied the concepts of each individual project, categorized chronically and individual by its scientific field of research. Our main scientific trends for this thesis are: -Satellite Integration studies & strategies into the 5G terrestrial networks -Integration of SDN and NFV technologies on 5G satellite component -Satellite’s role in the Internet of Things applications over 5G terrestrial networks -Satellite’s role in Content Distribution Networks & internet protocols impact over user’s Quality of Experience (QoE) over a satellite link -The future proposals upon the evolution of Satellite systems by upcoming improvements and corresponding standards Finally, we have created an Annex for technical details upon the evolution of physical layer of the satellite systems with the corresponding bibliography of this thesis for future study

    Delivery of 360° videos in edge caching assisted wireless cellular networks

    Get PDF
    In recent years, 360° videos have become increasingly popular on commercial social platforms, and are a vital part of emerging Virtual Reality (VR) applications. However, the delivery of 360° videos requires significant bandwidth resources, which makes streaming of such data on mobile networks challenging. The bandwidth required for delivering 360° videos can be reduced by exploiting the fact that users are interested in viewing only a part of the video scene, the requested viewport. As different users may request different viewports, some parts of the 360° scenes may be more popular than others. 360° video delivery on mobile networks can be facilitated by caching popular content at edge servers, and delivering it from there to the users. However, existing edge caching schemes do not take full potential of the unequal popularity of different parts of a video, which renders them inefficient for caching 360° videos. Inspired by the above, in this thesis, we investigate how advanced 360° video coding tools, i.e., encoding into multiple quality layers and tiles, can be utilized to build more efficient wireless edge caching schemes for 360° videos. The above encoding allows the caching of only the parts of the 360° videos that are popular in high quality. To understand how edge caching schemes can benefit from 360° video coding, we compare the caching of 360° videos encoded into multiple quality layers and tiles with layer-agnostic and tile-agnostic schemes. To cope with the fact that the content popularity distribution may be unknown, we use machine learning techniques, for both Video on Demand (VoD), and live streaming scenarios. From our findings, it is clear that by taking into account the aforementioned 360° video characteristics leads to an increased performance in terms of the quality of the video delivered to the users, and the usage of the backhaul links
    corecore