7 research outputs found
Enabling geometry-based 3-D tele-immersion with fast mesh compression and linear rateless coding
3-D tele-immersion (3DTI) enables participants in remote locations to share, in real time, an activity. It offers users interactive and immersive experiences, but it challenges current media-streaming solutions. Work in the past has mainly focused on the efficient delivery of image-based 3-D videos and on realistic rendering and reconstruction of geometry-based 3-D objects. The contribution of this paper is a real-time streaming component for 3DTI with dynamic reconstructed geometry. This component includes both a novel fast compression method and a rateless packet protection scheme specifically designed towards the requirements imposed by real time transmission of live-reconstructed mesh geometry. Tests on a large dataset show an encoding speed-up up to ten times at comparable compression ratio and quality, when compared with the high-end MPEG-4 SC3DMC mesh encoders. The implemented rateless code ensures complete packet loss protection of the triangle mesh object and a delivery delay within interactive bounds. Contrary to most linear fountain codes, the designed codec enables real-time progressive decoding allowing partial decoding each time a packet is received. This approach is compared with transmission over TCP in packet loss rates and latencies, typical in managed WAN and MAN networks, and heavily outperforms it in terms of end-to-end delay. The streaming component has been integrated into a larger 3DTI environment that includes state of the art 3-D reconstruction and rendering modules. This resulted in a prototype that can capture, compress transmit, and render triangle mesh geometry in real-time in realistic internet conditions as shown in experiments. Compared with alternative methods, lower interactive end-to-end delay and frame rates over three times higher are achieved
Network streaming and compression for mixed reality tele-immersion
Bulterman, D.C.A. [Promotor]Cesar, P.S. [Copromotor
Source coding for transmission of reconstructed dynamic geometry: a rate-distortion-complexity analysis of different approaches
Live 3D reconstruction of a human as a 3D mesh with commodity electronics is becoming a reality. Immersive applications (i.e. cloud gaming, tele-presence) benefit from effective transmission of such content over a bandwidth limited link. In this paper we outline different approaches for compressing live reconstructed mesh geometry based on distributing mesh reconstruction functions between sender and receiver. We evaluate rate-performance-complexity of different configurations. First, we investigate 3D mesh compression methods (i.e. dynamic/static) from MPEG-4. Second, we evaluate the option of using octree based point cloud compression and receiver side surface reconstruction
Scalable Video Streaming with Prioritised Network Coding on End-System Overlays
PhDDistribution over the internet is destined to become a standard approach for live broadcasting
of TV or events of nation-wide interest. The demand for high-quality live video
with personal requirements is destined to grow exponentially over the next few years. Endsystem
multicast is a desirable option for relieving the content server from bandwidth bottlenecks
and computational load by allowing decentralised allocation of resources to the users
and distributed service management. Network coding provides innovative solutions for a
multitude of issues related to multi-user content distribution, such as the coupon-collection
problem, allocation and scheduling procedure. This thesis tackles the problem of streaming
scalable video on end-system multicast overlays with prioritised push-based streaming.
We analyse the characteristic arising from a random coding process as a linear channel
operator, and present a novel error detection and correction system for error-resilient decoding,
providing one of the first practical frameworks for Joint Source-Channel-Network
coding. Our system outperforms both network error correction and traditional FEC coding
when performed separately. We then present a content distribution system based on endsystem
multicast. Our data exchange protocol makes use of network coding as a way to
collaboratively deliver data to several peers. Prioritised streaming is performed by means
of hierarchical network coding and a dynamic chunk selection for optimised rate allocation
based on goodput statistics at application layer. We prove, by simulated experiments, the
efficient allocation of resources for adaptive video delivery. Finally we describe the implementation
of our coding system. We highlighting the use rateless coding properties, discuss
the application in collaborative and distributed coding systems, and provide an optimised
implementation of the decoding algorithm with advanced CPU instructions. We analyse
computational load and packet loss protection via lab tests and simulations, complementing
the overall analysis of the video streaming system in all its components
Semantics-aware content delivery framework for 3D Tele-immersion
3D Tele-immersion (3DTI) technology allows full-body, multimodal interaction among geographically dispersed users, which opens a variety of possibilities in cyber collaborative applications such as art performance, exergaming, and physical rehabilitation. However, with its great potential, the resource and quality demands of 3DTI rise inevitably, especially when some advanced applications target resource-limited computing environments with stringent scalability demands. Under these circumstances, the tradeoffs between 1) resource requirements, 2) content complexity, and 3) user satisfaction in delivery of 3DTI services are magnified.
In this dissertation, we argue that these tradeoffs of 3DTI systems are actually avoidable when the underlying delivery framework of 3DTI takes the semantic information into consideration. We introduce the concept of semantic information into 3DTI, which encompasses information about the three factors: environment, activity, and user role in 3DTI applications. With semantic information, 3DTI systems are able to 1) identify the characteristics of its computing environment to allocate computing power and bandwidth to delivery of prioritized contents, 2) pinpoint and discard the dispensable content in activity capturing according to properties of target application, and 3) differentiate contents by their contributions on fulfilling the objectives and expectation of user’s role in the application so that the adaptation module can allocate resource budget accordingly. With these capabilities we can change the tradeoffs into synergy between resource requirements, content complexity, and user satisfaction.
We implement semantics-aware 3DTI systems to verify the performance gain on the three phases in 3DTI systems’ delivery chain: capturing phase, dissemination phase, and receiving phase. By introducing semantics information to distinct 3DTI systems, the efficiency improvements brought by our semantics-aware content delivery framework are validated under different application requirements, different scalability bottlenecks, and different user and application models.
To sum up, in this dissertation we aim to change the tradeoff between requirements, complexity, and satisfaction in 3DTI services by exploiting the semantic information about the computing environment, the activity, and the user role upon the underlying delivery systems of 3DTI. The devised mechanisms will enhance the efficiency of 3DTI systems targeting on serving different purposes and 3DTI applications with different computation and scalability requirements