1,777 research outputs found
Network streaming and compression for mixed reality tele-immersion
Bulterman, D.C.A. [Promotor]Cesar, P.S. [Copromotor
Omnidirectional view and multi-modal streaming in 3D tele-immersion system
3D Tele-immersion (3DTI) technology allows full-body, multi-modal content delivery among geographically dispersed users. In 3DTI, userâs 3D model will be captured by multiple RGB-D (color plus depth) cameras surround- ing userâs body. In addition, various sensors (e.g., motion sensors, medical sensors, wearable gaming consoles, etc.) specified by the application will be included to deliver a multi-modal experience.
In a traditional 2D live video streaming system, the interactivity of end users, choosing a specified viewpoint, has been crippled by the fact that they can only choose to see the physical scene captured by a physical camera, but not between two physical cameras. However, 3DTI system makes it possible rendering a 3D space where the viewers can view physical scene from arbitrary viewpoint.
In this thesis, we present systematic solutions of omnidirectional view in 3D tele-immersion system in a real-time manner and in an on-demand streaming manner, called FreeViewer and OmniViewer, respectively. we provide a complete multi-modal 3D video streaming/rendering solution, which achieves the feature of omnidirectional view in monoscopic 3D systems
MetaSpace II: Object and full-body tracking for interaction and navigation in social VR
MetaSpace II (MS2) is a social Virtual Reality (VR) system where multiple
users can not only see and hear but also interact with each other, grasp and
manipulate objects, walk around in space, and get tactile feedback. MS2 allows
walking in physical space by tracking each user's skeleton in real-time and
allows users to feel by employing passive haptics i.e., when users touch or
manipulate an object in the virtual world, they simultaneously also touch or
manipulate a corresponding object in the physical world. To enable these
elements in VR, MS2 creates a correspondence in spatial layout and object
placement by building the virtual world on top of a 3D scan of the real world.
Through the association between the real and virtual world, users are able to
walk freely while wearing a head-mounted device, avoid obstacles like walls and
furniture, and interact with people and objects. Most current virtual reality
(VR) environments are designed for a single user experience where interactions
with virtual objects are mediated by hand-held input devices or hand gestures.
Additionally, users are only shown a representation of their hands in VR
floating in front of the camera as seen from a first person perspective. We
believe, representing each user as a full-body avatar that is controlled by
natural movements of the person in the real world (see Figure 1d), can greatly
enhance believability and a user's sense immersion in VR.Comment: 10 pages, 9 figures. Video:
http://living.media.mit.edu/projects/metaspace-ii
3D-LIVE : live interactions through 3D visual environments
This paper explores Future Internet (FI) 3D-Media technologies and Internet of Things (IoT) in real and virtual environments in order to sense and experiment Real-Time interaction within live situations. The combination of FI testbeds and Living Labs (LL) would enable both researchers and users to explore capacities to enter the 3D Tele-Immersive (TI) application market and to establish new requirements for FI technology and infrastructure. It is expected that combining both FI technology pull and TI market pull would promote and accelerate the creation and adoption, by user communities such as sport practitioners, of innovative TI Services within sport events
Business Case and Technology Analysis for 5G Low Latency Applications
A large number of new consumer and industrial applications are likely to
change the classic operator's business models and provide a wide range of new
markets to enter. This article analyses the most relevant 5G use cases that
require ultra-low latency, from both technical and business perspectives. Low
latency services pose challenging requirements to the network, and to fulfill
them operators need to invest in costly changes in their network. In this
sense, it is not clear whether such investments are going to be amortized with
these new business models. In light of this, specific applications and
requirements are described and the potential market benefits for operators are
analysed. Conclusions show that operators have clear opportunities to add value
and position themselves strongly with the increasing number of services to be
provided by 5G.Comment: 18 pages, 5 figure
Source coding for transmission of reconstructed dynamic geometry: a rate-distortion-complexity analysis of different approaches
Live 3D reconstruction of a human as a 3D mesh with commodity electronics is becoming a reality. Immersive applications (i.e. cloud gaming, tele-presence) benefit from effective transmission of such content over a bandwidth limited link. In this paper we outline different approaches for compressing live reconstructed mesh geometry based on distributing mesh reconstruction functions between sender and receiver. We evaluate rate-performance-complexity of different configurations. First, we investigate 3D mesh compression methods (i.e. dynamic/static) from MPEG-4. Second, we evaluate the option of using octree based point cloud compression and receiver side surface reconstruction
Binaural Spatialization for 3D immersive audio communication in a virtual world
Realistic 3D audio can greatly enhance the sense of presence in
a virtual environment. We introduce a framework for capturing,
transmitting and rendering of 3D audio in presence of other
bandwidth savvy streams in a 3D Tele-immersion based virtual
environment. This framework presents an efficient implementation
for 3D Binaural Spatialization based on the positions of current
objects in the scene, including animated avatars and on the fly
reconstructed humans. We present a general overview of the
framework, how audio is integrated in the system and how it can
exploit the positions of the objects and room geometry to render
realistic reverberations using head related transfer functions.
The network streaming modules used to achieve lip-synchronization,
high-quality audio frame reception, and accurate localization for
binaural rendering are also presented. We highlight how large
computational and networking challenges can be addressed efficiently.
This represents a first step in adequate networking support for Binaural
3D Audio, useful for telepresence. The subsystem is successfully
integrated with a larger 3D immersive system, with state of art capturing
and rendering modules for visual data
Advancements and challenges towards a collaborative framework for 3D tele-immersive social networking
Social experiences realized through teleconferencing systems are still quite different from face to face meetings. The awareness that we are online and in a, to some extent, lesser real world are preventing us from really engaging and enjoying the event. Several reasons account for these differences and have been identified. We think it is now time to bridge these gaps and propose inspiring and innovative solutions in order to provide realistic, believable and engaging online experiences. We present a distributed and scalable framework named REVERIE that faces these challenges and provides a mix of these solutions. Applications built on top of the framework will be able to provide interactive, truly immersive, photo-realistic experiences to a multitude of users that for them will feel much more similar to having face to face meetings than the experience offered by conventional teleconferencing systems
- âŠ