2,033 research outputs found
Cutting the Cord in Virtual Reality
Today's virtual reality (VR) headsets require a cable connection to a PC or game console. This cable significantly limits the player’s mobility and hence her/his VR experience. The high data rate requirement of this link (multiple Gbps) precludes its replacement by WiFi. Thus, in this paper, we focus on using mmWave technology to deliver multi Gbps wireless communication between VR headsets and their game consoles. The challenge, however, is that mmWave signals can be easily blocked by the player's hand or head motion. We describe novel algorithms and system design that allow such mmWave links to sustain high data rates even in the presence of a blockage, enabling a high quality untethered VR experience.National Science Foundation (U.S.)Hong Kong University of Science and Technolog
Reliable Video Streaming over mmWave with Multi Connectivity and Network Coding
The next generation of multimedia applications will require the
telecommunication networks to support a higher bitrate than today, in order to
deliver virtual reality and ultra-high quality video content to the users. Most
of the video content will be accessed from mobile devices, prompting the
provision of very high data rates by next generation (5G) cellular networks. A
possible enabler in this regard is communication at mmWave frequencies, given
the vast amount of available spectrum that can be allocated to mobile users;
however, the harsh propagation environment at such high frequencies makes it
hard to provide a reliable service. This paper presents a reliable video
streaming architecture for mmWave networks, based on multi connectivity and
network coding, and evaluates its performance using a novel combination of the
ns-3 mmWave module, real video traces and the network coding library Kodo. The
results show that it is indeed possible to reliably stream video over cellular
mmWave links, while the combination of multi connectivity and network coding
can support high video quality with low latency.Comment: To be presented at the 2018 IEEE International Conference on
Computing, Networking and Communications (ICNC), March 2018, Maui, Hawaii,
USA (invited paper). 6 pages, 4 figure
Poster: Enabling Flexible Edge-assisted XR
Extended reality (XR) is touted as the next frontier of the digital future.
XR includes all immersive technologies of augmented reality (AR), virtual
reality (VR), and mixed reality (MR). XR applications obtain the real-world
context of the user from an underlying system, and provide rich, immersive, and
interactive virtual experiences based on the user's context in real-time. XR
systems process streams of data from device sensors, and provide
functionalities including perceptions and graphics required by the
applications. These processing steps are computationally intensive, and the
challenge is that they must be performed within the strict latency requirements
of XR. This poses limitations on the possible XR experiences that can be
supported on mobile devices with limited computing resources.
In this XR context, edge computing is an effective approach to address this
problem for mobile users. The edge is located closer to the end users and
enables processing and storing data near them. In addition, the development of
high bandwidth and low latency network technologies such as 5G facilitates the
application of edge computing for latency-critical use cases [4, 11]. This work
presents an XR system for enabling flexible edge-assisted XR.Comment: extended abstract of 2 pages, 1 figure, 2 table
FoVR: Attention-based VR Streaming through Bandwidth-limited Wireless Networks
Consumer Virtual Reality (VR) has been widely used in various application
areas, such as entertainment and medicine. In spite of the superb immersion
experience, to enable high-quality VR on untethered mobile devices remains an
extremely challenging task. The high bandwidth demands of VR streaming
generally overburden a conventional wireless connection, which affects the user
experience and in turn limits the usability of VR in practice. In this paper,
we propose FoVR, attention-based hierarchical VR streaming through
bandwidth-limited wireless networks. The design of FoVR stems from the insight
that human's vision is hierarchical, so that different areas in the field of
view (FoV) can be served with VR content of different qualities. By exploiting
the gaze tracking capacity of the VR devices, FoVR is able to accurately
predict the user's attention so that the streaming of hierarchical VR can be
appropriately scheduled. In this way, FoVR significantly reduces the bandwidth
cost and computing cost while keeping high quality of user experience. We
implement FoVR on a commercial VR device and evaluate its performance in
various scenarios. The experiment results show that FoVR reduces the bandwidth
cost by 88.9% and 76.2%, respectively compared to the original VR streaming and
the state-of-the-art approach
Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots
In the last decade, many medical companies and research groups have tried to
convert passive capsule endoscopes as an emerging and minimally invasive
diagnostic technology into actively steerable endoscopic capsule robots which
will provide more intuitive disease detection, targeted drug delivery and
biopsy-like operations in the gastrointestinal(GI) tract. In this study, we
introduce a fully unsupervised, real-time odometry and depth learner for
monocular endoscopic capsule robots. We establish the supervision by warping
view sequences and assigning the re-projection minimization to the loss
function, which we adopt in multi-view pose estimation and single-view depth
estimation network. Detailed quantitative and qualitative analyses of the
proposed framework performed on non-rigidly deformable ex-vivo porcine stomach
datasets proves the effectiveness of the method in terms of motion estimation
and depth recovery.Comment: submitted to IROS 201
- …