Interacting with New York City Data by HoloLens through Remote Rendering

In the digital era, Extended Reality (XR) is considered the next frontier. However, XR systems are computationally intensive, and they must be implemented within strict latency constraints. Thus, XR devices with finite computing resources are limited in terms of quality of experience (QoE) they can offer, particularly in cases of big 3D data. This problem can be effectively addressed by offloading the highly intensive rendering tasks to a remote server. Therefore, we proposed a remote rendering enabled XR system that presents the 3D city model of New York City on the Microsoft HoloLens. Experimental results indicate that remote rendering outperforms local rendering for the New York City model with significant improvement in average QoE by at least 21%. Additionally, we clarified the network traffic pattern in the proposed XR system developed under the OpenXR standard

Learning to Estimate 3D Human Pose from Point Cloud

3D pose estimation is a challenging problem in computer vision. Most of the existing neural-network-based approaches address color or depth images through convolution networks (CNNs). In this paper, we study the task of 3D human pose estimation from depth images. Different from the existing CNN-based human pose estimation method, we propose a deep human pose network for 3D pose estimation by taking the point cloud data as input data to model the surface of complex human structures. We first cast the 3D human pose estimation from 2D depth images to 3D point clouds and directly predict the 3D joint position. Our experiments on two public datasets show that our approach achieves higher accuracy than previous state-of-art methods. The reported results on both ITOP and EVAL datasets demonstrate the effectiveness of our method on the targeted tasks

Towards a QoE Model to Evaluate Holographic Augmented Reality Devices

Augmented reality (AR) technology is developing fast and provides users with new ways to interact with the real-world surrounding environment. Although the performance of holographic AR multimedia devices can be measured with traditional quality-of-service parameters, a quality-of-experience (QoE) model can better evaluate the device from the perspective of users. As there are currently no well-recognized models for measuring the QoE of a holographic AR multimedia device, we present a QoE framework and model it with a fuzzy inference system to quantitatively evaluate the device

Sitting Posture Recognition Using a Spiking Neural Network

To increase the quality of citizens' lives, we designed a personalized smart chair system to recognize sitting behaviors. The system can receive surface pressure data from the designed sensor and provide feedback for guiding the user towards proper sitting postures. We used a liquid state machine and a logistic regression classifier to construct a spiking neural network for classifying 15 sitting postures. To allow this system to read our pressure data into the spiking neurons, we designed an algorithm to encode map-like data into cosine-rank sparsity data. The experimental results consisting of 15 sitting postures from 19 participants show that the prediction precision of our SNN is 88.52%

Technical Evaluation of HoloLens for Multimedia: A First Look

A recently released cutting-edge AR device, Microsoft HoloLens, has attracted considerable attention with its advanced capabilities. In this article, we report the design and execution of a series of experiments to quantitatively evaluate HoloLens' performance in head localization, real environment reconstruction, spatial mapping, hologram visualization, and speech recognition

Evaluating and Improving the Depth Accuracy of Kinect for Windows v2

Microsoft Kinect sensor has been widely used in many applications since the launch of its first version. Recently, Microsoft released a new version of Kinect sensor with improved hardware. However, the accuracy assessment of the sensor remains to be answered. In this paper, we measure the depth accuracy of the newly released Kinect v2 depth sensor, and obtain a cone model to illustrate its accuracy distribution. We then evaluate the variance of the captured depth values by depth entropy. In addition, we propose a trilateration method to improve the depth accuracy with multiple Kinects simultaneously. The experimental results are provided to ascertain the proposed model and method

EVM-CNN: Real-Time Contactless Heart Rate Estimation from Facial Video

With the increase in health consciousness, noninvasive body monitoring has aroused interest among researchers. As one of the most important pieces of physiological information, researchers have remotely estimated the heart rate (HR) from facial videos in recent years. Although progress has been made over the past few years, there are still some limitations, like the processing time increasing with accuracy and the lack of comprehensive and challenging datasets for use and comparison. Recently, it was shown that HR information can be extracted from facial videos by spatial decomposition and temporal filtering. Inspired by this, a new framework is introduced in this paper to remotely estimate the HR under realistic conditions by combining spatial and temporal filtering and a convolutional neural network. Our proposed approach shows better performance compared with the benchmark on the MMSE-HR dataset in terms of both the average HR estimation and short-time HR estimation. High consistency in short-time HR estimation is observed between our method and the ground truth

Balance Fatigue Design of Cast Steel Nodes in Tubular Steel Structures

Cast steel nodes are being increasingly popular in steel structure joint application as their advanced mechanical performances and flexible forms. This kind of joints improves the structural antifatigue capability observably and is expected to be widely used in the structures with fatigue loadings. Cast steel node joint consists of two parts: casting itself and the welds between the node and the steel member. The fatigue resistances of these two parts are very different; the experiment results showed very clearly that the fatigue behavior was governed by the welds in all tested configurations. This paper focuses on the balance fatigue design of these two parts in a cast steel node joint using fracture mechanics and FEM. The defects in castings are simulated by cracks conservatively. The final crack size is decided by the minimum of 90% of the wall thickness and the value deduced by fracture toughness. The allowable initial crack size could be obtained through the integral of Paris equation when the crack propagation life is considered equal to the weld fatigue life; therefore, the two parts in a cast steel node joint will have a balance fatigue life