Masking the Effects of Delays in Human-to-Human Remote Interaction

Humans can interact remotely with each other through computers. Systems supporting this include teleconferencing, games and virtual environments. There are delays from when a human does an action until it is reflected remotely. When delays are too large, they will result in inconsistencies in what the state of the interaction is as seen by each participant. The delays can be reduced, but they cannot be removed. When delays become too large the effects they create on the human-to-human remote interaction can be partially masked to achieve an illusion of insignificant delays. The MultiStage system is a human-to-human interaction system meant to be used by actors at remote stages creating a common virtual stage. Each actor is remotely represented by a remote presence created based on a stream of data continuously recorded about the actor and being sent to all stages. We in particular report on the subsystem of MultiStage masking the effects of delays. The most advanced masking approach is done by having each stage continuously look for late data, and when masking is determined to be needed, the system switches from using a live stream to a pre-recorded video of an actor. The system can also use a computable model of an actor creating a remote presence substituting for the live stream. The present prototype uses a simple human skeleton model

A distributed remote presence system masking the effects of delays in human-to-human remote interaction

This dissertation presents MultiStage, a human-to-human interaction system meant to be use by actors on a stage to interact and perform with actors on other stages as if they were on the same stage. MultiStage includes several local side stages and a global side. It uses a publish-subscribe model to handle the handover of data streams. Local side produces data streams about actors to global side. Local sides also subscribe to data streams from global side to create remote presence about actors. Global side receives data streams and sends data streams back to the local side according to subscriptions. When actors interact with remote actors, the system amplifies actors’ actions by adding text and animations to the remote presences. When the remote presences lag behind too much because of network and processing delays, the system applies various techniques to mask the effects of delays, including switching rapidly to a prerecorded video or animations of individual actors. Another use of the MultiStage video distribution model is called pVD. The Personal Video Distribution (pVD) system supports sending and viewing live and stored videos between any of a single user’s computers, and allows for a smooth handover of play back between computers. The system avoids any third parties, and relies only on the user’s personal computers. The architecture is comprised of functionality for sending videos, subscribing to videos, and maintaining the video playback state. The design has a local side sending and viewing videos, and a global side coordinating the switching and distribution of videos, and maintaining subscriptions and video state. A set of experiments was conducted to document the performance of the prototype. The results show that pVD global side has low CPU usage, and supports a handful of simultaneous exchanges of videos on a wireless network