Operating-system support for distributed multimedia

Multimedia applications place new demands upon processors, networks and operating systems. While some network designers, through ATM for example, have considered revolutionary approaches to supporting multimedia, the same cannot be said for operating systems designers. Most work is evolutionary in nature, attempting to identify additional features that can be added to existing systems to support multimedia. Here we describe the Pegasus project's attempt to build an integrated hardware and operating system environment from\ud the ground up specifically targeted towards multimedia

Construction and Evaluation of an Ultra Low Latency Frameless Renderer for VR.

© 2016 IEEE.Latency-the delay between a users action and the response to this action-is known to be detrimental to virtual reality. Latency is typically considered to be a discrete value characterising a delay, constant in time and space-but this characterisation is incomplete. Latency changes across the display during scan-out, and how it does so is dependent on the rendering approach used. In this study, we present an ultra-low latency real-time ray-casting renderer for virtual reality, implemented on an FPGA. Our renderer has a latency of 1 ms from tracker to pixel. Its frameless nature means that the region of the display with the lowest latency immediately follows the scan-beam. This is in contrast to frame-based systems such as those using typical GPUs, for which the latency increases as scan-out proceeds. Using a series of high and low speed videos of our system in use, we confirm its latency of 1 ms. We examine how the renderer performs when driving a traditional sequential scan-out display on a readily available HMO, the Oculus Rift OK2. We contrast this with an equivalent apparatus built using a GPU. Using captured human head motion and a set of image quality measures, we assess the ability of these systems to faithfully recreate the stimuli of an ideal virtual reality system-one with a zero latency tracker, renderer and display running at 1 kHz. Finally, we examine the results of these quality measures, and how each rendering approach is affected by velocity of movement and display persistence. We find that our system, with a lower average latency, can more faithfully draw what the ideal virtual reality system would. Further, we find that with low display persistence, the sensitivity to velocity of both systems is lowered, but that it is much lower for ours

Optimized mobile thin clients through a MPEG-4 BiFS semantic remote display framework

According to the thin client computing principle, the user interface is physically separated from the application logic. In practice only a viewer component is executed on the client device, rendering the display updates received from the distant application server and capturing the user interaction. Existing remote display frameworks are not optimized to encode the complex scenes of modern applications, which are composed of objects with very diverse graphical characteristics. In order to tackle this challenge, we propose to transfer to the client, in addition to the binary encoded objects, semantic information about the characteristics of each object. Through this semantic knowledge, the client is enabled to react autonomously on user input and does not have to wait for the display update from the server. Resulting in a reduction of the interaction latency and a mitigation of the bursty remote display traffic pattern, the presented framework is of particular interest in a wireless context, where the bandwidth is limited and expensive. In this paper, we describe a generic architecture of a semantic remote display framework. Furthermore, we have developed a prototype using the MPEG-4 Binary Format for Scenes to convey the semantic information to the client. We experimentally compare the bandwidth consumption of MPEG-4 BiFS with existing, non-semantic, remote display frameworks. In a text editing scenario, we realize an average reduction of 23% of the data peaks that are observed in remote display protocol traffic

Fast Algorithms for the Computation of Ranklets

Ranklets are orientation selective rank features with applications to tracking, face detection, texture and medical imaging. We introduce efficient algorithms that reduce their computational complexity from O(N logN) to O(!N + k), where N is the area of the filter. Timing tests show a speedup of one order of magnitude for typical usage, which should make Ranklets attractive for real-time applications

Using image morphing for memory-efficient impostor rendering on GPU

Real-time rendering of large animated crowds consisting thousands of virtual humans is important for several applications including simulations, games and interactive walkthroughs; but cannot be performed using complex polygonal models at interactive frame rates. For that reason, several methods using large numbers of pre-computed image-based representations, which are called as impostors, have been proposed. These methods take the advantage of existing programmable graphics hardware to compensate the computational expense while maintaining the visual fidelity. Making the number of different virtual humans, which can be rendered in real-time, not restricted anymore by the required computational power but by the texture memory consumed for the variety and discretization of their animations. In this work, we proposed an alternative method that reduces the memory consumption by generating compelling intermediate textures using image-morphing techniques. In order to demonstrate the preserved perceptual quality of animations, where half of the key-frames were rendered using the proposed methodology, we have implemented the system using the graphical processing unit and obtained promising results at interactive frame rates

Mobile Map Browsers: Anticipated User Interaction for Data Pre-fetching

When browsing a graphical display of geospatial data on mobile devices, users typically change the displayed maps by panning, zooming in and out, or rotating the device. Limited storage space on mobile devices and slow wireless communications, however, impede the performance of these operations. To overcome the bottleneck that all map data to be displayed on the mobile device need to be downloaded on demand, this thesis investigates how anticipated user interactions affect intelligent pre-fetching so that an on-demand download session is extended incrementally. User interaction is defined as a set of map operations that each have corresponding effects on the spatial dataset required to generate the display. By anticipating user interaction based on past behavior and intuition on when waiting for data is acceptable, it is possible to device a set of strategies to better prepare the device with data for future use. Users that engage with interactive map displays for a variety of tasks, whether it be navigation, information browsing, or data collection, experience a dynamic display to accomplish their goal. With vehicular navigation, the display might update itself as a result of a GPS data stream reflecting movement through space. This movement is not random, especially as is the case of moving vehicles and, therefore, this thesis suggests that mobile map data could be pre-fetched in order to improve usability. Pre-fetching memory-demanding spatial data can benefit usability in several ways, but in particular it can (1) reduce latency when downloading data over wireless connections and (2) better prepare a device for situations where wireless internet connectivity is weak or intermittent. This thesis investigates mobile map caching and devises an algorithm for pre-fetching data on behalf of the application user. Two primary models are compared: isotropic (direction-independent) and anisotropic (direction-dependent) pre-fetching. A prefetching simulation is parameterized with many trajectories that vary in complexity (a metric of direction change within the trajectory) and it is shown that, although anisotropic pre-fetching typically results in a better pre-fetching accuracy, it is not ideal for all scenarios. This thesis suggests a combination of models to accommodate the significant variation in moving object trajectories. In addition, other methods for pre-fetching spatial data are proposed for future research