A Positional Timewarp Accelerator for Mobile Virtual Reality Devices

Mobile virtual reality devices are becoming more common, and yet their performance is still too low to be considered ideal. Frame rate and latency are two of the most important areas that should improve in order to provide a high-quality virtual reality experience. Meanwhile, positional tracking is improving the immersive experience of new mobile virtual reality devices by allowing users to physically move about in space and see their corresponding view matched in the virtual world. Timewarping is a technique that can improve the perceived latency and frame rate of virtual reality systems, but the positional variant of timewarping has proven to be difficult to implement on mobile devices due to the performance demands. A depth-informed positional time warp cannot be fully parallelized due to the depth test required for each pixel or group of pixels.This thesis proposes a positional timewarp hardware accelerator for mobile devices. The accelerator accepts a rendered frame and depth image and produces an updated frame corresponding to the user’s head position and orientation. The accelerator is compatible with existing deferred rendering engines for minimal modification of the software structure. Its execution time is directly proportional to the image resolution and is agnostic of the scene complexity. The accelerator’s size can be adjusted to meet the latency requirement for a given image resolution. It can be integrated into a system-on-chip or fabricated as a separate chip.Three examples are designed and simulated to show the performance potential of this accelerator architecture. The designs provide latencies of 15.43 ms, 11.58 ms and 9.27 ms for frame rates of 64.7, 86.4 and 107.9 frames per second, respectively. Although the visual side-effects may be insufficiently few to completely disregard the GPU’s frame rate, the accelerator can still improve the end-to-end positional latency and is also capable of substituting the GPU in the case of dropped frames

Urban Informatics

This open access book is the first to systematically introduce the principles of urban informatics and its application to every aspect of the city that involves its functioning, control, management, and future planning. It introduces new models and tools being developed to understand and implement these technologies that enable cities to function more efficiently – to become ‘smart’ and ‘sustainable’. The smart city has quickly emerged as computers have become ever smaller to the point where they can be embedded into the very fabric of the city, as well as being central to new ways in which the population can communicate and act. When cities are wired in this way, they have the potential to become sentient and responsive, generating massive streams of ‘big’ data in real time as well as providing immense opportunities for extracting new forms of urban data through crowdsourcing. This book offers a comprehensive review of the methods that form the core of urban informatics from various kinds of urban remote sensing to new approaches to machine learning and statistical modelling. It provides a detailed technical introduction to the wide array of tools information scientists need to develop the key urban analytics that are fundamental to learning about the smart city, and it outlines ways in which these tools can be used to inform design and policy so that cities can become more efficient with a greater concern for environment and equity

Urban Informatics

This open access book is the first to systematically introduce the principles of urban informatics and its application to every aspect of the city that involves its functioning, control, management, and future planning. It introduces new models and tools being developed to understand and implement these technologies that enable cities to function more efficiently – to become ‘smart’ and ‘sustainable’. The smart city has quickly emerged as computers have become ever smaller to the point where they can be embedded into the very fabric of the city, as well as being central to new ways in which the population can communicate and act. When cities are wired in this way, they have the potential to become sentient and responsive, generating massive streams of ‘big’ data in real time as well as providing immense opportunities for extracting new forms of urban data through crowdsourcing. This book offers a comprehensive review of the methods that form the core of urban informatics from various kinds of urban remote sensing to new approaches to machine learning and statistical modelling. It provides a detailed technical introduction to the wide array of tools information scientists need to develop the key urban analytics that are fundamental to learning about the smart city, and it outlines ways in which these tools can be used to inform design and policy so that cities can become more efficient with a greater concern for environment and equity

Urban Informatics

This open access book is the first to systematically introduce the principles of urban informatics and its application to every aspect of the city that involves its functioning, control, management, and future planning. It introduces new models and tools being developed to understand and implement these technologies that enable cities to function more efficiently – to become ‘smart’ and ‘sustainable’. The smart city has quickly emerged as computers have become ever smaller to the point where they can be embedded into the very fabric of the city, as well as being central to new ways in which the population can communicate and act. When cities are wired in this way, they have the potential to become sentient and responsive, generating massive streams of ‘big’ data in real time as well as providing immense opportunities for extracting new forms of urban data through crowdsourcing. This book offers a comprehensive review of the methods that form the core of urban informatics from various kinds of urban remote sensing to new approaches to machine learning and statistical modelling. It provides a detailed technical introduction to the wide array of tools information scientists need to develop the key urban analytics that are fundamental to learning about the smart city, and it outlines ways in which these tools can be used to inform design and policy so that cities can become more efficient with a greater concern for environment and equity

Advances in Image Processing, Analysis and Recognition Technology

For many decades, researchers have been trying to make computers’ analysis of images as effective as the system of human vision is. For this purpose, many algorithms and systems have previously been created. The whole process covers various stages, including image processing, representation and recognition. The results of this work can be applied to many computer-assisted areas of everyday life. They improve particular activities and provide handy tools, which are sometimes only for entertainment, but quite often, they significantly increase our safety. In fact, the practical implementation of image processing algorithms is particularly wide. Moreover, the rapid growth of computational complexity and computer efficiency has allowed for the development of more sophisticated and effective algorithms and tools. Although significant progress has been made so far, many issues still remain, resulting in the need for the development of novel approaches

An Architecture For Rendering Web 3d Virtual Environments On Computers With Limited Graphic Processing Power [uma Arquitetura Para Renderização De Ambientes Virtuais 3d Na Web Em Computadores Com Capacidade De Processamento Gráfico Limitada]

This paper presents an architecture for 3D virtual environment rendering on the Web applied to computers with limited capacity for graphics processing, which do not have a GPU for rendering the virtual environment. 3D rendering of the scene is done remotely, on demand, and employs 3D warping technique for generating views locally in the client. Thus, the client is able to render new views while waiting for data requested to the server or even when there are problems in the transmission or receiving delay. The proposed architecture allows the virtual environment rendering using residual images, which make the data transmission between client and server more efficient since less information is sent when compared to complete frames rendered from the 3D model. A method for taking advantage of the residual images received by clients is proposed and implemented, improving the quality of the resulting views instead of being discarded. Furthermore, an approach to reducing the quality degradation problem of the views locally rendered in the client is presented. © 2010 ACM.Brazilian Computer Society (SBC)Bao, P., Gourlay, D., Superview 3d image warping for visibility gap errors (2003) IEEE Transactions on Consumer Electronics, 49 (1), pp. 177-182. , FebBao, P., Gourlay, D., Remote Walkthrough over Mobile Networks Using 3-D Image Warping and Streaming (2004) IEE Proceedings - Vision, Image and Signal Processing, 151 (4), pp. 329-336. , AugBao, P., Gourlay, D., A Framework for Remote Rendering of 3-D Scenes on Limited Mobile Devices (2006) IEEE Transactions on Multimedia, 8 (2), pp. 382-389. , AprilBao, P., Gourlay, D., Li, Y., Mobile Collaborative Walkthrough Using 3-D Image Warping and Streaming (2004) Proceedings of the 26th International Conference on Information Technology Interfaces, 1, pp. 279-285. , JuneBoukerche, A., Pazzi, R.W.N., Enhancing Remote Walkthrough for E-learning Environments on Mobile Devices over Heterogeneous Networks (2006) Proceedings of 2006 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 13-18. , JulyBoukerche, A., Pazzi, R.W.N., A Peer-to-Peer Approach for Remote Rendering and Image Streaming in Walkthrough Applications (2007) Proceedings of 2007 IEEE International Conference on Communications, pp. 1692-1697. , JuneChang, C.-F., Ger, S.-H., Enhancing 3D Graphics on Mobile Devices by Image-Based Rendering (2002) Proceedings of 2002 Third IEEE Pacific Rim Conference on Multimedia, pp. 1105-1111. , London, UK, Springer-VerlagChen, S.E., QuickTime VR - An Image-Based Approach to Virtual Environment Navigation (1995) Computer Graphics, 29, pp. 29-38. , Annual Conference SeriesChim, J., Lau, R., Leong, H., Si, A., CyberWalk: A Web-based Distributed Virtual Walkthrough Environment (2003) IEEE Transactions on Multimedia, 5 (4), pp. 503-515. , DecDos Santos, M.C., Pedrini, H., Renderização de Cenas Tridimensionais Interativas em Computadores de Baixo Desempenho (2008) Anais Do V Workshop de Realidade Virtual e Aumentada, pp. 1-8. , Bauru, SPHoppe, H., Progressive Meshes (1996) SIGGRAPH '96: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 99-108. , New York, NY, USA, ACM(2008) Método de Compressão de Imagens Com Perda, , http://www.jpeg.org/, acesso em: 30 de julho de 2008Lei, Y., Jiang, Z., Chen, D., Bao, H., Image-Based Walkthrough over Internet on Mobile Devices (2004) Proceedings of the Grid and Cooperative Computing - GCC 2004 Workshops, pp. 728-735. , Springer Berlin(2008), http://www.ltsp.org/, LTSP.org. acesso em: 30 de julho de 2008Mark, W., (1999) Post-Rendering 3D Image Warping: Visibility, Reconstruction, and Performance for Depth-Image Warping, , PhD thesis, University of North Carolina, Chapel Hill, NC, USA, AprMcMillan, L., (1997) An Image-Based Approach to Three-Dimensional Computer Graphics, , PhD thesis, University of North Carolina, Apr(2008), http://www.opengl.org/, OpenGL.org. acesso em: 30 de julho de 2008Quax, P., Geuns, B., Jehaes, T., Lamotte, W., Vansichem, G., On the Applicability of Remote Rendering of Networked Virtual Environments on Mobile Devices (2006) Proceedings of the International Conference on Systems and Networks Communication, p. 16. , Washington, DC, USA, IEEE Computer SocietyQuillet, J.-C., Thomas, G., Granier, X., Guitton, P., Marvie, J.-E., Using Expressive Rendering for Remote Visualization of Large City Models (2006) Proceedings of the 11th International Conference on 3D Web Technology, pp. 27-35. , New York, NY, USA, ACMWinkelholz, C., Weiss, M., Cakuli, P., Schreiber, M., Renkewitz, H., (2009) Biblioteca para a Manipulação de Objetos Tridimensionais No Formato VRML e X3D, , http://open-activewrl.sourceforge.net/, acesso em: 15 de janeiro de 2009Womack, P., Leech, J., (1998) OpenGL® Graphics with the X Window System®, , http://www.opengl.org/documentation/specs/glx/glx1.3.pdf, acesso em: 15 de janeiro de 2009Yang, S., Kuo, C.-C.J., Robust Graphics Streaming in Walkthrough Virtual Environments via Wireless Channels (2003) Proceedings of the IEEE Global Telecommunications Conference, 6, pp. 3191-3195. , DecZheng, H., Boyce, J., An Improved UDP Protocol for Video Transmission over Internet-to-Wireless Networks (2001) IEEE Transactions on Multimedia, 3 (3), pp. 356-365. , Se