Image-based Techniques for the Creation and Display of Photorealistic Interactive Virtual Environments

International audienceIn this paper we introduce an image-based 3D capture process for the creation and display of photorealistic virtual environments (VEs). The resulting VEs aim to realistically recreate existing real-world scenes that can be displayed in a range of immersive VR systems using a high-quality, view-dependent algorithm and further enhanced using advanced vegetation, shadow display algorithms and 3D sound. The scenes, an archaeological site and an urban environment, were chosen according to real-world applications in the areas of urban planning/architecture and cultural heritage education. The users in each case are able to reconstruct or manipulate elements of the VEs according to their needs, as these have been specified through a detailed user requirements survey. Furthermore, a user task analysis and scenario-based approach has been adopted for the design of the virtual prototypes and the evaluation, which is currently underway. This work is being developed in the context of the EU-funded research project CREATE and the first examples of the prototype system in use are described and demonstrated in this paper

Towards Rapid Generation and Visualisation of Large 3D Urban Landscapes for Mobile Device Navigation

In this paper a procedural 3D modelling solution for mobile devices is presented based on scripting algorithms allowing for both the automatic and also semi-automatic creation of photorealistic quality virtual urban content. The combination of aerial images, GIS data, 2D ground maps and terrestrial photographs as input data coupled with a user-friendly customized interface permits the automatic and interactive generation of large-scale, accurate, georeferenced and fully-textured 3D virtual city content, content that can be specially optimized for use with mobile devices but also with navigational tasks in mind. Furthermore, a user-centred mobile virtual reality (VR) visualisation and interaction tool operating on PDAs (Personal Digital Assistants) for pedestrian navigation is also discussed. Via this engine, the import and display of various navigational file formats (2D and 3D) is supported, including a comprehensive front-end user-friendly graphical user interface providing immersive virtual 3D navigation

A pointillism style for the non-photorealistic display of augmented reality scenes

The ultimate goal of augmented reality is to provide the user with a view of the surroundings enriched by virtual objects. Practically all augmented reality systems rely on standard real-time rendering methods for generating the images of virtual scene elements. Although such conventional computer graphics algorithms are fast, they often fail to produce sufficiently realistic renderings. The use of simple lighting and shading methods, as well as the lack of knowledge about actual lighting conditions in the real surroundings, cause virtual objects to appear artificial. We have recently proposed a novel approach for generating augmented reality images. Our method is based on the idea of applying stylization techniques for reducing the visual realism of both the camera image and the virtual graphical objects. Special non-photorealistic image filters are applied to the camera video stream. The virtual scene elements are rendered using non-photorealistic rendering methods. Since both the camera image and the virtual objects are stylized in a corresponding way, they appear very similar. As a result, graphical objects can become indistinguishable from the real surroundings. Here, we present a new method for the stylization of augmented reality images. This approach generates a painterly "brush stroke" rendering. The resulting stylized augmented reality video frames look similar to paintings created in the "pointillism" style. We describe the implementation of the camera image filter and the non-photorealistic renderer for virtual objects. These components have been newly designed or adapted for this purpose. They are fast enough for generating augmented reality images in real-time and are customizable. The results obtained using our approach are very promising and show that it improves immersion in augmented reality

Visual communication in urban planning and urban design

This report documents the current status of visual communication in urban design and planning. Visual communication is examined through discussion of standalone and network media, specifically concentrating on visualisation on the World Wide Web(WWW).Firstly, we examine the use of Solid and Geometric Modelling for visualising urban planning and urban design. This report documents and compares examples of the use of Virtual Reality Modelling Language (VRML) and proprietary WWW based Virtual Reality modelling software. Examples include the modelling of Bath and Glasgow using both VRML 1.0 and 2.0. A review is carried out on the use of Virtual Worldsand their role in visualising urban form within multi-user environments. The use of Virtual Worlds is developed into a case study of the possibilities and limitations of Virtual Internet Design Arenas (ViDAs), an initiative undertaken at the Centre for Advanced Spatial Analysis, University College London. The use of Virtual Worlds and their development towards ViDAs is seen as one of the most important developments in visual communication for urban planning and urban design since the development plan.Secondly, photorealistic media in the process of communicating plans is examined.The process of creating photorealistic media is documented, examples of the Virtual Streetscape and Wired Whitehall Virtual Urban Interface System are provided. The conclusion is drawn that although the use of photo-realistic media on the WWW provides a way to visually communicate planning information, its use is limited. The merging of photorealistic media and solid geometric modelling is reviewed in the creation of Augmented Reality. Augmented Reality is seen to provide an important step forward in the ability to quickly and easily visualise urban planning and urban design information.Thirdly, the role of visual communication of planning data through GIS is examined interms of desktop, three dimensional and Internet based GIS systems. The evolution to Internet GIS is seen as a critical component in the development of virtual cities which will allow urban planners and urban designers to visualise and model the complexity of the built environment in networked virtual reality.Finally a viewpoint is put forward of the Virtual City, linking Internet GIS with photorealistic multi-user Virtual Worlds. At present there are constraints on how far virtual cities can be developed, but a view is provided on how these networked virtual worlds are developing to aid visual communication in urban planning and urban design

A Modular Approach to the Development of Interactive Augmented Reality Applications.

Augmented reality (AR) technologies are becoming increasingly popular as a result of the increase in the power of mobile computing devices. Emerging AR applications have the potential to have an enormous impact on industries such as education, healthcare, research, training and entertainment. There are currently a number of augmented reality toolkits and libraries available for the development of these applications; however, there is currently no standard tool for development. In this thesis we propose a modular approach to the organization and development of AR systems in order to enable the creation novel AR experiences. We also investigate the incorporation of the framework that resulted from our approach into game engines to enable the creation and visualization of immersive virtual reality experiences. We address issues in the development process of AR systems and provide a solution for reducing the time, cost and barrier of entry for development while simultaneously providing a framework in which researchers can test and apply advanced augmented reality technologies

An investigation to examine the most appropriate methodology to capture historical and modern preserved anatomical specimens for use in the digital age to improve access: a pilot study

Anatomico-pathological specimens constitute a valuable component of many medical museums or institutional collections but can be limited in their impact on account of both physical and intellectual inaccessibility. Further concerns relate to conservation as anatomical specimens may be subject to tissue deterioration, constraints imposed by spatial or financial limitations of the host institution, or accident-based destruction. In awareness of these issues, a simple and easily implementable methodology to increase accessibility, impact and conservation of anatomical specimens is proposed which combines photogrammetry, object virtual reality (object VR), and interactive portable document format (PDF) with supplementary historical and anatomical commentary. The methodology was developed using wet, dry, and plastinated specimens from the historical and modern collections in the Museum of Anatomy at the University of Glasgow. It was found that photogrammetry yielded excellent results for plastinated specimens and showed potential for dry specimens, while object VR produced excellent photorealistic virtual specimens for all materials visualised. Use of PDF as output format was found to allow for the addition of textual, visual, and interactive content, and as such supplemented the virtual specimen with multidisciplinary information adaptable to the needs of various audiences. The results of this small-scale pilot study indicate the beneficial nature of combining these established techniques into a methodology for the digitisation and utilisation of historical anatomical collections in particular, but also collections of material culture more broadly

U-DiVE: Design and evaluation of a distributed photorealistic virtual reality environment

This dissertation presents a framework that allows low-cost devices to visualize and interact with photorealistic scenes. To accomplish this task, the framework makes use of Unity’s high-definition rendering pipeline, which has a proprietary Ray Tracing algorithm, and Unity’s streaming package, which allows an application to be streamed within its editor. The framework allows the composition of a realistic scene using a Ray Tracing algorithm, and a virtual reality camera with barrel shaders, to correct the lens distortion needed for the use on an inexpensive cardboard. It also includes a method to collect the mobile device’s spatial orientation through a web browser to control the user’s view, delivered via WebRTC. The proposed framework can produce low-latency, realistic and immersive environments to be accessed through low-cost HMDs and mobile devices. To evaluate the structure, this work includes the verification of the frame rate achieved by the server and mobile device, which should be higher than 30 FPS for a smooth experience. In addition, it discusses whether the overall quality of experience is acceptable by evaluating the delay of image delivery from the server up to the mobile device, in face of user’s movement. Our tests showed that the framework reaches a mean latency around 177 (ms) with household Wi-Fi equipment and a maximum latency variation of 77.9 (ms), among the 8 scenes tested.Esta dissertação apresenta um framework que permite que dispositivos de baixo custo visualizem e interajam com cenas fotorrealísticas. Para realizar essa tarefa, o framework faz uso do pipeline de renderização de alta definição do Unity, que tem um algoritmo de rastreamento de raio proprietário, e o pacote de streaming do Unity, que permite o streaming de um aplicativo em seu editor. O framework permite a composição de uma cena realista usando um algoritmo de Ray Tracing, e uma câmera de realidade virtual com shaders de barril, para corrigir a distorção da lente necessária para usar um cardboard de baixo custo. Inclui também um método para coletar a orientação espacial do dispositivo móvel por meio de um navegador Web para controlar a visão do usuário, entregue via WebRTC. O framework proposto pode produzir ambientes de baixa latência, realistas e imersivos para serem acessados por meio de HMDs e dispositivos móveis de baixo custo. Para avaliar a estrutura, este trabalho considera a verificação da taxa de quadros alcançada pelo servidor e pelo dispositivo móvel, que deve ser superior a 30 FPS para uma experiência fluida. Além disso, discute se a qualidade geral da experiência é aceitável, ao avaliar o atraso da entrega das imagens desde o servidor até o dispositivo móvel, em face da movimentação do usuário. Nossos testes mostraram que o framework atinge uma latência média em torno dos 177 (ms) com equipamentos wi-fi de uso doméstico e uma variação máxima das latências igual a 77.9 (ms), entre as 8 cenas testadas