Cracks in the Glass: The Emergence of a New Image Typology from the Spatio-temporal Schisms of the 'Filmic' Virtual Reality Panorama

Virtual Reality Panoramas have fascinated me for some time; their interactive nature affording a spectatorial engagement not evident within other forms of painting or digital imagery. This interactivity is not generally linear as is evident in animation or film, nor is the engagement with the image reduced to the physical or visual border of the image, as its limit is never visible to the viewer in its entirety. Further, the time taken to interact and navigate across the Virtual Reality panorama’s surface is not reflected or recorded within the observed image. The procedural construction of the Virtual Reality panorama creates an a-temporal image event that denies the durée of its own index and creation. This is particularly evident in the cinematic experiments conducted by Jeffrey Shaw in the 1990s that ‘spatialised’ time and image through the fusion of the formal typology of the Panorama together with the cinematic moving-image, creating a new kind of image technology. The incorporation of the space enclosed by the panorama’s drum, into the conception and execution of the cinematic event, reveals an interesting conceptual paradox. Space and time infinitely and autonomously repeat upon each other as the linear trajectory of the singular cinematic shot is interrupted by a ‘time schism’ on the surface of the panorama. This paper explores what this conceptual paradox means to the evolution of emerging image-technologies and how Shaw’s ‘mixed-reality’ installation reveals a wholly new image typology that presents techniques and concepts though which to record, interrogate, and represent time and space in Architecture

Using a multimodal immersive environment to investigate perceptions in augmented virtual reality systems

The Collaborative-Research Augmented Immersive Virtual Environment Laboratory at Rensselaer is a state-of-the-art space that offers users the capabilities of multimodality and immersion. Realistic and abstract sets of data can be explored in a variety of ways, even in large group settings. This paper discusses the motivations of the immersive experience and the advantages over smaller scale and single-modality expressions of data. One experiment focuss on the influence of immersion on perceptions of architectural renderings. Its findings suggest disparities between participants’ judgment when viewing either two-dimensional printouts or the immersive CRAIVE-Lab screen. The advantages of multimodality are discussed in an experiment concerning abstract data exploration. Various auditory cues for aiding in visual data extraction were tested for their affects on participants’ speed and accuracy of information extraction. Finally, artificially generated auralizations are paired with recreations of realistic spaces to analyze the influences of immersive visuals on the perceptions of sound fields. One utilized method for creating these sound fields is a geometric ray-tracing model, which calculates the auditory streams of each individual loudspeaker in the lab to create a cohesive sound field representation of the visual space

Digital Dialectic: Forging New Paths of Inquiry in the Humanities

Digital Dialectic empowers humanities education with technology that sparks deeper contextual understanding of cultural artifacts and illuminates the multicultural nature of the humanities. Frederico Vigil's fresco, Mundos de Mestizaje, allegorically depicts 3000 years of Hispanic history, focusing on cross-cultural exchange of ideas. NHCC and ARTS Lab will create an interactive software application allowing users to explore the fresco, and through educational information embedded in the imagery, discover the dynamic nature of the humanities and their connection to Hispanidad. The asset will deploy on 2 interactive platforms: a digital dome presentation and a web-based viewer. The immersive dome piece will allow widespread audiences to view the fresco at actual scale and dive into details with high-resolution magnification; it will be distributed nationally and internationally to museums with fulldome theaters. The web-based viewer will allow self-guided exploration of the fresco

Videos in Context for Telecommunication and Spatial Browsing

The research presented in this thesis explores the use of videos embedded in panoramic imagery to transmit spatial and temporal information describing remote environments and their dynamics. Virtual environments (VEs) through which users can explore remote locations are rapidly emerging as a popular medium of presence and remote collaboration. However, capturing visual representation of locations to be used in VEs is usually a tedious process that requires either manual modelling of environments or the employment of specific hardware. Capturing environment dynamics is not straightforward either, and it is usually performed through specific tracking hardware. Similarly, browsing large unstructured video-collections with available tools is difficult, as the abundance of spatial and temporal information makes them hard to comprehend. At the same time, on a spectrum between 3D VEs and 2D images, panoramas lie in between, as they offer the same 2D images accessibility while preserving 3D virtual environments surrounding representation. For this reason, panoramas are an attractive basis for videoconferencing and browsing tools as they can relate several videos temporally and spatially. This research explores methods to acquire, fuse, render and stream data coming from heterogeneous cameras, with the help of panoramic imagery. Three distinct but interrelated questions are addressed. First, the thesis considers how spatially localised video can be used to increase the spatial information transmitted during video mediated communication, and if this improves quality of communication. Second, the research asks whether videos in panoramic context can be used to convey spatial and temporal information of a remote place and the dynamics within, and if this improves users' performance in tasks that require spatio-temporal thinking. Finally, the thesis considers whether there is an impact of display type on reasoning about events within videos in panoramic context. These research questions were investigated over three experiments, covering scenarios common to computer-supported cooperative work and video browsing. To support the investigation, two distinct video+context systems were developed. The first telecommunication experiment compared our videos in context interface with fully-panoramic video and conventional webcam video conferencing in an object placement scenario. The second experiment investigated the impact of videos in panoramic context on quality of spatio-temporal thinking during localization tasks. To support the experiment, a novel interface to video-collection in panoramic context was developed and compared with common video-browsing tools. The final experimental study investigated the impact of display type on reasoning about events. The study explored three adaptations of our video-collection interface to three display types. The overall conclusion is that videos in panoramic context offer a valid solution to spatio-temporal exploration of remote locations. Our approach presents a richer visual representation in terms of space and time than standard tools, showing that providing panoramic contexts to video collections makes spatio-temporal tasks easier. To this end, videos in context are suitable alternative to more difficult, and often expensive solutions. These findings are beneficial to many applications, including teleconferencing, virtual tourism and remote assistance

Preferences for Carsharing-Facilitated Neighbourhoods: a Latent-Class Model

One of the most prominent examples of the emerging sharing economy phenomenon is carsharing. Carsharing provides people short-term access to private vehicles without the higher cost and responsibilities that would come with the ownership of a private car. Carsharing programs provide a fleet of vehicles that are parked in a network of locations which are usually located in urban areas with easy access to other transport modes. According to Shaheen & Cohen (2013) the most prominent carsharing business models include: neighborhood residential; business; government and institutional fleets; transit-based; college and university-based; and personal vehicle sharing (use of privately-owned autos employed in shared-use vehicle services). Among these, neighborhood residential carsharing which focuses on mixed-use, urban and residential neighborhoods, is found to be the most common and profitable business model for providers. Residents who live in the neighborhoods with carsharing facilities do not have to own private cars but can use vehicles easily by subscribing to the carsharing service. To use the shared vehicles, residents only need to prebook the cars online before the trip. Once the preparation is done, residents can freely use vehicles until they annouce they have finished the using. According to Chen & Kockelman (2016), travelers who live in dense neighborhoods and join a carsharing service reduce their energy use and emissions. Moreover, in the same study it is found that the demand for parking infrastructure also decreases when travelers chose to join a carhsaring service instead of using private owned car. Decreased parking demand can result in obselete areas in cities which can be transformed into other land uses such as green facilties or more residential buildings. As modern cities are facing challenges such as traffic jam, air pollution, parking tension, urban land use shortage, carsharing facilities in neighborhoods can provide solutions for better living environments and increased quality of life for residents. In this study, we investigate the willingness of citizens that currently live in urban areas of The Netherlands, to move to a neighborhood with carsharing facilities. For that purpose, a stated choice experiment is designed by considering the attributes of neighborhood residential carsharing: These attributes can becategorized as the specifications of carsharing (carsharing costs, booking time, accessibility to home, parking distance to the destination), specifications of carsharing and housing interacted environment (commuting distance, public transport accessibility, housing location, private parking space, green density, children playing area safety) and specificaitons of houses (house type, ownership, size, price, built time). For data collection, a panel is used which consists of respondents from the Netherlands. In total 623 respondents are gathered for the analysis. In order to segment respondents we applied a latent class analysis. According to the results, the 2-class solution gave the best results. Rho-squared is found to be 0.327. The first class is strongly less willing to move to such neighborhoods; the second class is willing to live in a neighborhood carsharing. The strongly less willing group is mainly influenced by the dwelling characteristics such as size, cost and building year and neighborhood characteristics such as parking availability and commuting distance. Moreover, this group also prefers living in neighborhoods with carsharing facilities if carsharing has no costs (free). The second group who are willing to move in a neighborhood with carsharing facilities is mainly influenced by carsharing and residential attributes. This group prefers carsharing that has no cost with little booking time and 5 min walking distance to home. Moreover, just as the first group, they prefer dwellings with bigger living environments and lower costs. However, the influence of these dwelling preferences on their willingness to live in a neighborhood with a carsharing facility is less than the first group. These results are useful to identify the right socio-demographic groups, residential environments and develop strategies in order to establish casharing facilitated neighborhoods

A deep-dream virtual reality platform for studying altered perceptual phenomenology

Altered states of consciousness, such as psychotic or pharmacologically-induced hallucinations, provide a unique opportunity to examine the mechanisms underlying conscious perception. However, the phenomenological properties of these states are difficult to isolate experimentally from other, more general physiological and cognitive 36 effects of psychoactive substances or psychopathological conditions. Thus, simulating phenomenological aspects of altered states in the absence of these other more general effects provides an important experimental tool for consciousness science and psychiatry. Here we describe such a tool, which we call the Hallucination Machine. It comprises a novel combination of two powerful technologies: deep convolutional neural networks (DCNNs) and panoramic videos of natural scenes, viewed immersively through a head-mounted display (panoramic VR). By doing this, we are able to simulate visual hallucinatory experiences in a biologically plausible and ecologically valid way. Two experiments illustrate potential applications of the Hallucination Machine. First, we show that the system induces visual phenomenology qualitatively similar to classical psychedelics. In a second experiment, we find that simulated hallucinations do not evoke the temporal distortion commonly associated with altered states. Overall, the Hallucination Machine offers a valuable new technique for simulating altered phenomenology without directly altering the underlying neurophysiology

A critical practice-based exploration of interactive panoramas' role in helping to preserve cultural memory

I am enclosing the content of two DVDs which are integral part of the practice-based thesis.The rapid development of digital communication technologies in the 20th and 21st centuries has affected the way researchers look at ways memory – especially cultural memory – can be preserved and enhanced. State-of-the-art communication technologies such as the Internet or immersive environments support participation and interaction and transform memory into ‘prosthetic’ experience, where digital technologies could enable 'implantation' of events that have not actually been experienced. While there is a wealth of research on the preservation of public memory and cultural heritage sites using digital media, more can be explored on how these media can contribute to the cultivation of cultural memory. One of the most interesting phenomena related to this issue is how panoramas, which are immersive and have a well-established tradition in preserving memories, can be enhanced by recent digital technologies and image spaces. The emergence of digital panoramic video cameras and panoramic environments has opened up new opportunities for exploring the role of interactive panoramas not only as a documentary tool for visiting sites but mainly as a more complex technique for telling non-linear interactive narratives through the application of panoramic photography and panoramic videography which, when presented in a wrap-around environment, could enhance recalling. This thesis attempts to explore a way of preserving inspirational environments and memory sites in a way that combines panoramic interactive film and traversing the panoramic environment with viewing the photo-realistic panoramic content rather than computer-generated environment. This research is based on two case studies. The case study of Charles Church in Plymouth represents the topical approach to narrative and focuses on the preservation of the memory of the Blitz in Plymouth and the ruin of Charles Church which stands as a silent reminder of this event. The case study of Charles Causley reflects topographical approach where, through traversing the town of Launceston, viewers learn about Causley’s life and places that provided inspirations for his poems. The thesis explores through practice what can be done and reflects on positive and less positive aspects of preserving cultural memory in these case studies in a critical way. Therefore, the results and recommendations from this thesis can be seen as valuable contribution to the study of intermedia and cultural memory in general

Digital Urban - The Visual City

Nothing in the city is experienced by itself for a city’s perspicacity is the sum of its surroundings. To paraphrase Lynch (1960), at every instant, there is more than we can see and hear. This is the reality of the physical city, and thus in order to replicate the visual experience of the city within digital space, the space itself must convey to the user a sense of place. This is what we term the “Visual City”, a visually recognisable city built out of the digital equivalent of bricks and mortar, polygons, textures, and most importantly data. Recently there has been a revolution in the production and distribution of digital artefacts which represent the visual city. Digital city software that was once in the domain of high powered personal computers, research labs and professional software are now in the domain of the public-at-large through both the web and low-end home computing. These developments have gone hand in hand with the re-emergence of geography and geographic location as a way of tagging information to non-proprietary web-based software such as Google Maps, Google Earth, Microsoft’s Virtual Earth, ESRI’s ArcExplorer, and NASA’s World Wind, amongst others. The move towards ‘digital earths’ for the distribution of geographic information has, without doubt, opened up a widespread demand for the visualization of our environment where the emphasis is now on the third dimension. While the third dimension is central to the development of the digital or visual city, this is not the only way the city can be visualized for a number of emerging tools and ‘mashups’ are enabling visual data to be tagged geographically using a cornucopia of multimedia systems. We explore these social, textual, geographical, and visual technologies throughout this chapter

Omnidirectional underwater surveying and telepresence

Exploratory dives are traditionally the first step for marine scientists to acquire information on a previously unknown area of scientific interest. Manned submersibles have been the platform of choice for such exploration, as they allow a high level of environmental perception by the scientist on-board, and the ability to take informed decisions on what to explore next. However, manned submersibles have extremely high operation costs and provide very limited bottom time. Remotely operated vehicles (ROVs) can partially address these two issues, but have operational and cost constraints that restrict their usage. This paper discusses new capabilities to assist scientists operating lightweight hybrid remotely operated vehicles (HROV) in exploratory missions of mapping and surveying. The new capabilities, under development within the Spanish National project OMNIUS, provide a new layer of autonomy for HROVs by exploring three key concepts: Omni-directional optical sensing for collaborative immersive exploration, Proximity safety awareness and Online mapping during mission time.Peer Reviewe