End-to-end 3D video communication over heterogeneous networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Three-dimensional technology, more commonly referred to as 3D technology, has revolutionised many fields including entertainment, medicine, and communications to name a few. In addition to 3D films, games, and sports channels, 3D perception has made tele-medicine a reality. By the year 2015, 30% of the all HD panels at home will be 3D enabled, predicted by consumer electronics manufacturers. Stereoscopic cameras, a comparatively mature technology compared to other 3D systems, are now being used by ordinary citizens to produce 3D content and share at a click of a button just like they do with the 2D counterparts via sites like YouTube. But technical challenges still exist, including with autostereoscopic multiview displays. 3D content requires many complex considerations--including how to represent it, and deciphering what is the best compression format--when considering transmission or storage, because of its increased amount of data. Any decision must be taken in the light of the available bandwidth or storage capacity, quality and user expectations. Free viewpoint navigation also remains partly unsolved. The most pressing issue getting in the way of widespread uptake of consumer 3D systems is the ability to deliver 3D content to heterogeneous consumer displays over the heterogeneous networks. Optimising 3D video communication solutions must consider the entire pipeline, starting with optimisation at the video source to the end display and transmission optimisation. Multi-view offers the most compelling solution for 3D videos with motion parallax and freedom from wearing headgear for 3D video perception. Optimising multi-view video for delivery and display could increase the demand for true 3D in the consumer market. This thesis focuses on an end-to-end quality optimisation in 3D video communication/transmission, offering solutions for optimisation at the compression, transmission, and decoder levels.Brunel University - Isambard Research Scholarshi

Remote Visual Observation of Real Places Through Virtual Reality Headsets

Virtual Reality has always represented a fascinating yet powerful opportunity that has attracted studies and technology developments, especially since the latest release on the market of powerful high-resolution and wide field-of-view VR headsets. While the great potential of such VR systems is common and accepted knowledge, issues remain related to how to design systems and setups capable of fully exploiting the latest hardware advances. The aim of the proposed research is to study and understand how to increase the perceived level of realism and sense of presence when remotely observing real places through VR headset displays. Hence, to produce a set of guidelines that give directions to system designers about how to optimize the display-camera setup to enhance performance, focusing on remote visual observation of real places. The outcome of this investigation represents unique knowledge that is believed to be very beneficial for better VR headset designs towards improved remote observation systems. To achieve the proposed goal, this thesis presents a thorough investigation of existing literature and previous researches, which is carried out systematically to identify the most important factors ruling realism, depth perception, comfort, and sense of presence in VR headset observation. Once identified, these factors are further discussed and assessed through a series of experiments and usability studies, based on a predefined set of research questions. More specifically, the role of familiarity with the observed place, the role of the environment characteristics shown to the viewer, and the role of the display used for the remote observation of the virtual environment are further investigated. To gain more insights, two usability studies are proposed with the aim of defining guidelines and best practices. The main outcomes from the two studies demonstrate that test users can experience an enhanced realistic observation when natural features, higher resolution displays, natural illumination, and high image contrast are used in Mobile VR. In terms of comfort, simple scene layouts and relaxing environments are considered ideal to reduce visual fatigue and eye strain. Furthermore, sense of presence increases when observed environments induce strong emotions, and depth perception improves in VR when several monocular cues such as lights and shadows are combined with binocular depth cues. Based on these results, this investigation then presents a focused evaluation on the outcomes and introduces an innovative eye-adapted High Dynamic Range (HDR) approach, which the author believes to be of great improvement in the context of remote observation when combined with eye-tracked VR headsets. Within this purpose, a third user study is proposed to compare static HDR and eye-adapted HDR observation in VR, to assess that the latter can improve realism, depth perception, sense of presence, and in certain cases even comfort. Results from this last study confirmed the author expectations, proving that eye-adapted HDR and eye tracking should be used to achieve best visual performances for remote observation in modern VR systems

Visual discomfort whilst viewing 3D stereoscopic stimuli

3D stereoscopic technology intensifies and heightens the viewer s experience by adding an extra dimension to the viewing of visual content. However, with expansion of this technology to the commercial market concerns have been expressed about the potential negative effects on the visual system, producing viewer discomfort. The visual stimulus provided by a 3D stereoscopic display differs from that of the real world, and so it is important to understand whether these differences may pose a health hazard. The aim of this thesis is to investigate the effect of 3D stereoscopic stimulation on visual discomfort. To that end, four experimental studies were conducted. In the first study two hypotheses were tested. The first hypothesis was that the viewing of 3D stereoscopic stimuli, which are located geometrically beyond the screen on which the images are displayed, would induce adaptation changes in the resting position of the eyes (exophoric heterophoria changes). The second hypothesis was that participants whose heterophoria changed as a consequence of adaptation during the viewing of the stereoscopic stimuli would experience less visual discomfort than those people whose heterophoria did not adapt. In the experiment an increase of visual discomfort change in the 3D condition in comparison with the 2D condition was found. Also, there were statistically significant changes in heterophoria under 3D conditions as compared with 2D conditions. However, there was appreciable variability in the magnitude of this adaptation among individuals, and no correlation between the amount of heterophoria change and visual discomfort change was observed. In the second experiment the two hypotheses tested were based on the vergence-accommodation mismatch theory, and the visual-vestibular mismatch theory. The vergence-accommodation mismatch theory predicts that a greater mismatch between the stimuli to accommodation and to vergence would produce greater symptoms in visual discomfort when viewing in 3D conditions than when viewing in 2D conditions. An increase of visual discomfort change in the 3D condition in comparison with the 2D condition was indeed found; however the magnitude of visual discomfort reported did not correlate with the mismatch present during the watching of 3D stereoscopic stimuli. The visual-vestibular mismatch theory predicts that viewing a stimulus stereoscopically will produce a greater sense of vection than viewing it in 2D. This will increase the conflict between the signals from the visual and vestibular systems, producing greater VIMS (Visually- Induced Motion Sickness) symptoms. Participants did indeed report an increase in motion sickness symptoms in the 3D condition. Furthermore, participants with closer seating positions reported more VIMS than participants sitting farther away whilst viewing 3D stimuli. This suggests that the amount of visual field stimulated during 3D presentation affects VIMS, and is an important factor in terms of viewing comfort. In the study more younger viewers (21 to 39 years old) than older viewers (40 years old and older) reported a greater change in visual discomfort during the 3D condition than the 2D condition. This suggests that the visual system s response to a stimulus, rather than the stimulus itself, is a reason for discomfort. No influence of gender on viewing comfort was found. In the next experiment participants fusion capability, as measured by their fusional reserves, was examined to determine whether this component has an impact on reported discomfort during the watching of movies in the 3D condition versus the 2D condition. It was hypothesised that participants with limited fusional range would experience more visual discomfort than participants with a wide fusion range. The hypothesis was confirmed but only in the case of convergent and not divergent eye movement. This observation illustrates that participants capability to convergence has a significant impact on visual comfort. The aim of the last experiment was to examine responses of the accommodation system to changes in 3D stimulus position and to determine whether discrepancies in these responses (i.e. accommodation overshoot, accommodation undershoot) could account for visual discomfort experienced during 3D stereoscopic viewing. It was found that accommodation discrepancy was larger for perceived forwards movement than for perceived backwards movement. The discrepancy was slightly higher in the group susceptible to visual discomfort than in the group not susceptible to visual discomfort, but this difference was not statistically significant. When considering the research findings as a whole it was apparent that not all participants experienced more discomfort whilst watching 3D stereoscopic stimuli than whilst watching 2D stimuli. More visual discomfort in the 3D condition than in the 2D condition was reported by 35% of the participants, whilst 24% of the participants reported more headaches and 17% of the participants reported more VIMS. The research indicates that multiple causative factors have an impact on reported symptoms. The analysis of the data suggests that discomfort experienced by people during 3D stereoscopic stimulation may reveal binocular vision problems. This observation suggests that 3D technology could be used as a screening method to diagnose un-treated binocular vision disorder. Additionally, this work shows that 3D stereoscopic technology can be easily adopted to binocular vision measurement. The conclusion of this thesis is that many people do not suffer adverse symptoms when viewing 3D stereoscopic displays, but that if adverse symptoms are present they can be caused either by the conflict in the stimulus, or by the heightened experience of self-motion which leads to Visually-Induced Motion Sickness (VIMS)

Holoscopic 3D imaging and display technology: Camera/ processing/ display

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonHoloscopic 3D imaging “Integral imaging” was first proposed by Lippmann in 1908. It has become an attractive technique for creating full colour 3D scene that exists in space. It promotes a single camera aperture for recording spatial information of a real scene and it uses a regularly spaced microlens arrays to simulate the principle of Fly’s eye technique, which creates physical duplicates of light field “true 3D-imaging technique”. While stereoscopic and multiview 3D imaging systems which simulate human eye technique are widely available in the commercial market, holoscopic 3D imaging technology is still in the research phase. The aim of this research is to investigate spatial resolution of holoscopic 3D imaging and display technology, which includes holoscopic 3D camera, processing and display. Smart microlens array architecture is proposed that doubles spatial resolution of holoscopic 3D camera horizontally by trading horizontal and vertical resolutions. In particular, it overcomes unbalanced pixel aspect ratio of unidirectional holoscopic 3D images. In addition, omnidirectional holoscopic 3D computer graphics rendering techniques are proposed that simplify the rendering complexity and facilitate holoscopic 3D content generation. Holoscopic 3D image stitching algorithm is proposed that widens overall viewing angle of holoscopic 3D camera aperture and pre-processing of holoscopic 3D image filters are proposed for spatial data alignment and 3D image data processing. In addition, Dynamic hyperlinker tool is developed that offers interactive holoscopic 3D video content search-ability and browse-ability. Novel pixel mapping techniques are proposed that improves spatial resolution and visual definition in space. For instance, 4D-DSPM enhances 3D pixels per inch from 44 3D-PPIs to 176 3D-PPIs horizontally and achieves spatial resolution of 1365 × 384 3D-Pixels whereas the traditional spatial resolution is 341 × 1536 3D-Pixels. In addition distributed pixel mapping is proposed that improves quality of holoscopic 3D scene in space by creating RGB-colour channel elemental images

Establishing user immersion of consumer virtual reality products through qualitative measurement

New consumer technologies such as the Oculus Rift and other head-mounted displays (HMDs) in the field of virtual reality (VR) have introduced another method of experiencing immersive content. The effectiveness of these new mediums can be compared to more traditional products that are currently in use; such as televisions, desktop monitors, and variations which include 3D display capabilities. This raises the question of whether there is a notable difference between the immersive experiences each medium can offer. This study hypothesized that a significant difference in immersion could be identified between consumer devices. Also, using qualitative analysis, an attempt to identify what consumers believe comprises immersion could be devised. Data from 30 participants indicated that the immersive potential of consumer devices can be differentiated and compared. In addition the understanding of the term “immersion” is not clearly understood or shared by consumers.Master of Science (MSc) in Computational Scienc

Modern lithographic techniques applied to stereographic imaging

The main aim of the research has been to produce and evaluate a high-quality diffusion screen to display projected film and television images. The screens have also been found to effectively de-pixelate LCD arrays viewed at a magnification of approximately 4x. The production process relies on the formation of localized refractive index gradients in a photopolymer. The photopolymer, specially formulated and supplied by Du Pont, is exposed to actinic light through a precision contact mask to initiate polymerization within the exposed areas. As polymerization proceeds, a monomer concentration gradient exists between the exposed and unexposed regions allowing the monomer molecules to diffuse. Since the longer polymer chains do not diffuse as readily, the molecular concentration of the material, which is related to its refractive index, is then no longer uniform. The generation of this refractive index profile can, to some extent, be controlled by careful exposure of the photopolymer through the correct mask so that the resulting diffusion screen can be tailored to suit specific viewing requirements. [Continues.

A shuttle and space station manipulator system for assembly, docking, maintenance, cargo handling and spacecraft retrieval (preliminary design). Volume 3: Concept analysis. Part 1: Technical

Information backing up the key features of the manipulator system concept and detailed technical information on the subsystems are presented. Space station assembly and shuttle cargo handling tasks are emphasized in the concept analysis because they involve shuttle berthing, transferring the manipulator boom between shuttle and station, station assembly, and cargo handling. Emphasis is also placed on maximizing commonality in the system areas of manipulator booms, general purpose end effectors, control and display, data processing, telemetry, dedicated computers, and control station design

Stereoskooppiset vääristymät taiteellisena menetelmänä

Stereoscopy is a technique for generating a perception of depth using a pair of two-dimensional pictures. Digital technologies of the 21st century have greatly enhanced the possibilities for operating in the field of optical phenomena. Most of the technological problems restraining the acceptance of stereoscopic techniques in the 20th century have now been solved. This allows inexpensive experiments of unnatural visual conditions to be carried out. The goal of this study is to generate novel experiences through an exploration of ways of visualizing spatial dimensions and to provide new practices for observing shapes and objects in an artistic context. Purposefully challenging our conventions of perception and abandoning the desire for comfort defines the starting point for the structure of the art object created in this study. The careful construction of abnormal stereoscopic conditions can produce interesting experiences through a strong response from the physical and the cognitive structures of a human sensory system. With the art object of this study, the main idea is to construct a virtual environment simulating the process of eyes moving freely from the constraints of a human body. This animation is presented in the format of a stereoscopic video installation. Though the animation is distinctively a product of the digital age, and would most likely have been impossible to accurately produce without the aid of a computer, the emphasis of the whole study is clearly on the processes taking place within the human body. The most essential part of pragmatistic theory of art applied in this context is the differentiation of art objects from works of art – the former having physical existence and functioning as an initiator for the latter, which is a specific type of experience and not a physical object of any kind. The experience of the brain’s struggle to make sense of unusual visual conditions brought upon them by the art object constitutes the work of art in this study. However, neither pragmatism nor stereoscopic principles can function as a suitable concept for locating an artistic production in the field of visual arts. For this purpose a relationship between the artistic process of this study and optical art movement is established. Also, connections to new media art and minimalist art are explored. In conclusion, it is stated that, as demonstrated in this study, artistic practices applying the specific distortions of the stereoscopic effect have the potential for generating and directing a specific type of experience.Stereoskopia on tekniikka syvyysvaikutelman luomiseen kaksiulotteisen kuvaparin avulla. 2000-luvulla digitaaliset teknologiat ovat huomattavasti parantaneet mahdollisuuksia työskennellä optisten ilmiöiden kentällä. Suurin osa teknologisista ongelmista, jotka rajoittivat stereoskooppisten tekniikoiden yleistymistä 1900-luvulla, on nyt ratkaistu. Tämä mahdollistaa kustannustehokkaat kokeilut epäluonnollisilla visuaalisilla tiloilla. Tavoitteena tässä tutkimuksessa on luoda uusia kokemuksia tutkimalla tapoja tehdä tilallisten ulottuvuuksien visualisointeja sekä tarjota uusia käytäntöjä muotojen ja esineiden havainnointiin taiteellisessa kontekstissa. Havaitsemisen käytäntöjen tarkoituksellinen haastaminen sekä mukavuuden tavoittelusta luopuminen määrittävät lähtökohdat tässä tutkimuksessa valmistetun taide-esineen rakenteille. Normaalista poikkeavien stereoskooppisten tilojen huolellinen rakentaminen voi tuottaa kiinnostavia kokemuksia ihmisen sensoristen sekä kognitiivisten järjestelmien voimakkaisiin reaktioihin pohjautuen. Keskeisenä ideana tämän tutkimuksen yhteydessä valmistetussa taide-esineessä on luoda virtuaalinen ympäristö, joka simuloi ihmisruumiin rajoitteista vapautettujen silmien liikkumista. Tämä animaatio esitetään stereoskooppisena videoinstallaationa. Vaikka tutkimuksen animaatio on selkeästi digitaalisen aikakauden tuotos, ja sitä olisi todennäköisesti ollut mahdoton toteuttaa tarkasti ilman tietokonetta, kokonaisuudessaan tämä tutkimus painottaa ihmiskehon sisällä tapahtuvia prosesseja. Tässä kontekstissa käytetyn pragmatistisen taiteenteorian keskeisiä ajatuksia on taide-esineiden ja taideteosten erottelu toisistaan. Edellisellä on fyysinen olemassaolo ja se toimii alkuunpanijana jälkimmäiselle, joka puolestaan on tietyn tyyppinen kokemus eikä fysikaalinen esine. Kokemus aivojen käymästä kamppailusta niiden yrittäessä tehdä selkoa taide-esineen tuottamasta epätavallisesta visuaalisesta tilasta muodostaa taideteoksen tässä tutkimuksessa. On kuitenkin huomioitava, ettei pragmatismi eivätkä stereoskooppiset periaatteet voi toimia konseptina taiteellisen produktion paikantamiseen visuaalisen taiteen kentällä. Tätä tarkoitusta varten osoitetaan suhde tämän tutkimuksen ja optisen taiteen välillä. Lisäksi tarkastellaan yhteyksiä uusmediataiteeseen ja minimalistiseen taiteeseen. Yhteenvetona todetaan tämän tutkimuksen osoittavan, että tiettyjen stereoskooppisten vääristymien käytössä taiteellisena menetelmänä on potentiaalia synnyttää ja ohjata erityisiä kokemisen tapoja

Interchanging Interactive 3-d Graphics for Astronomy

We demonstrate how interactive, three-dimensional (3-d) scientific visualizations can be efficiently interchanged between a variety of mediums. Through the use of an appropriate interchange format, and a unified interaction interface, we minimize the effort to produce visualizations appropriate for undertaking knowledge discovery at the astronomer's desktop, as part of conference presentations, in digital publications or as Web content. We use examples from cosmological visualization to address some of the issues of interchange, and to describe our approach to adapting S2PLOT desktop visualizations to the Web. Supporting demonstrations are available at http://astronomy.swin.edu.au/s2plot/interchange/Comment: 10 pages, 7 figures, submitted to Publications of the Astronomical Society of Australia. v2. Revised title, revised figure 1, fixed typos, minor additions to future work sectio