Perceptually driven stereoscopic camera control in 3D virtual environments

Ankara : The Department of Computer Engineering and the Graduate School of Engineering and Science of Bilkent University, 2013.Thesis (Master's) -- Bilkent University, 2013.Includes bibliographical references leaves 56-59.Depth notion and how to perceive depth have long been studied in the eld of psychology, physiology, and even art. Human visual perception enables to perceive spatial layout of the outside world by using visual depth cues. Binocular disparity among these depth cues, is based on the separation between two di erent views that are observed by two eyes. Disparity concept constitutes the base of the construction of the stereoscopic vision. Emerging technologies try to replicate binocular disparity principles in order to provide 3D illusion and stereoscopic vision. However, the complexity of applying the underlying principles of 3D perception, confronted researchers the problem of wrongly produced stereoscopic contents. It is still a great challenge to give realistic but also comfortable 3D experience. In this work, we present a camera control mechanism: a novel approach for disparity control and a model for path generation. We try to address the challenges of stereoscopic 3D production by presenting comfortable viewing experience to users. Therefore, our disparity system approaches the accommodation/convergence con- ict problem, which is the most known issue that causes visual fatigue in stereo systems, by taking objects' importance into consideration. Stereo camera parameters are calculated automatically with an optimization process. In the second part of our control mechanism, the camera path is constructed for a given 3D environment and scene elements. Moving around important regions of objects is a desired scene exploration task. In this respect, object saliencies are used for viewpoint selection around scene elements. Path structure is generated by using linked B ezier curves which assures to pass through pre-determined viewpoints. Though there is considerable amount of research found in the eld of stereo creation, we believe that approaching this problem from scene content aspect provides a uniquely promising experience. We validate our assumption with user studies in which our method and existing two other disparity control models are compared. The study results show that our method shows superior results in quality, depth, and comfort.Kevinç, Elif BengüM.S

A Cooperative algorithm for stereo disparity computation.

by Or Siu Hang.Thesis (M.Phil.)--Chinese University of Hong Kong, 1991.Bibliography: leaves [102]-[105].Acknowledgements --- p.VChapter Chapter 1 --- IntroductionChapter 1.1 --- The problem --- p.1Chapter 1.1.1 --- The correspondence problem --- p.5Chapter 1.1.2 --- The problem of surface reconstruction --- p.6Chapter 1.2 --- Our goal --- p.8Chapter 1.3 --- Previous works --- p.8Chapter 1.3.1 --- Constraints on matching --- p.10Chapter 1.3.2 --- Interpolation of disparity surfaces --- p.12Chapter Chapter 2 --- Preprocessing of imagesChapter 2.1 --- Which operator to use --- p.14Chapter 2.2 --- Directional zero-crossing --- p.14Chapter 2.3 --- Laplacian of Gaussian --- p.16Chapter 2.3.1 --- Theoretical background of the Laplacian of Gaussian --- p.18Chapter 2.3.2 --- Implementation of the operator --- p.21Chapter Chapter 3 --- Disparity Layers GenerationChapter 3.1 --- Geometrical constraint --- p.23Chapter 3.2 --- Basic idea of disparity layer --- p.26Chapter 3.3 --- Consideration in matching --- p.28Chapter 3.4 --- effect of vertical misalignment of sensor --- p.37Chapter 3.5 --- Final approach --- p.39Chapter Chapter 4 --- Disparity combinationChapter 4.1 --- Ambiguous match from different layers --- p.52Chapter 4.2 --- Our approach --- p.54Chapter Chapter 5 --- Generation of dense disparity mapChapter 5.1 --- Introduction --- p.58Chapter 5.2 --- Cooperative computation --- p.58Chapter 5.2.1 --- Formulation of oscillation algorithm --- p.59Chapter 5.3 --- Interpolation by Gradient descent method --- p.69Chapter 5.3.1 --- Formulation of constraints --- p.70Chapter 5.3.2 --- Gradient projection interpolation algorithm --- p.72Chapter 5.3.3 --- Implementation of the algorithm --- p.78Chapter Chapter 6 --- Conclusion --- p.89ReferenceAppendix (Dynamical behavior of the cooperative algorithm

Coherent and Holographic Imaging Methods for Immersive Near-Eye Displays

Lähinäytöt on suunniteltu tarjoamaan realistisia kolmiulotteisia katselukokemuksia, joille on merkittävää tarvetta esimerkiksi työkoneiden etäkäytössä ja 3D-suunnittelussa. Nykyaikaiset lähinäytöt tuottavat kuitenkin edelleen ristiriitaisia visuaalisia vihjeitä, jotka heikentävät immersiivistä kokemusta ja haittaavat niiden miellyttävää käyttöä. Merkittävänä ratkaisuvaihtoehtona pidetään koherentin valon, kuten laservalon, käyttöä näytön valaistukseen, millä voidaan korjata nykyisten lähinäyttöjen puutteita. Erityisesti koherentti valaistus mahdollistaa holografisen kuvantamisen, jota käyttävät holografiset näytöt voivat tarkasti jäljitellä kolmiulotteisten mallien todellisia valoaaltoja. Koherentin valon käyttäminen näyttöjen valaisemiseen aiheuttaa kuitenkin huomiota vaativaa korkean kontrastin häiriötä pilkkukuvioiden muodossa. Lisäksi holografisten näyttöjen laskentamenetelmät ovat laskennallisesti vaativia ja asettavat uusia haasteita analyysin, pilkkuhäiriön ja valon mallintamisen suhteen. Tässä väitöskirjassa tutkitaan laskennallisia menetelmiä lähinäytöille koherentissa kuvantamisjärjestelmässä käyttäen signaalinkäsittelyä, koneoppimista sekä geometrista (säde) ja fysikaalista (aalto) optiikan mallintamista. Työn ensimmäisessä osassa keskitytään holografisten kuvantamismuotojen analysointiin sekä kehitetään hologrammien laskennallisia menetelmiä. Holografian korkeiden laskentavaatimusten ratkaisemiseksi otamme käyttöön holografiset stereogrammit holografisen datan likimääräisenä esitysmuotona. Tarkastelemme kyseisen esitysmuodon visuaalista oikeellisuutta kehittämällä analyysikehyksen holografisen stereogrammin tarjoamien visuaalisten vihjeiden tarkkuudelle akkommodaatiota varten suhteessa sen suunnitteluparametreihin. Lisäksi ehdotamme signaalinkäsittelyratkaisua pilkkuhäiriön vähentämiseksi, ratkaistaksemme nykyisten menetelmien valon mallintamiseen liittyvät visuaalisia artefakteja aiheuttavat ongelmat. Kehitämme myös uudenlaisen holografisen kuvantamismenetelmän, jolla voidaan mallintaa tarkasti valon käyttäytymistä haastavissa olosuhteissa, kuten peiliheijastuksissa. Väitöskirjan toisessa osassa lähestytään koherentin näyttökuvantamisen laskennallista taakkaa koneoppimisen avulla. Kehitämme koherentin akkommodaatioinvariantin lähinäytön suunnittelukehyksen, jossa optimoidaan yhtäaikaisesti näytön staattista optiikka ja näytön kuvan esikäsittelyverkkoa. Lopuksi nopeutamme ehdottamaamme uutta holografista kuvantamismenetelmää koneoppimisen avulla reaaliaikaisia sovelluksia varten. Kyseiseen ratkaisuun sisältyy myös tehokkaan menettelyn kehittäminen funktionaalisten satunnais-3D-ympäristöjen tuottamiseksi. Kehittämämme menetelmä mahdollistaa suurten synteettisten moninäkökulmaisten kuvien datasettien tuottamisen, joilla voidaan kouluttaa sopivia neuroverkkoja mallintamaan holografista kuvantamismenetelmäämme reaaliajassa. Kaiken kaikkiaan tässä työssä kehitettyjen menetelmien osoitetaan olevan erittäin kilpailukykyisiä uusimpien koherentin valon lähinäyttöjen laskentamenetelmien kanssa. Työn tuloksena nähdään kaksi vaihtoehtoista lähestymistapaa ristiriitaisten visuaalisten vihjeiden aiheuttamien nykyisten lähinäyttöongelmien ratkaisemiseksi joko staattisella tai dynaamisella optiikalla ja reaaliaikaiseen käyttöön soveltuvilla laskentamenetelmillä. Esitetyt tulokset ovat näin ollen tärkeitä seuraavan sukupolven immersiivisille lähinäytöille.Near-eye displays have been designed to provide realistic 3D viewing experience, strongly demanded in applications, such as remote machine operation, entertainment, and 3D design. However, contemporary near-eye displays still generate conflicting visual cues which degrade the immersive experience and hinders their comfortable use. Approaches using coherent, e.g., laser light for display illumination have been considered prominent for tackling the current near-eye display deficiencies. Coherent illumination enables holographic imaging whereas holographic displays are expected to accurately recreate the true light waves of a desired 3D scene. However, the use of coherent light for driving displays introduces additional high contrast noise in the form of speckle patterns, which has to be taken care of. Furthermore, imaging methods for holographic displays are computationally demanding and impose new challenges in analysis, speckle noise and light modelling. This thesis examines computational methods for near-eye displays in the coherent imaging regime using signal processing, machine learning, and geometrical (ray) and physical (wave) optics modeling. In the first part of the thesis, we concentrate on analysis of holographic imaging modalities and develop corresponding computational methods. To tackle the high computational demands of holography, we adopt holographic stereograms as an approximative holographic data representation. We address the visual correctness of such representation by developing a framework for analyzing the accuracy of accommodation visual cues provided by a holographic stereogram in relation to its design parameters. Additionally, we propose a signal processing solution for speckle noise reduction to overcome existing issues in light modelling causing visual artefacts. We also develop a novel holographic imaging method to accurately model lighting effects in challenging conditions, such as mirror reflections. In the second part of the thesis, we approach the computational complexity aspects of coherent display imaging through deep learning. We develop a coherent accommodation-invariant near-eye display framework to jointly optimize static display optics and a display image pre-processing network. Finally, we accelerate the corresponding novel holographic imaging method via deep learning aimed at real-time applications. This includes developing an efficient procedure for generating functional random 3D scenes for forming a large synthetic data set of multiperspective images, and training a neural network to approximate the holographic imaging method under the real-time processing constraints. Altogether, the methods developed in this thesis are shown to be highly competitive with the state-of-the-art computational methods for coherent-light near-eye displays. The results of the work demonstrate two alternative approaches for resolving the existing near-eye display problems of conflicting visual cues using either static or dynamic optics and computational methods suitable for real-time use. The presented results are therefore instrumental for the next-generation immersive near-eye displays

Stereoscopic high dynamic range imaging

Two modern technologies show promise to dramatically increase immersion in virtual environments. Stereoscopic imaging captures two images representing the views of both eyes and allows for better depth perception. High dynamic range (HDR) imaging accurately represents real world lighting as opposed to traditional low dynamic range (LDR) imaging. HDR provides a better contrast and more natural looking scenes. The combination of the two technologies in order to gain advantages of both has been, until now, mostly unexplored due to the current limitations in the imaging pipeline. This thesis reviews both fields, proposes stereoscopic high dynamic range (SHDR) imaging pipeline outlining the challenges that need to be resolved to enable SHDR and focuses on capture and compression aspects of that pipeline. The problems of capturing SHDR images that would potentially require two HDR cameras and introduce ghosting, are mitigated by capturing an HDR and LDR pair and using it to generate SHDR images. A detailed user study compared four different methods of generating SHDR images. Results demonstrated that one of the methods may produce images perceptually indistinguishable from the ground truth. Insights obtained while developing static image operators guided the design of SHDR video techniques. Three methods for generating SHDR video from an HDR-LDR video pair are proposed and compared to the ground truth SHDR videos. Results showed little overall error and identified a method with the least error. Once captured, SHDR content needs to be efficiently compressed. Five SHDR compression methods that are backward compatible are presented. The proposed methods can encode SHDR content to little more than that of a traditional single LDR image (18% larger for one method) and the backward compatibility property encourages early adoption of the format. The work presented in this thesis has introduced and advanced capture and compression methods for the adoption of SHDR imaging. In general, this research paves the way for a novel field of SHDR imaging which should lead to improved and more realistic representation of captured scenes

Absorbing new subjects: holography as an analog of photography

I discuss the early history of holography and explore how perceptions, applications, and forecasts of the subject were shaped by prior experience. I focus on the work of Dennis Gabor (1900–1979) in England,Yury N. Denisyuk (b. 1924) in the Soviet Union, and Emmett N. Leith (1927–2005) and Juris Upatnieks (b. 1936) in the United States. I show that the evolution of holography was simultaneously promoted and constrained by its identification as an analog of photography, an association that influenced its assessment by successive audiences of practitioners, entrepreneurs, and consumers. One consequence is that holography can be seen as an example of a modern technical subject that has been shaped by cultural influences more powerfully than generally appreciated. Conversely, the understanding of this new science and technology in terms of an older one helps to explain why the cultural effects of holography have been more muted than anticipated by forecasters between the 1960s and 1990s

Proceedings of the Augmented VIsual Display (AVID) Research Workshop

The papers, abstracts, and presentations were presented at a three day workshop focused on sensor modeling and simulation, and image enhancement, processing, and fusion. The technical sessions emphasized how sensor technology can be used to create visual imagery adequate for aircraft control and operations. Participants from industry, government, and academic laboratories contributed to panels on Sensor Systems, Sensor Modeling, Sensor Fusion, Image Processing (Computer and Human Vision), and Image Evaluation and Metrics

Exploring how object shape and binocular vision interact to make or break camouflage

Depth perception is a major component of 3D vision. There are many cues to depth; one particularly sensitive aspect is the vivid perception of depth created from having eyes with overlapping visual fields (binocular vision). As the eyes are located at different points in space, they see different views of the scene – these slight differences (called binocular disparity) can be used to obtain depth information. However, extracting depth from disparity requires complex visual processing. So why use binocular vision? Julesz (1971) proposed an explanation – camouflaged animals can fool the perception of some cues to 3D shape, but camouflage is ineffective against binocular vision. We would expect that animals with binocular vision could see the 3D shape of animals, despite their camouflage. Whilst commonly accepted, this hypothesis has not been tested in detail. In this thesis, we present experiments designed to establish how depth from binocular vision interacts with camouflage and object shape. Two main questions were addressed: First, we explored how the visual system represented depth information about 3D objects from binocular disparity. Objects with smooth depth edges (hill-shaped) were perceived with less depth than sharper edged objects. A computational model that segregated the object, then averaged the disparity over the segregated region emulated human performance. Finally, we found that disparity and luminance cues interacted to alter perceived depth. Secondly, we investigated if binocular vision could overcome camouflage. We found that camouflaged objects defined by luminance were detected faster when also defined by depth from disparity, thus reduces the effect of camouflage. Smooth objects were detected slower than sharp objects: an effect that was replicated in the real world, suggesting a camouflage technique to counter binocular vision. In summary, binocular vision is useful because it can detect camouflaged objects. However, smoother shapes take longer to spot, forming binocular (or stereoscopic) camouflage

Methodologies for distributed and higher dimensional geographic information

PhD ThesisIn today's digital era, cartography has changed its role, from that of a pure visual model of the Earth's surface, to an interface to other spatial and aspatial information. Along with this, representationa nd manipulation of graphical information in three-dimensional space is required for many applications. Problems and difficulties must be overcome in order to facilitate the move to three-dimensional models, multimedia, and distributed data. Can accurate measurements, at sufficient resolution, and using affordable resources be obtained? Will application software usefully process, in all aspects, models of the real world, sounds, and videos? Combined with this, the workplace is becoming distributed, requiring applications and data that can be used across the globe as easily as in the office. A distributed, three-dimensional, GIS is required with all the procedural and recording functionality of current two-dimensional systems. Such a GIS would maintain a model, typically comprised of solids of individual buildings, roads, utilities etc. with both external and internal detail, represented on a suitable digital terrain model. This research examines virtual reality software as part of an answer. Alternatively, can technologies such as HTML, VRML, and scripting, along with object-orientation and open systems, allow for the display and interrogation of networked data sets? The particular application of this technology, considered during this research, is the need for accurate reconstruction of historical urban monuments. The construction, manipulation, and exploration of these models is often referred to as virtual heritage. This research constructs an innovative and resource effective methodology, the Phoenix algorithm, which requires only a single image for creating three-dimensional models of buildings at large scale. The development of this algorithm is discussed and the results obtained from it are compared with those obtained using traditional three-dimensional capture techniques. Furthermore, possible solutions to the earlier questions are given and discussed

Stereoscopic 3D Technologies for Accurate Depth Tasks: A Theoretical and Empirical Study

In the last decade an increasing number of application fields, including medicine, geoscience and bio-chemistry, have expressed a need to visualise and interact with data that are inherently three-dimensional. Stereoscopic 3D technologies can offer a valid support for these operations thanks to the enhanced depth representation they can provide. However, there is still little understanding of how such technologies can be used effectively to support the performance of visual tasks based on accurate depth judgements. Existing studies do not provide a sound and complete explanation of the impact of different visual and technical factors on depth perception in stereoscopic 3D environments. This thesis presents a new interpretative and contextualised analysis of the vision science literature to clarify the role of di®erent visual cues on human depth perception in such environments. The analysis identifies luminance contrast, spatial frequency, colour, blur, transparency and depth constancies as influential visual factors for depth perception and provides the theoretical foundation for guidelines to support the performance of accurate stereoscopic depth tasks. A novel assessment framework is proposed and used to conduct an empirical study to evaluate the performance of four distinct classes of 3D display technologies. The results suggest that 3D displays are not interchangeable and that the depth representation provided can vary even between displays belonging to the same class. The study also shows that interleaved displays may suffer from a number of aliasing artifacts, which in turn may affect the amount of perceived depth. The outcomes of the analysis of the influential visual factors for depth perception and the empirical comparartive study are used to propose a novel universal 3D cursor prototype suitable to support depth-based tasks in stereoscopic 3D environments. The contribution includes a number of both qualitative and quantitative guidelines that aim to guarantee a correct perception of depth in stereoscopic 3D environments and that should be observed when designing a stereoscopic 3D cursor

Capturing Culture: The Practical Application of Holographic Recording for Artefacts Selected from the Heritage and Museums of the Arabian Peninsula

Recording cultural heritage is one of the most important issues for consideration in the twenty- first century. Safeguarding, protecting and preserving heritage, through effective mechanism, is of crucial importance. Holographic technology has the potential to offer an appropriate solution to solve issues in documenting, cataloguing and replaying the original optical information of the artefact in three-dimensional imaging. This thesis investigates the relationship between art and technology through holograms recorded as part of a practice-based research programme. It questions whether the holographic medium can be used to capture and disseminate information for use in audience interaction, and therefore raise public awareness, by solving the problem of displaying the original artefacts outside the museum context. Using holographic records of such valuable items has the potential to save them from being lost or destroyed, and opens up the prospect of a new form of virtual museum. This research examines the possibility of recording valuable and priceless artefacts using a mobile holographic recording system designed for museums. To this end, historical, traditional and cultural artefacts on display in Saudi heritage museums have been selected. This project involves the recording of ancient Arabian Peninsula cultural heritage, and in particular jewellery artefacts that we perceive as three-dimensional images created, using holographic wavefront information. The research adopts both qualitative and quantitative research methods and critical review of relevant literature on the holographic medium to determine how it might provide an innovative method of engaging museums in Saudi Arabia. The findings of this research offer an original contribution to knowledge and understanding for scholars concerned with conservation of Saudi Arabia’s cultural heritage