Media Space: an analysis of spatial practices in planar pictorial media.

The thesis analyses the visual space displayed in pictures, film, television and digital interactive media. The argument is developed that depictions are informed by the objectives of the artefact as much as by any simple visual correspondence to the observed world. The simple concept of ‘realism’ is therefore anatomised and a more pragmatic theory proposed which resolves some of the traditional controversies concerning the relation between depiction and vision. This is then applied to the special problems of digital interactive media. An introductory chapter outlines the topic area and the main argument and provides an initial definition of terms. To provide a foundation for the ensuing arguments, a brief account is given of two existing and contrasted approaches to the notion of space: that of perception science which gives priority to acultural aspects, and that of visual culture which emphasises aspects which are culturally contingent. An existing approach to spatial perception (that of JJ Gibson originating in the 1940s and 50s) is applied to spatial depiction in order to explore the differences between seeing and picturing, and also to emphasise the many different cues for spatial perception beyond those commonly considered (such as binocularity and linear perspective). At this stage a simple framework of depiction is introduced which identifies five components or phases: the objectives of the picture, the idea chosen to embody the objectives, the model (essentially, the visual ‘subject matter’), the characteristics of the view and finally the substantive picture or depiction itself. This framework draws attention to the way in which each of the five phases presents an opportunity for decision-making about representation. The framework is used and refined throughout the thesis. Since pictures are considered in some everyday sense to be ‘realistic’ (otherwise, in terms of this thesis, they would not count as depictions), the nature of realism is considered at some length. The apparently unitary concept is broken down into several different types of realism and it is argued that, like the different spatial cues, each lends itself to particular objectives intended for the artefact. From these several types, two approaches to realism are identified, one prioritising the creation of a true illusion (that the picture is in fact a scene) and the other (of which there are innumerably more examples both across cultures and over historical time) one which evokes aspects of vision without aiming to exactly imitate the optical stimulus of the scene. Various reasons for the latter approach, and the variety of spatial practices to which it leads, are discussed. In addition to analysing traditional pictures, computer graphics images are discussed in conjunction with the claims for realism offered by their authors. In the process, informational and affective aspects of picture-making are distinguished, a distinction which it is argued is useful and too seldom made. Discussion of still pictures identifies the evocation of movement (and other aspects of time) as one of the principal motives for departing from attempts at straightforward optical matching. The discussion proceeds to the subject of film where, perhaps surprisingly now that the depiction of movement is possible, the lack of straightforward imitation of the optical is noteworthy again. This is especially true of the relationship between shots rather than within them; the reasons for this are analysed. This reinforces the argument that the spatial form of the fiction film, like that of other kinds of depiction, arises from its objectives, presenting realism once again as a contingent concept. The separation of depiction into two broad classes – one which aims to negate its own mediation, to seem transparent to what it depicts, and one which presents the fact of depiction ostensively to the viewer – is carried through from still pictures, via film, into a discussion of factual television and finally of digital interactive media. The example of factual television is chosen to emphasise how, despite the similarities between the technologies of film and television, spatial practices within some television genres contrast strongly with those of the mainstream fiction film. By considering historic examples, it is shown that many of the spatial practices now familiar in factual television were gradually expunged from the classical film when the latter became centred on the concerns of narrative fiction. By situating the spaces of interactive media in the context of other kinds of pictorial space, questions are addressed concerning the transferability of spatial usages from traditional media to those which are interactive. During the thesis the spatial practices of still-picture-making, film and television are characterised as ‘mature’ and ‘expressive’ (terms which are defined in the text). By contrast the spatial practices of digital interactive media are seen to be immature and inexpressive. It is argued that this is to some degree inevitable given the context in which interactive media artefacts are made and experienced – the lack of a shared ‘language’ or languages in any new media. Some of the difficult spatial problems which digital interactive media need to overcome are identified, especially where, as is currently normal, interaction is based on the relation between a pointer and visible objects within a depiction. The range of existing practice in digital interactive media is classified in a seven-part taxonomy, which again makes use of the objective-idea-model-view-picture framework, and again draws out the difference between self-concealing approaches to depiction and those which offer awareness of depiction as a significant component of the experience. The analysis indicates promising lines of enquiry for the future and emphasises the need for further innovation. Finally the main arguments are summarised and the thesis concludes with a short discussion of the implications for design arising from the key concepts identified – expressivity and maturity, pragmatism and realism

Media Space: an analysis of spatial practices in planar pictorial media.

The thesis analyses the visual space displayed in pictures, film, television and digital interactive media. The argument is developed that depictions are informed by the objectives of the artefact as much as by any simple visual correspondence to the observed world. The simple concept of ‘realism’ is therefore anatomised and a more pragmatic theory proposed which resolves some of the traditional controversies concerning the relation between depiction and vision. This is then applied to the special problems of digital interactive media. An introductory chapter outlines the topic area and the main argument and provides an initial definition of terms. To provide a foundation for the ensuing arguments, a brief account is given of two existing and contrasted approaches to the notion of space: that of perception science which gives priority to acultural aspects, and that of visual culture which emphasises aspects which are culturally contingent. An existing approach to spatial perception (that of JJ Gibson originating in the 1940s and 50s) is applied to spatial depiction in order to explore the differences between seeing and picturing, and also to emphasise the many different cues for spatial perception beyond those commonly considered (such as binocularity and linear perspective). At this stage a simple framework of depiction is introduced which identifies five components or phases: the objectives of the picture, the idea chosen to embody the objectives, the model (essentially, the visual ‘subject matter’), the characteristics of the view and finally the substantive picture or depiction itself. This framework draws attention to the way in which each of the five phases presents an opportunity for decision-making about representation. The framework is used and refined throughout the thesis. Since pictures are considered in some everyday sense to be ‘realistic’ (otherwise, in terms of this thesis, they would not count as depictions), the nature of realism is considered at some length. The apparently unitary concept is broken down into several different types of realism and it is argued that, like the different spatial cues, each lends itself to particular objectives intended for the artefact. From these several types, two approaches to realism are identified, one prioritising the creation of a true illusion (that the picture is in fact a scene) and the other (of which there are innumerably more examples both across cultures and over historical time) one which evokes aspects of vision without aiming to exactly imitate the optical stimulus of the scene. Various reasons for the latter approach, and the variety of spatial practices to which it leads, are discussed. In addition to analysing traditional pictures, computer graphics images are discussed in conjunction with the claims for realism offered by their authors. In the process, informational and affective aspects of picture-making are distinguished, a distinction which it is argued is useful and too seldom made. Discussion of still pictures identifies the evocation of movement (and other aspects of time) as one of the principal motives for departing from attempts at straightforward optical matching. The discussion proceeds to the subject of film where, perhaps surprisingly now that the depiction of movement is possible, the lack of straightforward imitation of the optical is noteworthy again. This is especially true of the relationship between shots rather than within them; the reasons for this are analysed. This reinforces the argument that the spatial form of the fiction film, like that of other kinds of depiction, arises from its objectives, presenting realism once again as a contingent concept. The separation of depiction into two broad classes – one which aims to negate its own mediation, to seem transparent to what it depicts, and one which presents the fact of depiction ostensively to the viewer – is carried through from still pictures, via film, into a discussion of factual television and finally of digital interactive media. The example of factual television is chosen to emphasise how, despite the similarities between the technologies of film and television, spatial practices within some television genres contrast strongly with those of the mainstream fiction film. By considering historic examples, it is shown that many of the spatial practices now familiar in factual television were gradually expunged from the classical film when the latter became centred on the concerns of narrative fiction. By situating the spaces of interactive media in the context of other kinds of pictorial space, questions are addressed concerning the transferability of spatial usages from traditional media to those which are interactive. During the thesis the spatial practices of still-picture-making, film and television are characterised as ‘mature’ and ‘expressive’ (terms which are defined in the text). By contrast the spatial practices of digital interactive media are seen to be immature and inexpressive. It is argued that this is to some degree inevitable given the context in which interactive media artefacts are made and experienced – the lack of a shared ‘language’ or languages in any new media. Some of the difficult spatial problems which digital interactive media need to overcome are identified, especially where, as is currently normal, interaction is based on the relation between a pointer and visible objects within a depiction. The range of existing practice in digital interactive media is classified in a seven-part taxonomy, which again makes use of the objective-idea-model-view-picture framework, and again draws out the difference between self-concealing approaches to depiction and those which offer awareness of depiction as a significant component of the experience. The analysis indicates promising lines of enquiry for the future and emphasises the need for further innovation. Finally the main arguments are summarised and the thesis concludes with a short discussion of the implications for design arising from the key concepts identified – expressivity and maturity, pragmatism and realism

Perceptual modelling for 2D and 3D

Livrable D1.1 du projet ANR PERSEECe rapport a été réalisé dans le cadre du projet ANR PERSEE (n° ANR-09-BLAN-0170). Exactement il correspond au livrable D1.1 du projet

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Exploring the optical perception of image within glass

Within the contemporary world, 3D film and television imagery is at the cutting edge of visual technology, but for centuries we have been captivated by the creation of visual illusions/allusions that play with our perception of the world, from the auto-stereoscopic barrier methods pioneered in the late 17th century by the French painter G. A. Bois-Clair to the ‘Op’ art movement of the 1960s and, more recently, Patrick Hughes’ ‘reverse perspective’ paintings. By building on these new and old technologies I have extended my own practice, which engages with the 2D image as a 3D allusion/illusion in glass, by examining how this type of image can be created and perceived within glass. I have explored theories of optical perception in connection with the binocular recognition of depth and space, as well as kinetic clues to distance through motion parallax monitoring and assumptions about default linear perspective, light and inference within our personal schemata. - ‘Optical illusion’ is used to mean an instance of a wrong or misinterpreted perception of a sensory experience; the distortion of senses revealing how the brain organises and interprets visual information; an individual’s ability to perceive depth, 3D form and motion. - ‘Allusion’ is used to imply a symbolic or covert reference. My practical research focuses on the perceived creation of the 3D image within glass and explores the notion of glass as a facilitator in working with and challenging the themes of 3D image perception. I have particularly addressed artistic spatial illusionary methods, reverse perspective techniques, auto-stereoscopic image-based systems, parallax stereograms and lenticular print and lens technology. Through building on my previous practice of working with multiple-layered images within cast glass, combined with more complex and scientific optical methods, I have explored the perception of the image by working with new and old 3D technologies in order to produce a body of work which examines this perception within glass. During my research I have developed an original casting process, a vacuum-casting lost wax process for glass, in addition to producing an accurate industry standard lenticular glass lens. This research intends to provide a theoretical basis for new glass working techniques, both within the glass artist’s studio and in the commercial world of print, towards applications within architectural design, installation art and image-based artwork in general. This thesis is therefore a summation of the research that I have undertaken over the past six years and an attempt to give substance to the ideas and references that have preoccupied my own investigations over that period. I have structured the thesis into three themes: perspective; perception; and process but those three elements were never separate from each other and not only do they depend on each other, their purpose is, in some way, to combine in the creation of my finished pieces

Depth aware RCNN

Image object detection networks that depend on region proposal networks (RPN) have achieved state-of-art results. As RPN is trained to share convolutional features with the actual classification layers in the network, features learned by the convolutional backbones may have subtle impact on the RPN. A successful approach comes from RGB-D image object detection, where the convolutional layers learn not just RGB features, but also depth features. In this thesis, we study the problem of simultaneously localizing objects as well as estimating their depth. We propose to use one backbone network for two tasks and show that multi-task learning with shared weights can have reciprocating benefits. Our experiments show that when combined with depth prediction in the network, the object detection branch in our model outperforms Faster-RCNN on the challenging KITTI detection benchmark and the Cityscapes dataset. Likewise, the performance of our depth prediction branch is slightly better compared with methods using the same depth prediction architecture

St. George and the Dragon: Design and production of a cultural heritage museum installation using media archaeology

Media archaeology is a field of media research investigating new media cultures through material manifestations. Although often recognized as an approach to art, its use as an approach to design has not been fully explored. Media archaeology can be valuable because it offers alternative qualities of mediation, as a design palette, to that of prescriptive common media devices. This thesis describes a media archaeological approach towards the design of a cultural heritage media installation, exhibited at Häme Castle between April–December 2017, and produced as a collaboration between the National Museum of Finland (Kansallismuseo) and the Systems of Representation research group in the Department of Media at Aalto University in Finland. The installation displayed a multi-view stereoscopic (3D) digital reconstruction of a medieval sculptural scene of St. George and the Dragon, based on preserved, fragmented medieval sculptures from the museum’s archives. Four stereoscopic video viewers were synchronized to a rotating central physical display, affording visitors an effect of augmented reality, without the need for a mainstream augmented reality implementation. Though the work was time-limited and project-driven, the design approach achieved a well-integrated installation that was sensitive to the aims of an exhibition of sculpture within a cultural heritage museum: artistry, materiality, interpretation. This thesis therefore seeks to argue that media archaeological approaches to design can identify historical ideas that can be remediated into relevancy for new contexts, and, in spite of their historical connotations, foster engaging technological experiences for the contemporary audience, that are sensitive to the aims of an exhibition of cultural heritage