How will immersive virtual reality (VR) cognitively affect the audiences who interface with it to interpret Shakespeare performances? Current theories of performance and cognition are based on theatre and film audiences, but VR performances combine features of both media: a disembodied spectral presence, like a theatrical audience; and a flexible range of locations and vantage points, like a film audience. This paper imagines how VR might adapt a sequence from Orson Welles' 1952 film *Othello*. It asks not only what would be lost and gained in the transition, but what cognitive ecology results from a medium that limits our interaction with and our movements within a performed narrative

Michael Ullyot

Humanities Commons

1“Wear your eyes thus”: Toward a Cognitive Ecology of VR ShakespeareMICHAEL ULLYOTTwo men walk along a crenellated wall atop the fort of Cyprus. The first insinuates suspicions into the other’s mind through veiled statements and leading questions: “Did Michael Cassio, when you wooed my lady, | Know of your love?”[footnoteRef:1] Roused by these tactics, the second man grows impatient (“By heaven, I’ll know thy thoughts!”) and then incredulous (“I’ll see before I doubt”).[footnoteRef:2] “Look to your wife,” replies the first, “Wear your eyes thus: not jealous nor secure.”[footnoteRef:3] [1: 	William Shakespeare, Othello, ed. Edward Pechter, New York: Norton, 2004 (3.3.94-95)]  [2: 	3.3.164, 192]  [3: 	3.3.199-200] Orson Welles films this scene in a long tracking shot, following Othello and Iago’s conversation along the ramparts and into the fortress. In this signature shot, “the centrepiece” of Welles’ 1952 film, light and shadow correspond to trust and distrust as Iago seduces Othello with insinuations. The shot combines “several aural and visual rhythms, each in conflict with the others, which builds to a growing sense of unease in the viewer”: the men’s footfalls, the crashing waves, the intermittent cannons, and the outlying rocks before Othello and Iago descend into the darkened, enclosed fortress.[footnoteRef:4] Welles frames these shifting locations through fluid perspectives, directing viewers’ eyes on the action of Othello’s growing suspicions. We can shift our eyes within the frame, to watch the waves or to notice Micheál MacLiammóir’s facial expressions as Iago, but that is all; within that rectangle we see only what the camera sees, and though time and space we move as it moves. [4: 	Jack Jorgens, Shakespeare on Film, Bloomington: Indiana U P, 1977, 184; cit. Samuel Crowl, Shakespeare and Film: A Norton Guide, New York: Norton, 2008, 173.] Film directors in 2018 operate under identical conditions as Welles did in 1952. They may use more kinetic editing or more closeups than Welles did, but the essential vocabulary and grammar of film are fixed. But the emerging platform of virtual reality (VR) is enabling some directors to expand the filmic frame beyond its unidirectional rectangle, to the 360-degree vantage on performances unfolding in immersive environments. Audiences wear an audio-visual interface like an Oculus Rift that fools our senses into believing what we see and hear is real. Immersive platforms have been used, thus far, more for static experiences and games than for narratives and performances, but production companies like WeVR are expanding the repertoire into storytelling. In this paper, I argue that existing theories of the cognitive ecology of audiences watching performances in physical environments confront new problems when those audiences watch performances in a virtual environment. My core question is what VR’s cognitive ecology owes both to film and to theatre, and how current theories of cognition must be expanded to encompass this new medium. Although my argument is applicable to any performance, I am focusing on Shakespeare not only for reasons of familiarity. It is likely that his plays will be the first to be captured in this medium, for the same reason they dominated early (silent) film, and for the same reason the National Theatre and the Royal Shakespeare Company have broadcast to many ‘live’-to-cinema productions of his plays: because new media need the imprimatur of established artists to legitimize themselves. When we watch a Shakespeare (or any) performance in VR, we combine the theatrical experience of a fixed vantage with film’s ability to alter our perspectives on diverse locations. Yet the experience is something wholly different, merely analogous to film and theatre spectatorship. VR is more immersive than film, but far less flexible. Its performances can happen anywhere, much like in film. The viewer’s position is fixed, as it is in the theatre, yet that position might be at the centre of the action. Perhaps the best analogy is to the processional carts of the medieval cycle dramas — but instead of wagons trundling past, entire scenes shift around you. To illustrate the differences between film and VR, consider how an audience might watch the same scene from Welles’ Othello in VR, had Welles been a VR rather than a film director. We would “wear [our] eyes” and ears by donning a face-mask with headphones in a purpose-equipped enclosed space with sensors to track our limited movements. To see Welles and MacLiammóir walk along the ramparts, we would ‘float’ alongside their three-dimensional bodies. (I use that word because the VR user is typically a disembodied presence, signalled at most by wands or controllers, rather than an avatar or simulated presence; thus we would float rather than walk alongside the actors.[footnoteRef:5]) We would hear the dialogue, ambient sounds, music, and any post-production sound effects in our headphones, which modulate the sound relative to our head’s orientation. That brings me to the most significant difference between film and VR: the viewer’s vantage on the action is fixed, but our orientation toward that action is fluid. We could watch the actors, and note subtle variations of light and shadow and facial expression as a film camera would suggest we do; but we could just as easily watch a seagull flying overheard, or a sentry receding into the middle distance, or a ship docking in the harbour. Without a frame, our eyes can wander freely. [5: 	This is not the scenario that Janet Murray describes in Hamlet on the Holodeck: The Future of Narrative in Cyberspace (Cambridge, MA: MIT Press, 1997). The fictional ‘holodeck’ platform is fully interactive, whereas VR performances are only for consumption. Viewers are more like H. G. Wells’ Invisible Man than like Jean-Luc Picard.] This spatial shift is what decisively distinguishes VR from film spectatorship. Instead of looking through a rectangle onto a narrow range of action, viewers are immersed into an environment where action unfolds around us. Our eyes are no longer controlled by the camera’s orientation, because VR camera-rigs film in every direction. If Andrei Tarkovsky’s metaphor for film was “sculpting in time,” the VR director James Kaelan expanded it in a recent interview to “time plus space — cinema plus architecture.”[footnoteRef:6] [6: 	Andrew Marantz, “Studio 360,” The New Yorker, April 25, 2016, 86-94; 94.] Architecture is why I began with Welles’ Othello: because the location of this scene is integral to its meaning on film. The sequence I described, the conversation along the ramparts, could be preserved in VR while upholding the director’s intentions. But when the scene’s location moves indoors, to the exchange between Iago and Othello that uses filmic frames and perspectives to achieve its effects of light and shadow, and that culminates in a mirror shot of Othello’s brooding face, the VR interface permits too much freedom to reliably convey Welles’ meaning. The director of a VR performance would have to decide, initially, where the viewer is to witness the action — near its centre, or from the periphery, or from above or below, or through a doorway or window, or from a moving position. But these vantage decisions cannot control the viewer’s orientation or attention to that action. I am focusing on media environments and interfaces in this paper, not only because they are the ambit of this seminar but also because “[t]he boundaries of cognitive systems,” as Philip Robbins and Murat Aydede write, “lie outside the envelope of individual organisms, encompassing features of the physical and social environment.”[footnoteRef:7] Or as Rhonda Blair and Amy Cook put it more succinctly, “cognitive systems are inseparable from cultural systems.”[footnoteRef:8] My core assertion in this paper is that VR is one cultural system that will coexist with others, like film and theatre. Yet it is a shift of medium that must alter the cognitive ecology of Shakespeare’s audiences. VR does not (indeed cannot) challenge Edwin Hutchins’ claim that “[c]ultural activity systems have cognitive properties of their own that are different from the cognitive properties of the individuals who participate in them.”[footnoteRef:9] But the flexibility of audience perceptions in the VR environment, and the immateriality of its representations, must alter claims like Alva Noë’s that “[t]he world makes itself available to the perceiver through physical movement and interaction.”[footnoteRef:10] In a virtual world that precludes interaction, and offers only virtual, circumscribed movement, how does that alter its cognitive ecology?[footnoteRef:11] [7: 	Philip Robbins and Murat Aydede, The Cambridge Handbook of Situated Cognition, New York: Cambridge UP, 2009: 10; cit. Rhonda Blair and Amy Cook, Theatre, Performance and Cognition: Languages, Bodies and Ecologies, London: Bloomsbury, 2016, 6.]  [8: 	Theatre, Performance and Cognition, 3.]  [9: 	Edwin Hutchins, Cognition in the Wild Cambridge, MA: MIT Press, 1995, 287.]  [10: 	Alva Noë, Action in Perception, Cambridge, MA: MIT Press, 2004: 1; cit. Blair and Cook, Theatre, Performance and Cognition, 7.]  [11: 	I here confess ignorance of cognitive film theory, and need to do further research.] It remains to be seen how the fluctuating grammar and vocabulary of VR will be translated rather than transposed from film and theatre. For instance, the medium poses new problems for transitions between scenes in different locations: move too abruptly and you will disorient the viewer, at least initially; move too slowly and you lose the narrative momentum. Even the acceleration and deceleration of a (concealed) camera dolly need to adapt to a virtual viewer’s sense of vertigo or balance. VR directors in 2018 are “at the Lumière-brothers stage — little experiments, like pointing the camera at a moving train and seeing what happens,” says Kaelan (94). “We’re watching the semiotics come together in front of our eyes,” adds Julia Kaganskiy of the New Museum (90). These directors are trying different vantage points relative to the action of the scene, and using different techniques to signal where viewers should direct our attention. Much as theatre directors do, they are using subtle cues of lighting and sound and the orientation of other actors in virtual space to direct the viewer’s attention.[footnoteRef:12] The coming years will see VR performances move from imminent to actual, and test theorizations like mine in this paper with virtual examples. [12: 	“In most modern theatres, lighting offers a powerful mechanism for managing attention that in other circumstances must be managed aurally and interpersonally” (Evelyn B. Tribble, Cognition in the Globe: Attention and Memory in Shakespeare’s Theatre, New York: Palgrave Macmillan, 2011, 158).] 

“Wear your eyes thus”: Toward a Cognitive Ecology of VR Shakespeare

Abstract

Similar works

Full text

Available Versions

Humanities Commons