We interviewed creative professionals at a number of London visual effects and post-production houses. We report on the key issues raised in those interviews: desirable new technologies, infrastructure challenges, personnel and process management

Dodgson, N.

Patterson, J.

Willis, P.

English

Enlighten

       Dodgson, N., Patterson, J., and Willis, P. (2010) What's up prof? Current issues in the visual effects & post-production industry. Leonardo: Art Science and Technology, 43 (1). pp. 92-93. ISSN 0024-094X  http://eprints.gla.ac.uk/47904/  Deposited on: 9 January 2012   Enlighten – Research publications by members of the University of Glasgow http://eprints.gla.ac.uk TransacTions92      LEONARDO, Vol. 43, No. 1,  pp. 92–93, 2010 ©2010 isasTWHAT’S UP PROF?CURRENT ISSUES IN THE VISUAL EFFECTS & POST-PRODUCTION INDUSTRYNeil Dodgson, University of Cambridge, Computer Laboratory, CB3 0FD, U.K. E-mail: <nad@cl.cam.ac.uk> John Patterson, University of Glasgow, Dep’t of Computing Science, G12 8QQ U.K. E-mail: <jwp@dcs.gla.ac.uk> Phil Willis, University of Bath, Computer Science Dep’t, BA2 7AY U.K. E-mail: P.J.Willis@bath.ac.ukSubmitted: 25/2/2009AbstractWe interviewed creative professionals at a number of London visual effects and post-production houses. We report on the key issues raised in those interviews: desirable new technologies, infrastruc-ture challenges, personnel and process management. Visual effects companies began to estab-lish themselves, in the film industry, in the 1980s. The potential of computers became fully apparent during the 1990s when they began to generate realistic imagery [1]. In the U.K. alone, visual effects and post-production are now worth over a billion U.S. dollars.Today, the industry faces many issues critical to its future. To get a snapshot of current issues, we interviewed a range of creative professionals in London in De-cember 2008. In particular, we elicited how those professionals in the creative industry thought that the universities could best help them.  The Organizations We visited six organizations [A–F] rep-resenting different facets of the industry: A. A large visual effects company, dealing mostly with movies. The company employs 20 technical staff, 400 artists, plus management. B. A medium-sized post-production company, working on advertising, television, and movies. The com-pany has over 100 employees, mostly visual effects artists. C. A software developer with 50 em-ployees, producing software for post-production and visual effects.  D. A systems developer with 70 em-ployees producing combined soft-ware and hardware solutions for colour grading. E. A scanning and recording house, a member of an international group providing full services to the film industry, specializing in converting between digital and analogue media. F. An independent consultancy spe-cializing in coordinating research projects in this industry. The Issues We asked each organization to discuss current problems and desires. We subse-quently categorised them three ways: 1. Desirable new technologies. 2. Infrastructure. 3. Managing people and process. 1. Desirable new technologies a) Human in the loop. There is much good university research on fully-automatic methods for image processing and computer vision. These work well at the low quality end of the market (e.g., segmentation and 3D reconstruction). However, this work has had little impact on the high quality end, where every-thing is still done manually. It would be useful to investigate methods that solve particular problems (e.g., optical flow, boundary detection, and object detec-tion) to help a human being either to direct the automated algorithm or to ad-just the output of the automated algo-rithm quickly and efficiently. In either case the semi-automatic method will only be useful if the result is superior to the manual method while taking less time to achieve. [D] b) Repurposing. Research is needed into effective ways to reuse both footage and 3D models. Models tend to be made anew for each sequel. This is under-standable as technology moves on, but it is increasingly expensive. However, we also find that the 3D models used for a movie are not used for the simultane-ously-released accompanying game. How can we make better use of existing assets? [C,F] c) Finding assets. The databases of as-sets are now so large that we need to develop better ways to catalogue them and to search both images and 3D mod-els. There are usually many different versions of a given asset: it is vital to find the correct version, not just the cor-rect asset. [A,F] d) 3D reconstruction. Reasonable methods for the reconstruction of 3D objects exist but they work best with frame-synchronised views from binocu-lar cameras. The next challenge is the extraction of data of good enough quality for the reconstruction of a complete 3D scene from multiple movie cameras. Some aspects of this problem remain challenging. Support for 3D (stereo-scopic) movie-making has become a priority for the industry following the popularity of recent 3D releases. [2,C] e) Artistic control of physical simula-tion. Movie effects need to be visually plausible but the simulations that under-lie them do not have to be physically realistic nor work for longer than the shot. There has been considerable re-search on producing physically realisticsimulations. The industry needs physi-cally plausible simulation that can be directed and modified by the artist [3]. For example, can we build a water simu-lator where the artist can control where the water goes? Could we make a cloth simulator which is physically plausible but which gives the artist control over specific behaviours? How do we make things that look plausible when they are physically impossible? [A,E] f) Making convincing digital humans.Human beings are good at recognizing and analysing the appearance and behav-iour of other human beings. It is still difficult to make a convincing digital human. We know that there is evidence that a digital human that is not quite convincing is more disturbing to the average viewer than a digital human that is clearly not meant to be realistic (“the uncanny valley” [4]). Compounding this is the problem that it is difficult to cap-ture good face data and difficult to pro-duce plausible animation of face data. Acquisition of human motion on set or on a soundstage is particularly expensive and therefore is only used if it is abso-lutely necessary. [A] g) Breaking free from pixels. A non-pixel format (e.g., that in [5]) could be useful to break free from the problem of producing the same material at many different resolutions and needing to en-sure that the original material is always shot at the highest resolution that you will need. Such a format would need to be able to handle all the processing that we currently do on pixelised images. In the long term there would need to be input devices (cameras) and output de-vices (projectors) that could handle the non-pixel format. [B] 2. Infrastructure a) Trans-coding media between digital formats. There has been a proliferation of formats. For example, a single work can be required in a dozen different for-mats resulting in a lot of CPU time and staff time converting between them. One way in which we could tackle this is to develop a video version of Adobe’s Portable Document Format, a single file format that can be converted at need  Transactions     93TransacTionseither at the player or at the server when the player requests the file. [B] b) Backup of large data stores. A post-production or visual effects house pro-duces gigabytes of new data each day. At the small end of the scale, a 2K DPX movie frame requires 12MB, and a 4K frame can require as much as 144MB. At the large end, an advertising poster can be rendered with up to 600 megapixels, requiring 1.8GB. One company uses a 160 TB file store; another mentioned data volumes of several hundred tera-bytes. One company reported that no vendor of off-site backup was able to cope with the quantity of new data that they produce. Two companies com-mented that, because of the volume problem, they maintain their backups on site, with the obvious risks. [A,B,D,F] c) Keeping up with technology. Tech-nology changes rapidly. Companies de-vote much resource to making best use of new technology to speed up processes and keep ahead of the competition. It is not just a question of optimizing the effects algorithms: one company re-ported that less than 20% of their code did the actual effects work, with the rest of the code being required for data man-agement. [D] d) Archiving and cataloguing assets.Archiving everything is problematic. If done, cataloguing is important (see 1(c)). For example, an upcoming feature film has 1700 effects shots, with 4 million assets, variations on those assets produce 10 million identifiable objects. These take up several hundred terabytes. How do we archive something like this? There are many subsidiary questions within this problem: for example, is it sufficient to store the original imagery and models along with a description of the process to get to the final shot? [A,F] e) Archiving footage in perpetuity. In addition to archiving assets in the short to medium term, there is a desire to ar-chive the finished product forever. All physical media deteriorates, whether physical film, magnetic tape, or optical disc. Film has a life of around 40 years, though this varies considerably with storage conditions [6]. Some film has survived reasonably intact over 70 years [7]. LTO Ultrium (½" digital ar-chive tape) has a predicted life of 15–30 years [8]. Can we develop mechanisms that robustly store digital footage for decades or centuries? If so, can we automatically migrate existing film ar-chives to secure digital media. This is not a small problem: the British Film Institute has an archive of 150,000 mov-ies [9]. The Internet Movie Database [10] reports 14,692 movies released in 2008, the equivalent of a hundred mil-lion feet of film per year. [E,F] f) Healing the 2D/3D divide. There are currently separate workflows for 2D data (images) and 3D data (modelling). It would be useful to join the workflows in some way, especially as stereoscopic movies become more popular. [2,C] g) Improving digital capture. There are currently no digital capture devices that can compete with film in quality of captured imagery. [E] 3. Managing people and process a) Managing creative input. A decade ago, visual effects artists were generally aware of the underlying technology and of the entire pipeline from concept to the finished film. Today, young artists, while still skilled creatively, are far less knowledgeable technically. They can thus either fail to use the full power of the technology or fail to understand the implications of their actions for the later stages of the pipeline. [A] b) Managing workflow. The current methods for visual effects and post-production follow a production line: each step in the process building on the previous one. Can we break free of this production line method and provide ef-fective feedback loops between the dif-ferent links in the production chain? [A]c) Managing a large workforce. The industry once consisted of small compa-nies, within each of which everyone knew everyone else. Over the last dec-ade, several of the companies have be-come too large to work in this way. How do we manage this creative, collabora-tive process when people in different parts of the chain do not know each other and have only a basic understanding of each other’s roles? [A] d) Managing client expectations. Vis-ual effects are now an ordinary part of the production pipeline, rather than any-thing special. Some movies now have over a thousand effects shots and even non-effects movies employ a lot of digi-tal post-production. For example, a re-cent live-action movie with no visual effects still had over 900 shots that re-quired CGI post-production, such as changing the sky colour and moving or removing background elements. Much effects work is time-consuming and la-bour-intensive. Many effects are gener-ated using one-off solutions that are thrown together to get the result wanted by the director. Despite these difficulties, the companies find that their clients have little appreciation of which effects are straightforward to produce and which are extraordinarily expensive. There is a common belief that, if they have seen an effect in some other movie, then it must be straightforward to produce. [A,B] Implications and Conclusions With regard to research timescales, the universities and companies differ. The companies need solutions to their current problems, on a timescale of 6 to 24 months. The universities need to work on problems that will become pressing in 5 to 10 years time or on problems for which no solution is obvious to industry. The latter are those problems to which no company will devote resources but for which a solution would be useful, if one could be found. Computer graphics and image proc-essing researchers are best placed to tackle the development of new technolo-gies in (1). These are also the problems best suited to university timescales. We are working with some of the companies to research certain of these. Our col-leagues in networking, information re-trieval, databases, and engineering are best placed to tackle research issues in infrastructure (2), particularly how to handle backup and archive of large data-sets. The managerial issues (3) demon-strate that some of the biggest problems facing the industry have little to do with technology and everything to do with people.References and Notes 1. Richard Rickitt, Special Effects: the history & technique (Virgin Books, 2000). 2. Lenny Lipton, “Digital stereoscopic cinema: the 21st century”, Proc. SPIE 6803, 2008. 3. Ronen Barzel, “Faking Dynamics of Ropes and Springs”, IEEE Computer Graphics & Applications17(3), pp. 31–39, 1997. 4. F.C. Gee, W.N. Browne, K. Kawamura, “Un-canny valley revisited”, IEEE International Work-shop on Robot and Human Interactive Communication 2005 (ROMAN 2005), pp. 151–157, 2005, 5. John Patterson, Philip J. Willis, “Image Process-ing and Vectorisation” International patent applica-tion PCT/GB2007/002470, filed 5 July 2007, U.K.6. James M. Riley, IPI Storage Guide for Acetate Film, Image Permanence Institute, 1993. 7. British Film Institute Mitchell & Kenyon Collec-tion, <www.bfi.org.uk/ features/mk/>, accessed 26 February 2009. 8. Sun Microsystems LTO Ultrium tape cartridge specifications, <nle.ch/dl/LTO.pdf>, accessed 26 February 2009. 9. British Film Institute National Archive, <www.bfi.org.uk/nftva/>, accessed 25 February 2009. 10. Internet Movie Database, <www.imdb.org>, accessed 25 February 2009. 

Database, accessed 25

Digital stereoscopic cinema: the 21st century”,

Faking Dynamics of Ropes and Springs”,

IPI Storage Guide for Acetate Film, Image Permanence Institute,

LTO Ultrium tape cartridge specifications, accessed 26

Special Effects: the history & technique (Virgin Books,

Uncanny valley revisited”,

What's up prof? Current issues in the visual effects &#38; post-production industry

Enlighten: Publications

http://eprints.gla.ac.uk/47904/1/47904.pdf

What's up prof? Current issues in the visual effects & post-production industry

Abstract

Similar works

Full text

Available Versions

Enlighten

Enlighten: Publications

What's up prof? Current issues in the visual effects &#38; post-production industry

Abstract

Similar works

Full text

Available Versions

Enlighten

Enlighten: Publications

What's up prof? Current issues in the visual effects & post-production industry