The basis of this project is to develop a portfolio of works as a multimedia producer in the increasingly intertwined fields of music, sound and visual production. The final deliverable for this project will be a collection of works and documentation presented in the form of a website, focusing on but not limited to the role as Head of Music and Sound for a film produced in conjunction with fellow MPTI candidate Matt McIntyre, who is the head visual producer for the film. It will be shown at the City of Arts and Sciences IMAX Hemispheric Dome in 6.1 3D Sound in June 2017.https://remix.berklee.edu/graduate-studies-production-technology/1069/thumbnail.jp

Ramirez, AJ

English

Berklee Research Media and Information Exchange (REMIX)

Culminating Experience Written Report  AJ Ramirez, MPTI Candidate 2017 A Three-Dimensional Producer Audio/Visual Production focused on Dimorphic, a 6.1 3D Sound Film Project INTRODUCTION The basis of this project was to develop a portfolio of works as a multimedia producer in the increasingly intertwined fields of music, sound and visual production. The final deliverable for this project is a collection of content and documentation presented in the form of a website, focusing on the role as Head of Music and Sound for a film produced in conjunction with fellow MPTI candidate Matt McIntyre. The film was shown in the City of Arts and Sciences IMAX Hemispheric Theatre in 6.1 3D sound in June and July 2017. The personal objectives were as follows: • Gain experience working with artists to realize an artistic vision in the context of a large scale multimedia project • Practice directing teams of musicians and composers/sound designers in a creative project management role • Develop and demonstrate skills relevant to the intersection of music and visual production • Explore the technological possibilities of the medium, in this case a 6-channel 3D sound system • Develop and expand upon online portfolio of works STATE OF THE ART REVIEW Music and Sound Direction Implementation of music and sound design into film is nothing new. The first implementation of music and audio into film is widely considered to have with The Jazz Singer, produced in 1927.5   Experimentation with surround sound, or multichannel audio began only a few years later, in 1941 with Disney’s Fanstasia.6  Fast forward to the modern day and music and film are now intrinsically tied together, with most motion pictures featuring some elements of music and sound design, and associated sound systems range from basic stereo setups to 22.2 surround sound, which uses 24 individual speakers.7   Much, though certainly not all, music prepared for film these days is considered very cinematic and often has certain characteristics, such as orchestral instrumentation, deeply emotive composition, heavy thematic elements, etc.  This project differed from such an approach, in that the music was developed in the way that electronic music production often occurs: iteratively, rather than through scoring, and with composition and sound design occurring simultaneously. All music and sound design was done in Ableton Live across a team of composers working within a common structure, and was then consolidated to be mixed in Avid Pro Tools with the help of 3D mixing software called MNTN by Taucher Sound8. 3D/Surround Audio Mixing This project presented a unique opportunity, because the Hemispheric IMAX dome has neither a traditional 5.1 nor 7.1 Surround setup. Instead, it has a 6 channel system, similar to 5.1 but with an additional top speaker. For this kind of format there is not as formalized method for music and sound production.9   Pro Tools HD, and other DAWs, allow for 6.1 mixing, but not for a 6th Top speaker, but instead a 6th Center Side speaker. While such a system could have been used, it would have resulted in inaccurate spatialization as sound objects move in the space. Further, it would have made the mixing process more difficult for me in knowing how to make sonic elements translate a specific motion in space.  This led to researching the possibilities that exist to technologically accomplish this project, to be able to mix for this unique venue and loudspeaker configuration.  Making a decision matrix helped to determine what pieces of software would accomplish my goals, focusing on accessibility, the ability to work with a stimulated surround atmosphere in headphones, the flexibility of output configurations, and the ability to control discretion and divergence between channels. Fig 1. Decision Matrix for 6.1 3D Mixing This process led to choosing a program called MNTN, which achieves all of these goals. It is a relatively new piece of technology – it appears that no other software has allowed for the creation of custom multichannel sound spaces with as simple integration into common DAWs, such as Ableton Live or Pro Tools.  Fig 2. Rear view of 6.1 3D Space Design in MNTN  PROGRAM ACCESS HRTF/BINAURAL/STEREOCUSTOM SPACE DESIGNDIVERGENCE CONTROLAVID PRO TOOLS LIMITED (HD) ✓- ✓- ✓LOGIC PRO X EASY ✓ ✓ ✓-DOLBY DIGITAL PLUS  DIFFICULT ✓ X ✓IOSONO  DIFFICULT X ✓ ✓MNTN EASY ✓+ ✓+ ✓+DESCRIPTION OF THE CULMINATING EXPERIENCE Roles and tasks As Head of Music and Sound Production for the film, these are the roles and tasks that the project involved. • Assist in conceptualizing the film, specifically the role music, sound, and voiceover would have  • Establish influences and references to guide development • Determine collaborators, align vision, set expectations • Develop script and voiceover recording plan, set timecode • Create strategy for visual and musical development based on time constraints • Research and establish technical setup • Plan mix architecture and composition template based on project parameters • Develop musical and sonic content alongside collaborators • Guide and direct development based on iterative approach • Test content on-site • Consolidate music from collaborators, prepare streamlined session according to mix architecture and 6.1 technical requirements • Arrange, compose and sound design to fill out the film and create a coherent and consistent sonic palette  • Maintain constant creative communication between audio and visual development • Request and obtain resources for completing the project (on-campus mixing environments) • Configure and calibrate mix environments for accurate reproduction for surround sound mixing • Finalize content (music, sound and voiceover) and mix to surround • Based on on-site test learnings, iterate the mix and prepare for final file formatting based on audio standards • Confirm successful final tests • Prepare documentation and continue to assist in future ramifications for the work  Documentation Due to the nature of this project and the fact that the final execution can only be experienced on-site, in the IMAX with 6-channel 3D sound, the documentation of this project was crucial in communicating the scope and success of the film. Throughout each step of the process, video documentation has been kept in mind. Beyond the standard CE documentation video, documentation of the experience includes a series of videos going behind the scenes to detail the production process, from initial conception, to planning, to execution, to screening. Premiere and screenings Through collaboration between the City of Arts and Sciences and Berklee, it was arranged for the film to be shown at the Hemispheric IMAX theatre in the summer of 2017. The premiere happened on May 31st, 2017, followed by two Spanish screenings on the 8th and 15th of June, and one final screening on July 8th during commencement weekend. Fig 3. Premiere night poster  INNOVATIVE ASPECTS OF THE WORK Customized 3D sound spatialization  This project was an opportunity to work in 6-channel, 3D sound. This can be differentiated from stereo or surround sound, because 3D sound acts in both horizontal and vertical plane. Rather than having  a level field in which sound is distributed, the Hemispheric Theatre has a 6.1 system that include a Top speaker in addition to the traditional 5.1 setup: L, C, R, Lr, Rr, and LFE. Fig 3. Hemispheric Dome Panorama Fig 4. MNTN 6.1 3D space Fig 5. MNTN 360º plugin panner Fig 6. Pro Tools 6.1 panner  In Figure 3, the locations of the 6 speakers dome can be seen in a panoramic view from the position of the audience. Then, in an aerial view, Figure 4 shows the virtual dome designed within MNTN. From this, the speaker position on the surround panner of the MNTN plugin can be derived, as seen in Figure 5. This demonstrates the crucial different between surround mixing in Pro Tools and MNTN. Figure 6 shows Pro Tools’ panner, which places the Top speaker in the real. Therefore, a panning motion within Pro Tools Surround Panner would not be accurately replicated in the Hemispheric theatre.  A panning motion in a circle on Pro Tools HD would suddenly jump through the Top speaker, which in the dome is situated closer to the front of the theatre. In order to know throughout the process that the mix would translate the way in the venue (at least in terms of spatialization), it required a solution outside of the capabilities of normal DAWs. The factors were taken into consideration can be viewed in the decision matrix in Figure 1. Creative sound design and music production Beyond the technicalities of mixing for surround sound and the organizational processes of the project, Dimorphic also provided a unique opportunity in terms of creative boundaries for sound and music production. Since the film was billed as an “Immersive Audio/Visual Exploration of the Brain and Mind” there was a lot of room for creative interpretation and exploration. To organize the creative development of the project, references were explored collectively and qualitative descriptions of the desired music while each piece was envisioned. Fig 5. Music style/feel references 8 total composers contributed to the project. In order to generate material to work with, as well as give the film a consistent sonic flavor, recording sessions were held of raw improvisatory sound such as heavily effected guitars with multiple amps, and analog synth noise/drone sessions. This material was used heavily throughout the 38:24 of the film, both as source material for music, and for foley/SFX sound. CHAPTER DESCRIPTION/INSPIRATION EST. LENGTH REFERENCESIntro Colorful, saturated, thick and growing but sporadic sound design, lush and powerful. Ends on a high note. 3 minFlying Lotus, Sound of Ceres, Studio OCTAttention Inspired,energyic sporadic, offcolor crazyness, settling into a groove with high energy. 5 minAntonio Sanchez, Amon Tobin, DiploReason Chill, reawakening and returning from off color to oncolor, growing energy and steadying, mechanical, computery. 5 minOneohtrixPointNever, Aphex Twin, Mount KimbleMemory Getting into trippy and spacey stuff, using manipulated guitar sound design loopy pedal noise action and synths of course 5 min Future Death, Bon Iver, TychoEmotionPeacful, textural, starting more rhythmic then devolving into simpler, full sounding drones but with a definite analog aesthetic (not too new age). Move into a meditation during which there will be less VO, and the audience will be guided into a pensive exploration of their state of emotion and awareness.7 min Evenings, Pierce Warnecke, TeebsPerceptionAwakened consciousness, not unity but fullness and crazy otherwordliness. Using a lot of futurisic sound design but still spacey and not overly bassy. Builds over a long time. Try "wordly" textures like the hang drum used in Psychic Projections. The idea of the section is that the audience is entering an awaked level of consciousness about their brain and understanding of the world.5 min Xo:re, Quanta, Mumushku, Zebbler Encanti ExperienceOutro Big epic builds into resolving, peaceful sounding, steady but clearing ending music. 5 min Hot Sugar, Daedalus, Nicolas JaarFig 6. Improvisatory noise sessions   NEW SKILLS ACQUIRED Dimorphic was a chance to see through the entire process of mixing for a truly large sound system. Since all collaborators were asked to share their original project files, every aspect of the sound was able to be controlled. For every single gain stage, eq decision, or space/time manipulation, an attempt was made to mix while keeping the integrity of the audio fully intact.  Fig 7. Sala E 6.1 Mini Dome  This attention to detail especially came into play in the final stages of printing the full 6-channel mixes, with aspects such as metering loudness, managing the dynamic range of the audio, controlling peaks, managing bass, maintaining the intelligibility of the voiceover.  Along with this came the process of setting up and calibrating surround mixing environments. Campus space was made available both in Sala E of the Palau, as well as Studio A on Berklee campus, and with the help of several faculty mentors, considerations such as loudness metering, time alignment, LFE distribution, divergence control, accurate frequency response, multichannel ducking, and more were learned. And this is to say little of applying skills and techniques in the sound itself, from use of tools like EQ and compression to heavy pitch/time manipulation techniques, as well as specific sound design techniques and tools, specifically within Ableton Live. CHALLENGES Expected Managing to create a film with consistently engaging voiceover proved to be the most difficult aspect of the film, with having had to test out both of the original choices for the English and Spanish versions, record practice takes, record final takes and touchup sessions, then comp to time, control the intelligibility and dynamics of the voice in the full mix, all while allowing room for the sound and music to work. In this case, the VO guided the content, as both the visuals and music were only able to develop once the script was finalized and a scratch take of VO was recorded and comped to timecode. Fig 8. Ableton Live full film set, organized into STEM groups to be printed for surround mixing File management was another expected challenge. Along with this came the specifics of turning 10-12 individual Ableton Live projects into one a set of STEMs that could be more easily mixed to 6.1 in Pro Tools. A mix architecture had to be planned out as soon as the technological aspects of the project were known. Unexpected The unexpected challenge of this project involved the specifics of preparing the audio for its final delivery. Without having a prior education on topics like loudness metering, full mix dynamic range control, multichannel I/O, etc., it couldn’t have been known that these subjects would be as challenging as they were. FUTURE RAMIFICATIONS The specifics are yet to be determined, but there is a plan to pursue licensing and distribution opportunities to have it screened at other theaters around the world, as well as at film festivals. The intention will be to prepare a press kit that explains the project in full, and opens the channel for negotiating a licensing fee. Further, both stereo and 5.1 mixes will be done to maximize compatibility for both online documentation as well as future screenings. CONCLUSION Despite the limited amount of time, and the constant problem of learning everything while doing it at the same time, Dimorphic was a success. An engaging, cohesive film with full-frame IMAX visuals and 6.1 3D sound was produced, and so much was learned from the experience. Screenings thus far have been successful, with audiences reporting it to be “so awesome” and “totally professional.” FOOTNOTES/CITATIONS 1. http://www.theverge.com/2016/4/25/11503124/beyonce-lemonade-visual-album-format-bjork-rihanna 2. http://www.dayfornight.io/media/ 3. http://change-underground.com/rise-vj 4. https://www.marmosetmusic.com/ 5. https://web.archive.org/web/20120725072639/http://www.amps.net/newsletters/issue23/23_jazz.htm 6. http://www.sonicscoop.com/2014/01/21/music-in-5-1-dimensions-how-the-best-surround-mixers-approach-the-soundstage/  7. http://www.nhk.or.jp/strl/publica/bt/en/fe0045-6.pdf  8. http://mntn.rocks 9. http://www.diffen.com/difference/5.1_Surround_Sound_vs_7.1_Surround_Sound 

A Multimedia Producer

https://remix.berklee.edu/cgi/viewcontent.cgi?filename=1&article=1069&context=graduate-studies-production-technology&type=additional

A Multimedia Producer

Abstract

Similar works

Full text

Available Versions

Berklee Research Media and Information Exchange (REMIX)