de Bokx, M.

Fryazinov, Oleg

Bournemouth University Research Online

The relation between sound and music explored through animationand 3D printingMarinthe de Bokx and Oleg FryazinovThe National Centre for Computer Animation, Bournemouth University, UKmarinthedebokx@hotmail.co.uk, ofryazinov@bournemouth.ac.ukAbstractThis paper describes the way to combine sound, music, animation and 3D printing in stop-motion animation. Byusing sound processing and 3D animation, the ensemble of objects was created where each sound is visualised andhas its own style and characteristics. The final stop-motion animation was created by using 3D printing. In the paperwe present the steps of the project whose output was an experimental stop motion animation along with discussingtechnical details of each step.IntroductionIn this project we present a way to combine sound, music, animation and 3D printing and explore the processof visualizing noise using 3D printing as a medium. From artistic point of view the project allows us toindividualise every model in the animation sequence. By using sound processing and 3D animation, we havecreated an ensemble of the objects, where each sound is visualised and has its own style and characteristics.We implemented stop-motion animation where each model is sound-driven and carries their own personalcharacteristics.This project heavily focuses on artistic impression due to some technical restrictions. We created theentire piece within two months by using an inexpensive 3D printer as this was the only one readily availableon campus. These limitations had a big impact on the way the project was implemented and shaped.This paper discusses the stages of our project, whose output was an experimental stop motion animation[1]. A screenshot from the animation is shown in Fig. 1 . Potential extension of this project can be seen infuture music videos or sound related shorts.Figure 1 : A still from the final video with added annotations labelling the names of the models.Relevant work: 3D printing in AnimationThe animation industry uses 3D printing mainly as a supplement for stop-motion animation or for rapidprototyping purposes to create feature films [2, 3] or adverts [4]. In his melodic work Digital Pots, JonathanKeep [5] combined sound with 3D printing. Our project differs from Keeps in that we did not want to alterthe printing process but alter the way the models would move and their aesthetics, relating to the sound thateach model is connected to. MIDI compositions is also well known for their CG project Animusic [6].Sound-driven modelling and animationThe main challenges during this project are to convey sound through movement to explore how differentnoises would create different shapes and motions. We chose to use an abstract music composition that webuilt from seven different noises. These noises were mostly created by using the first author’s own voicerecorded in the ”Garage Band” software as it would create a warm acoustic tone and give the ensemblefeeling and personal attachment. We chose these noises so that they bounce off and compliment each other,helping to make the models feel more connected with each other.The naming of each noise (see annotations in Fig 1) helped us to keep the workflow clean and efficient,this also helped in the later stages of 3D printing when we labelled each model printed. These namesmainly act as onomatopoeia, meaning that the name represents its noise. For each noise we created separateanimated model. Due to the nature of frame- by- frame animation, we expected a significant number ofmodels were going to be printed. Therefore, we chose to keep the geometry of the models to be as easy toprint as possible. By starting from objects with simple geometry, such as cubes, cylinders and pyramids,we made shapes which could be fabricated quickly without the need of support structures. This meant thatthe models were ready to be animated as soon as they came off the printing plate. We chose a cylindricalbase shape for the models, having the same base for all the models simplified our task and helped to create aconsistent style across the seven different sounds. Overall, in this project the shape of the model was purelydesigner’s decision. The seven shapes which correspond to the noises are shown in Fig. 1. For each shapewe created an animation sequence of poses. Two key poses were used, one for ”no amplitude” sound and asecond one for the ”maximum amplitude”. The modelling system automatically created intermediate posesusing interpolation and deformation techniques. Each model has its own rig, inner skeleton, which allowedfor the deformation during animation.Each noise was used as an instrument in a music composition, which was created as a multitrack record-ing. This music composition was provided to the Houdini modelling system, where each track was separatelyanalysed and amplitude of each noise was extracted and linked to the change of the amplitude to the timelineof the animation. In the end by using sound processing we created a curve representing the change of theamplitude which we used in the animation software for the animating of the different CG models. The laststep for connecting sound to animation was to align the curve of the amplitude of the noise for each modelwith the deformation curve that defines the animation.Figure 2 : Animation sequence of poses for the Quiet noise, colour coded.3D printing for stop motion animationStop motion animation typically ranges from 12 to 30 frames per second, where more frames per secondgives a smoother overall motion. Due to the time limitations faced during this project, we animated using12 frames per second. This also added the classical stop-motion frame- by- frame aesthetic to the finalanimation. Limited the amount of poses we needed to print and created a bouncy hand- placed aesthetic thatwe liked.To reduce the number of models for printing in this project, we used the following approach. In theanimation software, the motion for each model was discretized based on the target frame rate thus givingus the set of poses for each shape. Each shape was colour-coded based on the deformation (see Fig 2). Bylooking at the set of resulting poses, we visually identified and omitted poses that appeared to be duplicatesof each other. The resulting set was sent to be printed. For the whole animation, we printed 77 differentposes for the seven models. Due to the intricacy of the sound, some sounds had more poses to be printed off.Before printing, we made one more adjustment for the models. In the stop motion animation the modelshad to be aligned, so additional geometry was added to be sure that no rotational or translational adjustmentwould be required. Initially in the project, we were going to place the models using holes in the base, buteventually we decided to add extra geometry, which is hidden in the final animation (see Fig 4a).Simplicity of design and the fact that the models did not need any support structure allowed us to printthe entire set of poses in a relatively short amount of time, approximately 70 hours excluding failed printsand tests. To show the texture of the model more clearly all the prints are white. The printed poses for onemodel can be seen in Fig 3. During the printing process, the speed of printing was crucial and therefore thequality sometimes suffered. One of the issues that appeared from the fast printing was under-extrusion (seefor example leftmost model in Fig 3). We decided to leave it as it is, as we felt that it added more life intothe objects and created a bigger sense of movement.Figure 3 : Every pose printed in the accurate sequence for the noise: Shriek. The poses look verysimilar, though every little change shows up clearly during the stop motion animation.Setting and shooting 3D printed stop motion animationTo make the shooting process fast and efficient, few steps were done. First, we labeled all the printed poseson their bases, this was not visible in the final animation (see Fig 4a). During the shooting we kept the posessorted into their own groups in numerical order for easy access. We used a photography set up for stopmotion animation, including a static camera on a tripod, remote shutter, and a static background. To preventmodels from shifting during photographing, we used holes in the base of models and additional models’tabs (see Fig 4b) fitted to the black cloth by loops. Though the loops worked better, and took less time tocreate, very careful placement of the poses was required in order to prevent extra slots to be visible in thefinal animation.To prepare for animation, we put all the frames of the final animation into a spreadsheet that containsinformation about each frame and which poses appear in it. To ease the post- processing in this project, wedid not eliminate duplicate frames whilst shooting. For the final animation we shot 722 frames in one day.To keep the viewer’s attention on the models, we used a black backdrop behind the models. This eliminatedany harsh lines in the background and tonal variation. By setting the ISO as low as possible, we achievedminimal grain in our images. We set the shutter speed to the maximum sync speed in correlation with theflash, creating a clean black backdrop. We imported the resulting pictures into Premiere Pro and combined(a) (b)Figure 4 : Extra geometry, black backdrop and placements for the models.them with the soundtrack.Conclusions and discussionThis animation is intended to remove the gap between music-driven computer animation and stop-motionanimation by using 3D printing with a strong artistic component in the final product. We believe that weachieved the goals that we set in the beginning of this project, namely connecting sound and music to thatanimated shapes and creating stop motion animation from physical models. The shapes in the animation gottheir own movements, which were driven by both music and the 3D printing process. This allowed us toachieve the effects which are hard to reproduce by using pure computer generated imagery.In this project we have established a pipeline for similar projects which include all the steps startingfrom sound and music processing and transforming CG animation into the physical world of stop-motionanimation by using 3D printing. This project was intended to be small scale, however workflow employedin this project can be easily reproduced for similar projects, especially if more shape analysis and similartechniques are used to automatise the process. For example, one of the areas for potential use could be musicvideos employing stop motion animation; another larger area can be traditional animation which employs 3Dprinted components. Finally, one possible extension for this project is the creation of similar stop- motionanimations by employing more variations in the colour of the 3D printed poses and more advanced cameramovement. We hope this project pushes for further investigation towards this direction.AcknowledgementsThe practical aspect of the presented animation project was created by the first author, who is an arts anddesign undergraduate student, within an Innovations unit, supervised by a second author. The written paperwas combined between both authors. The authors would like to thank Visual Research Group of the NationalCentre for Computer Animation for an access to the printing hardware and general support.References[1] M. de Bokx, An Ensemble of Sound (2016), https://vimeo.com/158674384 (as of May. 31, 2016).[2] J. Alger, The art and making of ParaNorman(2012), Chronicle Books.[3] P. Brotherton, The art of The Boxtrolls(2014), Chronicle Books.[4] M. Kawano Y. and Takayama, TRANSFORMING (2015), https://vimeo.com/124307239[5] J. Keep, Digital Pots (2011), www.keep-art.co.uk/digital.html[6] W. Lythe, Animusic: a computer animation video album (2011), Animusic inc., DVD.

The relation between sound and music explored through animation and 3D printing

Abstract

Similar works

Full text

Available Versions

Bournemouth University Research Online