Abstract. In this paper we announce the release of a large dataset of storytelling monologue with motion capture for the head and body. Initial tests on the dataset indicate that head motion is more dependant on the speaker than the style of speech

Ben Youssef, Atef

Braude, David A.

Shimodaira, Hiroshi

English

In this paper we announce the release of a large dataset of storytelling monologue with motion capture for the head and body. Initial tests on the dataset indicate that head motion is more dependant on the speaker than the style of speech

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      The University of Edinburgh Head-Motion and Audio Storytelling(UoE-HAS) DatasetCitation for published version:Braude, DA, Shimodaira, H & Ben Youssef, A 2013, 'The University of Edinburgh Head-Motion and AudioStorytelling (UoE-HAS) Dataset'. ???in??? Proc. of Intelligent Virtual Agents. ???pages??? 466-467.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Author final version (often known as postprint)Published In:Proc. of Intelligent Virtual AgentsGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 20. Feb. 2015The University of Edinburgh Head-Motionand Audio Storytelling (UoE - HAS) DatasetDavid A. Braude, Hiroshi Shimodaira, and Atef Ben YoussefCentre for Speech Technology Research,School of Informatics, University of Edinburgh,10 Crichton Street, Edinburgh, EH8 9AB, UK{d.a.braude,h.shimodaira,abenyou}@inf.ed.ac.ukAbstract. In this paper we announce the release of a large datasetof storytelling monologue with motion capture for the head and body.Initial tests on the dataset indicate that head motion is more dependanton the speaker than the style of speech.Keywords: Head Motion, Dataset.1 IntroductionThere are very few datasets that have tracked head and body motion duringspeech. Those that do exist tend to be short and have very few speakers. Toaddress this lack we are making available a dataset of storytelling monologuesthat was recorded at the University of Edinburgh. This dataset contains speechand motion capture of the head and upper body and so is suitable for researchinto virtual avatars or body language. We are also in the process of transcribingthe speech to open up avenues of research into utilising linguistic information,and enable the data to be used in speech research.2 Description of the DatasetThe subjects were 16 UK native English speakers. Nine were female and sevenwere male. Ahead of the recording session the participants were given ﬁve stories,these were classical fairy-tales that they should have been familiar with fromchildhood.During the recordings the participants were given the story on a teleprompter(Read speech) and then were asked to retell the story in their own words (Freespeech). Previous recordings showed when speakers were asked to choose theirown stories for free speech but these stories generally lasted less than two min.They were seated during the recording and they were instructed to tell the storyas if to an adult native English speaker.Five motion capture markers were placed on the chest and the participantswore another four markers on a hat to capture the body and head motion respec-tively. The motion capture was done with the Natural Point, Optitrack systemThe UoE-HAS Dataset 467Table 1. Lengths of recordings (min:sec)Read Free TotalTotal 371:14 323:22 694:36Table 2. Mean Cross Entropy distance between diﬀerent utterancesintra interSpeaker 0.97 3.15Style 2.86 3.23using seven V100:R2 cameras at a 100Hz sampling rate. Audio was captured us-ing a free-standing directional microphone. The audio was captured at 44100Hzwith 32-bit depth and down-sampled to 16-bit .WAV format using Audacity.The audio and motion capture start at the same frame. Table 1 shows the totallengths of recordings available.To obtain the dataset please visit either the SSPNet Project1 or CSTR2 web-sites where further details about the dataset are provided.3 Speaker and Scenario DependencyTo determine the similarity of the speakers the Cross Entropy distance was used[1]. A multidimensional Gaussian distribution was used to model the data.The mean distance between examples of the same type (intra) and examplesof diﬀerent types (inter) is given in Table 2. Style refers to whether it was reador free speech.From Table 2 it is clear that there are diﬀerences between read and free speechand head motion from diﬀerent speakers and the speaker dependence is higherthan the style dependence.4 Future WorkWe are currently in the process of recording a large dataset of dialogues. It willalso have a large amount of speakers and long samples from each speaker.Acknowledgements. This work was supported by EU FP7 SSPNet (grantagreement no. 231287) The ﬁnancial assistance of the South African NationalResearch Foundation (NRF) towards this research is hereby acknowledged. Opin-ions expressed and conclusions arrived at, are those of the author and are notnecessarily to be attributed to the NRF.Reference1. Hele´n, M., Virtanen, T.: Audio Query by Example Using Similarity Measures be-tween Probability Density Functions of Features. EURASIP Journal on Audio,Speech, and Music Processing 2010, 1–12 (2010)1 http://sspnet.eu/2 http://www.cstr.ed.ac.uk/

Audio Query by Example Using Similarity Measures between Probability Density Functions of Features.

The University of Edinburgh Head-Motion and Audio Storytelling (UoE-HAS) Dataset

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.395.8989

The University of Edinburgh Head-Motion and Audio Storytelling (UoE-HAS) Dataset

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer