The bodily expression of negation in Polish

The aim of this paper is to examine the bodily expressions of negation in Polish with the use of both audiovisual and syntactic material. The concept of NOT expresses many different degrees of rejection in such areas as belief (I don't know, I doubt), judgment (bad), emotion (I don't want) and action (I don't do). We have found a whole range of reactions reflecting negation, including movements of the head, arms, and hands, facial expressions, intonation, and proxemic communication. Multimodal illustrations point to both the polymorphism of the act of negating, and the embodied sources of negation

Aspects of gestural alignment in task-oriented dialogues

Aspects of gestural alignment in task-oriented dialogues Interlocutors in a conversation influence each other in a number of dimensions. This process may lead to observable changes in their communicative behaviour. The directions and profiles of these changes are often correlated with the quality of interaction and may predict its success. In the present study, the gestural component of communication is scrutinised for changes that may reflect the process of alignment. Two types of task-oriented dialogues between teenagers are recorded and annotated for gestures and their features. We hypothesize that the dialogue task type (collaborative vs. competitive), as well as certain culture-specific properties of alignment that differ between German and Polish pairs, may significantly influence the process of communication. In order to explore the data and detect tendencies in gestural behaviour, automatised annotation mining and statistical exploration have been used, including a moving frame approach aimed at the investigation of co-occurring strokes as well as re-occurring strokes and their features. Significant differences between German and Polish speakers, as well as between the two dialogue types, have been found in the number of gestures, stroke duration and amplitude.   Aspekty wzajemnego dopasowania gestykulacji w dialogach zadaniowych Uczestnicy konwersacji wpływają na siebie wzajemnie w wielu różnych wymiarach. Proces ten może prowadzić do obserwowalnych zmian w ich zachowaniach komunikacyjnych. Kierunek i charakter tych zmian często wiążą się z jakością interakcji i mogą pozwolić przewidzieć jej sukces. Celem niniejszego badania jest eksploracja gestowego składnika komunikacji pod kątem zjawisk, które mogą odzwierciedlać zjawisko wzajemnego dopasowania uczestników. Zarejestrowano dwa typy dialogów zadaniowych z udziałem nastolatków. Nagrania anotowano pod kątem gestów i ich cech. W badaniu postawiono hipotezę, iż typ zadania dialogowego (ukierunkowane na współpracę vs. na konkurencję), jak również pewne specyficzne cechy kulturowe dopasowania komunikacyjnego, które różnicują polskie i niemieckie pary rozmówców, mogą oddziaływać na proces komunikacji. W celu eksploracji danych oraz wykrycia tendencji cechujących zachowania gestowe, wykorzystano zautomatyzowane metody analizy anotacji oraz techniki statystyczne. Uwzględniono podejście oparte na ruchomych oknach czasowych, umożliwiające analizę współwystępowania jak i powtarzania gestów właściwych i ich właściwości. Wykazano istotne różnice między rozmówcami polskimi i niemieckimi oraz między typami zadań dialogowych w zakresie liczby gestów, czasu trwania oraz amplitudy gestu właściwego

Gesture and Speech in Interaction - 4th edition (GESPIN 4)

International audienceThe fourth edition of Gesture and Speech in Interaction (GESPIN) was held in Nantes, France. With more than 40 papers, these proceedings show just what a flourishing field of enquiry gesture studies continues to be. The keynote speeches of the conference addressed three different aspects of multimodal interaction:gesture and grammar, gesture acquisition, and gesture and social interaction. In a talk entitled Qualitiesof event construal in speech and gesture: Aspect and tense, Alan Cienki presented an ongoing researchproject on narratives in French, German and Russian, a project that focuses especially on the verbal andgestural expression of grammatical tense and aspect in narratives in the three languages. Jean-MarcColletta's talk, entitled Gesture and Language Development: towards a unified theoretical framework,described the joint acquisition and development of speech and early conventional and representationalgestures. In Grammar, deixis, and multimodality between code-manifestation and code-integration or whyKendon's Continuum should be transformed into a gestural circle, Ellen Fricke proposed a revisitedgrammar of noun phrases that integrates gestures as part of the semiotic and typological codes of individuallanguages. From a pragmatic and cognitive perspective, Judith Holler explored the use ofgaze and hand gestures as means of organizing turns at talk as well as establishing common ground in apresentation entitled On the pragmatics of multi-modal face-to-face communication: Gesture, speech andgaze in the coordination of mental states and social interaction.Among the talks and posters presented at the conference, the vast majority of topics related, quitenaturally, to gesture and speech in interaction - understood both in terms of mapping of units in differentsemiotic modes and of the use of gesture and speech in social interaction. Several presentations explored the effects of impairments(such as diseases or the natural ageing process) on gesture and speech. The communicative relevance ofgesture and speech and audience-design in natural interactions, as well as in more controlled settings liketelevision debates and reports, was another topic addressed during the conference. Some participantsalso presented research on first and second language learning, while others discussed the relationshipbetween gesture and intonation. While most participants presented research on gesture and speech froman observer's perspective, be it in semiotics or pragmatics, some nevertheless focused on another importantaspect: the cognitive processes involved in language production and perception. Last but not least,participants also presented talks and posters on the computational analysis of gestures, whether involvingexternal devices (e.g. mocap, kinect) or concerning the use of specially-designed computer software forthe post-treatment of gestural data. Importantly, new links were made between semiotics and mocap data

Understanding demonstrative reference in text: A new taxonomy based on a new corpus

Endophoric demonstratives such as this and that are among the most frequently used words in written texts. Nevertheless, it remains unclear how exactly they should be subdivided and classified in terms of their different types of use. Here, we develop a new taxonomy of endophoric demonstratives based on a large-scale corpus including three written genres: news items, encyclopedic texts, and book reviews. The taxonomy enables analysts to reliably code endophoric demonstratives based on objectively applicable criteria, while at the same time making them aware of many subtle borderline cases. We consider the taxonomy as a theoretical foundation for future theoretical and empirical work into endophoric demonstratives, and as an analytical tool allowing researchers to unify and compare the results of studies on endophoric demonstratives coming from different genres and languages

Action Categorisation in Multimodal Instructions

We present an explorative study for the (semi-)automatic categorisation of actions in Dutch multimodal first aid instructions, where the actions needed to successfully execute the procedure in question are presented verbally and in pictures. We start with the categorisation of verbalised actions and expect that this will later facilitate the identification of those actions in the pictures, which is known to be hard. Comparisons of and user-based experimentation with the verbal and visual representations will allow us to determine the effectiveness of picture-text combinations and will eventually support the automatic generation of multimodal documents. We used Natural Language Processing tools to identify and categorise 2,388 verbs in a corpus of 78 multimodal instructions (MIs). We show that the main action structure of an instruction can be retrieved through verb identification using the Alpino parser followed by a manual election operation. The selected main action verbs were subsequently generalised and categorised with the use of Cornetto, a lexical resource that combines a Dutch Wordnet and a Dutch Reference Lexicon. Results show that these tools are useful but also have limitations which make human intervention essential to guide an accurate categorisation of actions in multimodal instructions