MULTI-MODAL TASK INSTRUCTIONS TO ROBOTS BY NAIVE USERS

This thesis presents a theoretical framework for the design of user-programmable robots. The objective of the work is to investigate multi-modal unconstrained natural instructions given to robots in order to design a learning robot. A corpus-centred approach is used to design an agent that can reason, learn and interact with a human in a natural unconstrained way. The corpus-centred design approach is formalised and developed in detail. It requires the developer to record a human during interaction and analyse the recordings to find instruction primitives. These are then implemented into a robot. The focus of this work has been on how to combine speech and gesture using rules extracted from the analysis of a corpus. A multi-modal integration algorithm is presented, that can use timing and semantics to group, match and unify gesture and language. The algorithm always achieves correct pairings on a corpus and initiates questions to the user in ambiguous cases or missing information. The domain of card games has been investigated, because of its variety of games which are rich in rules and contain sequences. A further focus of the work is on the translation of rule-based instructions. Most multi-modal interfaces to date have only considered sequential instructions. The combination of frame-based reasoning, a knowledge base organised as an ontology and a problem solver engine is used to store these rules. The understanding of rule instructions, which contain conditional and imaginary situations require an agent with complex reasoning capabilities. A test system of the agent implementation is also described. Tests to confirm the implementation by playing back the corpus are presented. Furthermore, deployment test results with the implemented agent and human subjects are presented and discussed. The tests showed that the rate of errors that are due to the sentences not being defined in the grammar does not decrease by an acceptable rate when new grammar is introduced. This was particularly the case for complex verbal rule instructions which have a large variety of being expressed

Semantic innovation and grammaticalization of the English loanword type in colloquial Hindi

Recent years have seen increased interest in the grammaticalization of type nouns cross-linguistically. This thesis contributes to the growing body of research in the field by using naturally occurring data to investigate innovative uses of the English loanword type in colloquial Hindi by residents of the North Indian Himalayan town of Pithoragarh. A semantic-pragmatic and syntactic analysis of the spoken data reveals that the functions of type in Hindi mirror the grammaticalized functions of type cognates in other languages, including uses as a similative and a hedge, and incipient uses as a quotative marker, a general extender and a discourse marker. The emergence of type in a language already replete with lexical options offers a unique opportunity for comparison between the functions of type and those of equivalent Hindi lexemes. Of particular import is the analysis of aisa ‘like this’ and related lexemes, a class known as ‘similative demonstratives’ (van der Auwera and Sahoo, 2020). Highly prolific in Hindi, similative demonstratives share many of the extended functions of synonyms of like and of type, but also ‘discourse-structuring devices’ associated with English such and German so (König, 2020). A ‘phoric’ construction formed with loanword type appears in similar contexts to similative demonstrative aisa beyond the remit of a modifier, showing evidence of semantic-pragmatic extension and decategorialization. This thesis presents a motivated argument that type may have modelled itself on aisa and be grammaticalizing along a similar path or indeed paths. Finally, the study offers a novel contribution to research into type noun grammaticalization due to type’s status as a recent loanword. Whilst being fully integrated into the Hindi lexicon there is evidence that loanword type also retains its status as an English-origin lexeme, functioning as a flag for English code-switches alongside its hedging role

The functional significance of cross-sensory correspondences in infant-directed speech

Evidence suggesting that infants appreciate a range of cross-sensory correspondences is growing rapidly (see Dolscheid, Hunnius, Casasanto & Majid, 2014; Fernández-Prieto, Navarra & Pons, 2015; Haryu & Kajikawa, 2012; Mondloch & Maurer, 2004; Walker, Bremner, Mason, Spring, Mattock, Slater, & Johnson, 2010; Walker, Bremner, Lunghi, Dolscheid, Barba & Simion, 2018), and yet there is no known attempt to establish the functional significance of these correspondences in infancy. Research shows that speakers manipulate their prosody (i.e. melody of spoken language) to communicate the meaning of unfamiliar words and do so in ways that exploit the cross-sensory correspondences between, for example, pitch and size (Nygaard, Herold & Namy, 2009) and pitch and height (Shintel, Nusbaum & Okrent, 2006). But do infants attend to a speaker’s prosody in this context to interpret the meaning of unfamiliar words? The aim of this thesis is to further establish how infant-directed speakers use prosody to communicate the cross-sensory meanings of words and, for the first time, identify whether infants capitalise on their sensitivity to cross-sensory correspondences to resolve linguistic uncertainty. In Experiment 1 – 4 we identify a list of novel pseudowords to use in all experiments being reported. These pseudowords were judged by participants as being neutral in terms of their sound-symbolic potential, allowing us to rule out the impact of sound-symbolism in our investigation. Experiment 5 provides support for earlier studies revealing cross-sensory correspondences in infant-directed speech. When presented with pseudowords spoken in a prosodically meaningful way, 13-month-old infants demonstrated a preference for objects that were contradictory to the cross-sensory acoustic properties of speech (e.g. lower-pitch voice with higher objects) (Experiment 6), and adults failed to match pseudowords with objects based on the prosodic information that was provided (Experiment 7). However, Experiment 8 provides evidence that 24-month-olds match pseudowords spoken in a higher-pitch voice, and at a faster rate, with objects that are visually higher in space. The implications of these findings are discussed, with suggestions as to how they can be usefully extended

Aboriginality and English : report to the Australian Research Council

The relation of aboriginality to English has important implications for communication between Aborigines and other Australians, and especially for the education of Aboriginal and other Australian children within a context of reconciliation. The investigation of which this is the final report derives from the assumptions that Aboriginal English has been maintained at least in part because of its function\u27 as a bearer of aboriginality and that, by exploring the nature of the distinctiveness of this dialect and the historical circumstances of its formation and ongoing development we may better understand how to provide appropriately for the communicative and educational needs of its speakers within a society where the dialect operates alongside the significantly different variety which is generally called Australian English

Interpreting language-learning data

This book provides a forum for methodological discussions emanating from researchers engaged in studying how individuals acquire an additional language. Whereas publications in the field of second language acquisition generally report on empirical studies with relatively little space dedicated to questions of method, the current book gave authors the opportunity to more fully develop a discussion piece around a methodological issue in connection with the interpretation of language-learning data. The result is a set of seven thought-provoking contributions from researchers with diverse interests. Three main topics are addressed in these chapters: the role of native-speaker norms in second-language analyses, the impact of epistemological stance on experimental design and/or data interpretation, and the challenges of transcription and annotation of language-learning data, with a focus on data ambiguity. Authors expand on these crucial issues, reflect on best practices, and provide in many instances concrete examples of the impact they have on data interpretation

The Chepang language: Phonology, Nominal and Verbal morphology - synchrony and diachrony of the varieties of the Lothar and Manahari Rivers

N/