Modality relativity : the influence of sign language and spoken language on conceptual categorization

This dissertation investigates the effects of language modality on conceptual categorization. It is hypothesized that spoken and signed languages have an influence on the processing and structure of conceptual categories. Two aspects of language modality in which spoken and signed languages show major differences are Simultaneity and Iconicity. Compared to the audio-vocal modality of spoken languages, the visual-gestural modality of signed languages is well suited for simultaneously producing a large amount of information and moreover, shows a much greater disposition for iconic signs. The simultaneous articulation in signed languages is only possible for concepts which are syntagmatically related (noun related to an attribute, action etc.) and the so-called polycomponential signs are iconic. Paradigmatically related concepts (noun related to a superordinate, coordinate, subordinate etc.) are expressed in a linear sequence as in spoken languages and can be, but are not necessarily, iconic. Because of this difference between the languages in two modalities they are particularly interesting for the study of the underlying semantic concepts. At the focus of this thesis are especially the questions whether simultaneity and iconicity lead to differences in discourse and communication and as a consequence to different semantic relations between concepts. It is hypothesized that the preference for simultaneous structures in the visual-gestural modality lead to stronger relations between syntagmatically related concepts compared to paradigmatically related ones in the visual-gestural modality. Thus, the general objective of this thesis is to investigate the issue of ‘Modality Relativity’ in terms of the old controversy of ‘Linguistic Relativity’, i.e. whether language can affect thought. A series of experiments will be reported testing whether the two aspects of simultaneity and iconicity affect the semantic relations of concepts in any significant ways and thus, have an influence on the formation of conceptual knowledge. The first part of the thesis addresses the theoretical background. An overview of static and dynamic theories of concepts and categorization is given. The issue of the organization of concepts in the general cognitive system and the question whether concepts are acquired or innate is discussed. In the following the influence of language on perceptual and conceptual categorization will be examined. The Whorfian Linguistic Relativity Principle will be outlined and the basic arguments will be presented for discursive clarity. Subsequently, objections against Whorf’s ideas will be addressed and discussed in some detail. In the next paragraph the focus is on experimental studies about Linguistic Relativity within spoken language cultures and how new evidence changed the theoretical climate and discussion about Linguistic Relativity. The issue of Language Modality, i.e. the condition in which linguistic signs are produced and perceived, will be introduced. Commonalities and differences between German Sign Language (GSL) and German Spoken Language (GSpL) and some non-linguistic studies, which reveal some evidence for an impact of language modality on cognition, are discussed. Finally, in the fourth paragraph, the influence of simultaneity and iconicity of signed languages on conceptual and perceptual categorization is examined. The theoretical part of the thesis ends with the formulation of research questions relating to iconicity and simultaneity. In the empirical part of the thesis, several experiments will be presented, with the aim of contributing and broadening the present research on linguistic relativity. The experiments focus on cross-linguistic differences between German Sign Language (GSL) users, German Spoken Language (GSpL) users and cross-modal bilinguals (GSL and GSpL). Different language comprehension tasks, i.e. verification task, triad-comparison-task and memory-recognition task, are utilized, in order to study the influence of language modality on conceptual structures. Reaction times and Number of Choices, both in linguistic and non-linguistic tasks, will be used as dependent variables. The studies combine a comparison of structural diversities between sign language and spoken language related to language-modality with a demonstration of highly distinctive patterns of concept structure. Based on the results of the empirical studies, the last chapter finally will summarize and discuss the influence of simultaneity and iconicity on conceptualization processes. It will be argued that the studies do provide some evidence for the hypothesis that the modality of the language one uses might qualitatively alter the structure of concepts. The empirical research presented in this thesis aims at expanding the scope of inquiry and discussion on the linguistic relativity proposal by including the aspect of language modality