Cascaded and thresholded processing in visual word recognition: does the Dual Route Cascaded model require a threshold?

The current thesis aims to investigate cascaded processing in visual word recognition by testing the predictions of the Dual Route Cascaded (DRC) model of reading. Despite widespread acceptance of the idea that visual language processing is cascaded, there are circumstances in which such an account is not easily reconciled with the data produced by skilled readers. Recent experiments involving factorial manipulations in reading showed, in particular, additive effects of stimulus quality (i.e., clear vs. degraded stimuli) with letter string length and orthographic neighbourhood size in nonword reading and with word frequency and lexicality when words and nonwords were mixed in the task, thus suggesting that information processing implicated in visual word recognition must be at least partially thresholded. Six experiments have been presented in this thesis: on one hand, a new variable that has a role when the stimuli are degraded – the Total Letter Confusability – has been introduced; on the other hand, the effects due to list composition have been analyzed when the stimuli were degraded in the task. In general, the results obtained suggest a novel interpretation of the additivities previously observed; these findings have been explained within the DRC model which also correctly simulates a significant amount of the data. The empirical evidence collected so far clearly indicates that there is currently no need to assume thresholded processing in the reading system

Processing long-distance dependencies: an experimental investigation of grammatical illusions in English and Spanish

A central concern in the study of sentence comprehension has to do with defining the role that grammatical information plays during the incremental interpretation of language. In order to successfully achieve the complex task of understanding a linguistic message, the language comprehension system (the parser) must – among other things – be able to resolve the wide variety of relations that are established between the different parts of a sentence. These relations are known as linguistic dependencies. Linguistic dependencies are subject to a diverse range of grammatical constraints (e.g. syntactic, morphological, lexical, etc.), and how these constraints are implemented in real-time comprehension is one of the fundamental questions in psycholinguistic research. In this quest, the focus has been often placed on studying the sensitivity that language users exhibit to grammatical contrasts during sentence processing. The grammatical richness with which the parser seems to operate makes it even more interesting when the results of sentence processing do not converge with the constraints of the grammar. Misalignments between grammar and parsing provide a unique window into the principles that guide language comprehension, and their study has generated a fruitful research program

WORD SENSE DISAMBIGUATION WITHIN A MULTILINGUAL FRAMEWORK

Word Sense Disambiguation (WSD) is the process of resolving the meaning of a word unambiguously in a given natural language context. Within the scope of this thesis, it is the process of marking text with explicit sense labels. What constitutes a sense is a subject of great debate. An appealing perspective, aims to define senses in terms of their multilingual correspondences, an idea explored by several researchers, Dyvik (1998), Ide (1999), Resnik & Yarowsky (1999), and Chugur, Gonzalo & Verdejo (2002) but to date it has not been given any practical demonstration. This thesis is an empirical validation of these ideas of characterizing word meaning using cross-linguistic correspondences. The idea is that word meaning or word sense is quantifiable as much as it is uniquely translated in some language or set of languages. Consequently, we address the problem of WSD from a multilingual perspective; we expand the notion of context to encompass multilingual evidence. We devise a new approach to resolve word sense ambiguity in natural language, using a source of information that was never exploited on a large scale for WSD before. The core of the work presented builds on exploiting word correspondences across languages for sense distinction. In essence, it is a practical and functional implementation of a basic idea common to research interest in defining word meanings in cross-linguistic terms. We devise an algorithm, SALAAM for Sense Assignment Leveraging Alignment And Multilinguality, that empirically investigates the feasibility and the validity of utilizing translations for WSD. SALAAM is an unsupervised approach for word sense tagging of large amounts of text given a parallel corpus — texts in translation — and a sense inventory for one of the languages in the corpus. Using SALAAM, we obtain large amounts of sense annotated data in both languages of the parallel corpus, simultaneously. The quality of the tagging is rigorously evaluated for both languages of the corpora. The automatic unsupervised tagged data produced by SALAAM is further utilized to bootstrap a supervised learning WSD system, in essence, combining supervised and unsupervised approaches in an intelligent way to alleviate the resources acquisition bottleneck for supervised methods. Essentially, SALAAM is extended as an unsupervised approach for WSD within a learning framework; in many of the cases of the words disambiguated, SALAAM coupled with the machine learning system rivals the performance of a canonical supervised WSD system that relies on human tagged data for training. Realizing the fundamental role of similarity for SALAAM, we investigate different dimensions of semantic similarity as it applies to verbs since they are relatively more complex than nouns, which are the focus of the previous evaluations. We design a human judgment experiment to obtain human ratings on verbs’ semantic similarity. The obtained human ratings are cast as a reference point for comparing different automated similarity measures that crucially rely on various sources of information. Finally, a cognitively salient model integrating human judgments in SALAAM is proposed as a means of improving its performance on sense disambiguation for verbs in particular and other word types in general

Text-image synergy for multimodal retrieval and annotation

Text and images are the two most common data modalities found on the Internet. Understanding the synergy between text and images, that is, seamlessly analyzing information from these modalities may be trivial for humans, but is challenging for software systems. In this dissertation we study problems where deciphering text-image synergy is crucial for finding solutions. We propose methods and ideas that establish semantic connections between text and images in multimodal contents, and empirically show their effectiveness in four interconnected problems: Image Retrieval, Image Tag Refinement, Image-Text Alignment, and Image Captioning. Our promising results and observations open up interesting scopes for future research involving text-image data understanding.Text and images are the two most common data modalities found on the Internet. Understanding the synergy between text and images, that is, seamlessly analyzing information from these modalities may be trivial for humans, but is challenging for software systems. In this dissertation we study problems where deciphering text-image synergy is crucial for finding solutions. We propose methods and ideas that establish semantic connections between text and images in multimodal contents, and empirically show their effectiveness in four interconnected problems: Image Retrieval, Image Tag Refinement, Image-Text Alignment, and Image Captioning. Our promising results and observations open up interesting scopes for future research involving text-image data understanding.Text und Bild sind die beiden häufigsten Arten von Inhalten im Internet. Während es für Menschen einfach ist, gerade aus dem Zusammenspiel von Text- und Bildinhalten Informationen zu erfassen, stellt diese kombinierte Darstellung von Inhalten Softwaresysteme vor große Herausforderungen. In dieser Dissertation werden Probleme studiert, für deren Lösung das Verständnis des Zusammenspiels von Text- und Bildinhalten wesentlich ist. Es werden Methoden und Vorschläge präsentiert und empirisch bewertet, die semantische Verbindungen zwischen Text und Bild in multimodalen Daten herstellen. Wir stellen in dieser Dissertation vier miteinander verbundene Text- und Bildprobleme vor: • Bildersuche. Ob Bilder anhand von textbasierten Suchanfragen gefunden werden, hängt stark davon ab, ob der Text in der Nähe des Bildes mit dem der Anfrage übereinstimmt. Bilder ohne textuellen Kontext, oder sogar mit thematisch passendem Kontext, aber ohne direkte Übereinstimmungen der vorhandenen Schlagworte zur Suchanfrage, können häufig nicht gefunden werden. Zur Abhilfe schlagen wir vor, drei Arten von Informationen in Kombination zu nutzen: visuelle Informationen (in Form von automatisch generierten Bildbeschreibungen), textuelle Informationen (Stichworte aus vorangegangenen Suchanfragen), und Alltagswissen. • Verbesserte Bildbeschreibungen. Bei der Objekterkennung durch Computer Vision kommt es des Öfteren zu Fehldetektionen und Inkohärenzen. Die korrekte Identifikation von Bildinhalten ist jedoch eine wichtige Voraussetzung für die Suche nach Bildern mittels textueller Suchanfragen. Um die Fehleranfälligkeit bei der Objekterkennung zu minimieren, schlagen wir vor Alltagswissen einzubeziehen. Durch zusätzliche Bild-Annotationen, welche sich durch den gesunden Menschenverstand als thematisch passend erweisen, können viele fehlerhafte und zusammenhanglose Erkennungen vermieden werden. • Bild-Text Platzierung. Auf Internetseiten mit Text- und Bildinhalten (wie Nachrichtenseiten, Blogbeiträge, Artikel in sozialen Medien) werden Bilder in der Regel an semantisch sinnvollen Positionen im Textfluss platziert. Wir nutzen dies um ein Framework vorzuschlagen, in dem relevante Bilder ausgesucht werden und mit den passenden Abschnitten eines Textes assoziiert werden. • Bildunterschriften. Bilder, die als Teil von multimodalen Inhalten zur Verbesserung der Lesbarkeit von Texten dienen, haben typischerweise Bildunterschriften, die zum Kontext des umgebenden Texts passen. Wir schlagen vor, den Kontext beim automatischen Generieren von Bildunterschriften ebenfalls einzubeziehen. Üblicherweise werden hierfür die Bilder allein analysiert. Wir stellen die kontextbezogene Bildunterschriftengenerierung vor. Unsere vielversprechenden Beobachtungen und Ergebnisse eröffnen interessante Möglichkeiten für weitergehende Forschung zur computergestützten Erfassung des Zusammenspiels von Text- und Bildinhalten

The effect of input frequency and linguistic complexity on the learning of bei2 constructions in Cantonese preschoolers

Thesis (B.Sc)--University of Hong Kong, 2005.Also available in print.A dissertation submitted in partial fulfilment of the requirements for the Bachelor of Science (Speech and Hearing Sciences), The University of Hong Kong, June 30, 2005.published_or_final_versionSpeech and Hearing SciencesBachelorBachelor of Science in Speech and Hearing Science

Estimating Color-Concept Associations from Image Statistics

To interpret the meanings of colors in visualizations of categorical information, people must determine how distinct colors correspond to different concepts. This process is easier when assignments between colors and concepts in visualizations match people's expectations, making color palettes semantically interpretable. Efforts have been underway to optimize color palette design for semantic interpretablity, but this requires having good estimates of human color-concept associations. Obtaining these data from humans is costly, which motivates the need for automated methods. We developed and evaluated a new method for automatically estimating color-concept associations in a way that strongly correlates with human ratings. Building on prior studies using Google Images, our approach operates directly on Google Image search results without the need for humans in the loop. Specifically, we evaluated several methods for extracting raw pixel content of the images in order to best estimate color-concept associations obtained from human ratings. The most effective method extracted colors using a combination of cylindrical sectors and color categories in color space. We demonstrate that our approach can accurately estimate average human color-concept associations for different fruits using only a small set of images. The approach also generalizes moderately well to more complicated recycling-related concepts of objects that can appear in any color.Comment: IEEE VIS InfoVis 2019 ACM 2012 CSS: 1) Human-centered computing, Human computer interaction (HCI), Empirical studies in HCI 2) Human-centered computing, Human computer interaction (HCI), HCI design and evaluation methods, Laboratory experiments 3) Human-centered computing, Visualization, Empirical studies in visualizatio

Cognitive network science: A review of research on cognition through the lens of network representations, processes, and dynamics

10.1155/2019/2108423Complexity2019210842

Perceptual Learning of German Sounds: Evidence from Functional Load (FL) and High- Variability Phonetic Training (HVPT)

The objective of this thesis is to empirically test the practical implications of the functional load (FL) principle in German. The findings informed the selection of German phonemic contrasts for perceptual training of L2 German learners in a follow-up study. Previous research has suggested that sound contrasts carrying a high FL play a central role in conveying meaning, which closely links to the notions of intelligibility and comprehensibility of spoken utterances. Recent attention to FL in second language (L2) English pronunciation pedagogy highlights its role in selecting appropriate L2 sounds to train. In Study 1, the FL hierarchy of German and the impact of high vs. low FL segments on intelligibility and comprehensibility is tested among L1 German listeners. Results show that high FL errors have a more detrimental effect than low FL errors, but two errors are more severe than one, regardless of FL classification. Study 2 explores two types (i.e., audio and audiovisual) of high-variability phonetic training (HVPT) for challenging German sound contrasts among beginner L2 learners. HVPT employs multiple talkers and variable phonetic environments, thereby enhancing discrimination of sound contrasts. Results showed that especially audiovisual HVPT led to reduced discrimination accuracy, suggesting a need to investigate its use for training beginner learners. These findings shed light upon FL’s applicability in conjunction with word recognition models, thereby guiding future work on FL in L2 pronunciation pedagogy. They also provide insights into the theoretical implications of the HVPT technique in fostering perceptual abilities among beginner L2 learners

Orthographic and phonological priming effects in the same-different task.

Masked priming tasks have been used widely to study early orthographic processes-the coding of letter position and letter identity. Recently, using masked priming in the same-different task Lupker, Nakayama, and Perea (2015a) reported finding a phonological priming effect with primes presented in Japanese Katakana, and English target words presented in the Roman alphabet, and based on this finding, suggested that previously reported effects in the same-different task in the literature could be based on phonology rather than orthography. In this article, the authors explain why the design of Lupker et al.'s experiment does not address this question; they then report 2 new experiments that do. The results indicate that the priming produced by orthographically similar primes in the same-different task for letter strings presented in the Roman alphabet is almost exclusively orthographic in origin, and phonology makes little contribution. The authors offer an explanation for why phonological priming was observed when the prime and target are presented in different scripts but not when they are presented in the same script. (PsycINFO Database Record (c) 2018 APA, all rights reserved)