Skip to main content
Article thumbnail
Location of Repository

The Missing Link between Morphemic Assemblies and Behavioral Responses:a Bayesian Information-Theoretical model of lexical processing

By Fermin Moscoso del Prado Martin, Kostic Aleksandar and Filipovic-Djurdjevic Dusica


We present the Bayesian Information-Theoretical (BIT) model of lexical processing: A mathematical model illustrating a novel approach to the modelling of language processes. The model shows how a neurophysiological theory of lexical processing relying on Hebbian association and neural assemblies can directly account for a variety of eects previously observed in behavioral experiments. We develop two information-theoretical measures of the distribution of usages of a word or morpheme. These measures are calculated through unsupervised means from corpora. We show that our measures succesfully predict responses in three visual lexical decision datasets investigating the processing of in ectional morphology in Serbian and English languages, and the eects of polysemy and homonymy in English. We discuss how our model provides a neurophysiological grounding for the facilitatory and inhibitory eects of dierent types of lexical neighborhoods. In addition, our results show how, under a model based on neural assemblies, distributed patterns of activation naturally result in the arisal of discrete symbol-like structures. Therefore, the BIT model oers a point of reconciliation in the debate between distributed connectionist and discrete localist models. Finally, we argue that the modelling framework exemplied by the BIT model, is a powerful tool for integrating the different levels of the description of the human language processing system

Topics: Neurolinguistics, Statistical Models, Language, Neural Modelling, Computational Linguistics, Computational Neuroscience, Semantics, Morphology, Machine Learning, Psycholinguistics, Cognitive Psychology, Neural Nets, Artificial Intelligence
Year: 2006
OAI identifier:

Suggested articles


  1. (2005). A principal components regression analysis of simple word recognition', Manuscript submitted for publication,
  2. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction and representation of knowledge',
  3. (1959). Applications of information theory to psychology: a summary of basic concepts, methods, and results,
  4. (2004). Bayesian integration in sensorimotor learning',
  5. (1991). Bayesian mixture modeling by Montecarlo simulation,
  6. (2005). Breaking the tyranny of learning: a broad-coverage distributed connectionist model of visual word recognition', Manuscript, MRC{Cognition and Brain Sciences Unit.
  7. (2005). Changing places: A cross-language perspective on frequency and family size in Hebrew and Dutch',
  8. (1949). Communication in the presence of noise', doi
  9. (1991). Conditioning Diagnostics: Collinearity and Weak Data in Regression,
  10. (1991). Corticostriatal cell assemblies in selective attention and in representation of predictable and controllable events: a general statement of corticostriatal interplay and the role of striatal dopamine',
  11. (1996). Data Analysis: A Bayesian Tutorial, doi
  12. (2005). Data mining at the intersection of psychology and linguistics,
  13. (1978). Ecient screening of nonnormal regression models', doi
  14. (2005). Edgeworth approximation of multivariate dierential entropy', doi
  15. (2004). Estimating mutual information',
  16. (1989). Fast learning in networks of locally tuned processing units',
  17. (1989). Frequency and neighborhood size eects on lexical access: Activation or search?',
  18. (1992). Frequency and neighborhood size eects on lexical access: Similarity or orthographic redundancy?',
  19. (2004). From parts to wholes: mechanisms of development in infant visual object processing', doi
  20. (2003). Guilt by association: Gleaning meaning from contextual co- occurrence',
  21. (1997). How well can passage meaning be derived without using word order? A comparison of latent semantic analysis and humans.,
  22. (2003). In morphology and word meaning: orthogonal or co-implicative domains?, in
  23. (2000). Information Theory and the Brain,
  24. (2003). Information Theory, Inference, and Learning Algorithms, doi
  25. (1991). Informational approach to processing in morphology: Standard data reconsidered',
  26. (1995). Informational load constraints on processing in morphology,
  27. (1994). Interactive-activation as a framework for understanding morphological processing', doi
  28. (2001). Kvantitativi opis strukture srpskog jezika { Korpus Srpskog Jezika (`Quantitative description of Serbian language structure { the Corpus of Serbian Language')',
  29. (1998). Markov chain sampling methods for Dirichlet process mixture models.,
  30. (2002). Memory traces for in axes as shown by the mismatch negativity', doi
  31. (2000). Modelling clarity of change in spontaneous speech,
  32. (2005). Morphological family size in a morphologically rich language: The case of Finnish compared to Dutch and Hebrew',
  33. (2005). Multivariate Edgeworth-based entropy approximation, doi
  34. (2001). Phonotactics, density, and entropy spoken word recognition', doi
  35. (1999). Probabilistic phonotactics and neighborhood activation in spoken word recognition', doi
  36. (1996). Producing high-dimensional semantic spaces from lexical cooccurrence', doi
  37. (2004). Putting the bits together: An information theoretical perspective on morphological processing',
  38. (1979). Recognition of axed words and the word frequency eect', doi
  39. (1998). Recognizing spoken words: the Neighborhood Activation Model', doi
  40. (2001). Rethinking the word frequency eect: The neglected role of distributional information in lexical processing', doi
  41. (1996). Semantic ambiguity eects in word identi
  42. (1995). Semantic and associative priming in highdimensional semantic space,
  43. (1997). Singulars and plurals in Dutch: Evidence for a parallel dual route model',
  44. (1965). Sintakticke funkcije padezih oblika u srpskohrvatskom jeziku (`Syntactic functions of cases in Serbo-Croatian language')', Institute for Experimental Phonetics and Speech Pathology,
  45. (1949). Statistical behavioristics and sequences of responses',
  46. (2001). Testing the distributional hypothesis: The in of context judgements of semantic similarity,
  47. (2000). The direct route: Mediated priming in semantic space,
  48. (2005). The eects of homonymy and polysemy on lexical access: and MEG study',
  49. (1997). The eects of orthographic similarity on lexical retrieval: Resolving neighborhood con
  50. (2005). The eects of the amount of information on processing of in morphol53 ogy', Manuscript submitted for publication,
  51. (2002). The English Lexicon Project: A web-based repository of descriptive and behavioral measures for 40,481 english words and nonwords.',
  52. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information',
  53. (1982). Vision: A computational investigation into the human representation and processing of visual information, Freeman & Co.,
  54. (1997). Why safe is better than fast: The relatedness of a word's meaning aects lexical decision times',

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.