The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative

Construction Grammar (CxG) is a paradigm from cognitive linguistics emphasising the connection between syntax and semantics. Rather than rules that operate on lexical items, it posits constructions as the central building blocks of language, i.e., linguistic units of different granularity that combine syntax and semantics. As a first step towards assessing the compatibility of CxG with the syntactic and semantic knowledge demonstrated by state-of-the-art pretrained language models (PLMs), we present an investigation of their capability to classify and understand one of the most commonly studied constructions, the English comparative correlative (CC). We conduct experiments examining the classification accuracy of a syntactic probe on the one hand and the models’ behaviour in a semantic application task on the other, with BERT, RoBERTa, and DeBERTa as the example PLMs. Our results show that all three investigated PLMs are able to recognise the structure of the CC but fail to use its meaning. While human-like performance of PLMs on many NLP tasks has been alleged, this indicates that PLMs still suffer from substantial shortcomings in central domains of linguistic knowledge

Visualization and memorization

Previous investigations have suggested that visual memory may involve short-term (STVM) and long-term (LTVM) components. Evidence for this comes from the functional differences between visual memory tested after short, unfilled retention intervals (STVM conditions), and performance measured after any interpolated task with a high mental load (LTVM conditions). The suggestion is that stimulus information is maintained over short, unfilled intervals by visualization, an active, voluntary control process utilizing central resources. Under LTVM conditions interference prevents active maintenance, and the item must be memorized. The aim of this thesis was to provide further evidence on the functional distinction, and the nature of the underlying processes. A number of experiments were conducted using novel matrix patterns as stimulus materials, and on-line control to allow precise manipulation of timing and other display parameters. The dissociation of STVM and LTVM was reflected in several results: STVM and LTVM (a) have different requirements for display time (b) differ in the consistency of performance over trials (c) they involve different coding processes at acquisition and (d) they show quite different relations between accuracy of performance and mean response time. In contrast to this, varying the exposure of a recognition test probe did not dissociate STVM and LTVM performance, and the provision of feedback and retrieval cues durin recall had no clearly interpretable effect. Visualization is a limited capacity process, insofar as it is restricted to one item or presentation at a time, and can maintain information up to a certain level of complexity. Visualized descriptions are constructed rapidly from short display times, and have general application to this class of novel visual patterns. With other evidence, this suggests that visualization is based on low-level 'figural' descriptions, specifying stimuli as a spatial arrangement of shapes formed by groupings of the pattern elements. LTVM performance increases slowly and irregularly with display time and there is a wide variation in performance over trials. Higher-level, 'semantic' descriptions contribute to memorization, and these cannot be applied rapidly and consistently to randomly generated abstract patterns. The results have widespread implications for theories of visual memory. Single-process theories which deny any distinction between short- and long-term memory are ruled out by the data. Other models which (a) consider STVM as an 'activated' part of LTVM or (b) claim the dichotomy arises from simple distinctions in coding or storage or retrieval do not give a complete account of the results. The 'modal' model is also rejected since prolonged visualization of an item after stimulus offset does not lead to an increase in LTVM. To account for this latter finding, it is proposed that visualization and: the elaborate encoding processes required for memorization compete for-central processing resources

Class-incremental learning: survey and performance evaluation

For future learning systems incremental learning is desirable, because it allows for: efficient resource usage by eliminating the need to retrain from scratch at the arrival of new data; reduced memory usage by preventing or limiting the amount of data required to be stored -- also important when privacy limitations are imposed; and learning that more closely resembles human learning. The main challenge for incremental learning is catastrophic forgetting, which refers to the precipitous drop in performance on previously learned tasks after learning a new one. Incremental learning of deep neural networks has seen explosive growth in recent years. Initial work focused on task incremental learning, where a task-ID is provided at inference time. Recently we have seen a shift towards class-incremental learning where the learner must classify at inference time between all classes seen in previous tasks without recourse to a task-ID. In this paper, we provide a complete survey of existing methods for incremental learning, and in particular we perform an extensive experimental evaluation on twelve class-incremental methods. We consider several new experimental scenarios, including a comparison of class-incremental methods on multiple large-scale datasets, investigation into small and large domain shifts, and comparison on various network architectures

Implications of Computational Cognitive Models for Information Retrieval

This dissertation explores the implications of computational cognitive modeling for information retrieval. The parallel between information retrieval and human memory is that the goal of an information retrieval system is to find the set of documents most relevant to the query whereas the goal for the human memory system is to access the relevance of items stored in memory given a memory probe (Steyvers & Griffiths, 2010). The two major topics of this dissertation are desirability and information scent. Desirability is the context independent probability of an item receiving attention (Recker & Pitkow, 1996). Desirability has been widely utilized in numerous experiments to model the probability that a given memory item would be retrieved (Anderson, 2007). Information scent is a context dependent measure defined as the utility of an information item (Pirolli & Card, 1996b). Information scent has been widely utilized to predict the memory item that would be retrieved given a probe (Anderson, 2007) and to predict the browsing behavior of humans (Pirolli & Card, 1996b). In this dissertation, I proposed the theory that desirability observed in human memory is caused by preferential attachment in networks. Additionally, I showed that documents accessed in large repositories mirror the observed statistical properties in human memory and that these properties can be used to improve document ranking. Finally, I showed that the combination of information scent and desirability improves document ranking over existing well-established approaches

Direct-access retrieval during sentence comprehension: Evidence from Sluicing

Language comprehension requires recovering meaning from linguistic form, even when the mapping between the two is indirect. A canonical example is ellipsis, the omission of information that is subsequently understood without being overtly pronounced. Comprehension of ellipsis requires retrieval of an antecedent from memory, without prior prediction, a property which enables the study of retrieval in situ ( Martin and McElree, 2008 and Martin and McElree, 2009). Sluicing, or inflectional-phrase ellipsis, in the presence of a conjunction, presents a test case where a competing antecedent position is syntactically licensed, in contrast with most cases of nonadjacent dependency, including verb–phrase ellipsis. We present speed–accuracy tradeoff and eye-movement data inconsistent with the hypothesis that retrieval is accomplished via a syntactically guided search, a particular variant of search not examined in past research. The observed timecourse profiles are consistent with the hypothesis that antecedents are retrieved via a cue-dependent direct-access mechanism susceptible to general memory variables

The Word Frequency Effect: Relationship of Lexical Entries Between the Primary and Secondary Language

The current study examined the effect of differential word frequency on the relationship of lexical entries between the primary and secondary language. Ninety Urdu-English bilingual participants were used, and their performance was compared to forty-five English monolinguals matched for age and education. The task for both participant groups was a lexical decision task with 60 high and 60 low frequency English words. The stimulus set consisted of four frequency conditions High English-High Urdu, High English-Low Urdu, Low English-High Urdu and Low English-Low Urdu. A general frequency effect was observed – all participants responded faster to high frequency targets than low than low frequency target words. There was also a main effect of language experience with bilinguals producing longer reaction times than monolinguals. In addition, a frequency effect was observed in response times for high frequency English words as a function of their Urdu pair frequency. These results reveal a cross language frequency differential effect consistent with models proposing non-selective access of lexical processing in bilinguals

Nonpharmacological interventions to improve depression, anxiety, and quality of life (QoL) in people with dementia: an overview of systematic reviews

This overview aimed to systematically synthesize evidence from existing systematic reviews to signpost practitioners to the current evidence base on nonpharmacological interventions to improve depression, anxiety, and quality of life (QoL) in people with dementia and to discuss priorities for future research. The databases MEDLINE, PsycINFO, Scopus, and Cochrane Central Register of Controlled Trials were searched in August 2017 with an updated search in January 2019. Fourteen systematic reviews of randomized controlled trials of nonpharmacological interventions were identified. Dementia stage was rated moderate or severe in the majority of the reviews and type of dementia varied. Interventions reported to be effective were cognitive stimulation (QoL: standardized mean difference [SMD] = 0.38), music-based therapeutic interventions (depression: SMD = −0.27, anxiety: SMD = −0.43, QoL: SMD = 0.32), and psychological treatments (mainly cognitive behavior therapy; depression: SMD = −0.22, anxiety: MD = −4.57). Although health-care professionals are recommended to continue using these approaches, future research needs to focus on the type and form of interventions that are most effective for different stages and types of dementia

Augmented reality and scene examination

The research presented in this thesis explores the impact of Augmented Reality on human performance, and compares this technology with Virtual Reality using a head-mounted video-feed for a variety of tasks that relate to scene examination. The motivation for the work was the question of whether Augmented Reality could provide a vehicle for training in crime scene investigation. The Augmented Reality application was developed using fiducial markers in the Windows Presentation Foundation, running on a wearable computer platform; Virtual Reality was developed using the Crytek game engine to present a photo-realistic 3D environment; and a video-feed was provided through head-mounted webcam. All media were presented through head-mounted displays of similar resolution to provide the sole source of visual information to participants in the experiments. The experiments were designed to increase the amount of mobility required to conduct the search task, i.e., from rotation in the horizontal or vertical plane through to movement around a room. In each experiment, participants were required to find objects and subsequently recall their location. It is concluded that human performance is affected not merely via the medium through which the world is perceived but moreover, the constraints governing how movement in the world is controlled

Orientation of the Practitioner in Correctional Work: Continuities in the Empirical Study of Professionalism and the Conditions of Practice

This study examines the relative influence of professional education, the conditions of practice and other factors on the social worker's orientation to the welfare of his clients. The hypothesis, that professionally trained social workers are more oriented to the welfare of their clients than are their untrained co-workers, is tested. The relationship between the extent of perceived organizational constraints and the worker's orientation is assessed to determine if functional autonomy is related to practitioner orientation. The study also examines whether professional education generates commitment to the professional norms of social work. When a worker agrees with a standardized prescription for practice, does agreement imply legitimation, or the usefulness, of the prescription--or both? Inter- and intra-positional consensus, on evaluations of the legitimacy and usefulness of practice prescriptions, is examined in order to locate formal and informal organizational sources of influence on practice orientation. One thousand seventy-five respondents from twenty-three geographically distributed state probation and/or parole agency system populations answered a questionnaire which included instruments treating practitioner orientation, functional autonomy, and the legitimacy and utility of a set of professional prescriptions for practice which were standardized on a national sample of "transmitters" of professional norms--casework teachers. As hypothesized, trained practitioners were more client welfare oriented than those who were not trained. When employing organization was held constant, this finding persevered in a majority, but not all, of the employing organizations. These findings held when status, tenure and experience were also held constant. Female practitioners with every type and at every level of education were more client welfare oriented than male practitioners. Sex, or its social concomitants, and professional education emerge as independent sources of client welfare orientation. Regardless of its sources, practitioner orientation was specified by organizational contingencies. Among these, two elements of caseload composition reduced differences between trained and untrained workers: (1) probation caseloads; (2) adult caseloads. In contrast to earlier findings, the practitioner's perception of his freedom to determine case decisions is not related to his practice orientation. Functional autonomy may be a function of the practitioner's visibility, which is related to organizational complexity. Practitioners with rural caseloads perceive themselves as having greater autonomy than those with urban caseloads. Although professional education exerts a powerful influence on the worker's orientation to the welfare of his clients, it isn't the influence which educators are likely to want. Workers who consistently agree with professional prescriptions for practice do not consistently legitimate them when they are required to consider both their legitimacy and their usefulness. When social workers must consider more than one implication of "agreement" at a time, they do not make judgements which are uniformly consistent with professional norms. Some of the evidence suggests that practitioners tend to legitimate what they believe to be useful. There is consensus, within and among organizational positions, on evaluations of legitimacy and utility of practice prescriptions. Workers' perceptions of supervisors' evaluations are accurate. The substantial consensus on punitive case actions includes legitimation of breaches of confidentiality, routinized forms of persecution of homosexuals, and the automatic response to initiate revocation proceedings for physically aggressive children or clients who engage in extended sexual affairs. Although professionally trained workers are differently oriented to these matters than untrained workers, a large proportion of trained practitioners contribute to the consensus on punitive case decisions. Finally, there is a minor trend in the data indicating somewhat greater consensus among workers than between workers and supervisors. Similarly, there is a greater worker-supervisor consensus than worker-top administrator consensus. This suggests that elective relationships among organizational peers may yield more powerful influences on practice orientation than formally defined hierarchically structured organizational relationships