MISGENDERED: Limits of Large Language Models in Understanding Pronouns

Content Warning: This paper contains examples of misgendering and erasure that could be offensive and potentially triggering. Gender bias in language technologies has been widely studied, but research has mostly been restricted to a binary paradigm of gender. It is essential also to consider non-binary gender identities, as excluding them can cause further harm to an already marginalized group. In this paper, we comprehensively evaluate popular language models for their ability to correctly use English gender-neutral pronouns (e.g., singular they, them) and neo-pronouns (e.g., ze, xe, thon) that are used by individuals whose gender identity is not represented by binary pronouns. We introduce MISGENDERED, a framework for evaluating large language models' ability to correctly use preferred pronouns, consisting of (i) instances declaring an individual's pronoun, followed by a sentence with a missing pronoun, and (ii) an experimental setup for evaluating masked and auto-regressive language models using a unified method. When prompted out-of-the-box, language models perform poorly at correctly predicting neo-pronouns (averaging 7.7% accuracy) and gender-neutral pronouns (averaging 34.2% accuracy). This inability to generalize results from a lack of representation of non-binary pronouns in training data and memorized associations. Few-shot adaptation with explicit examples in the prompt improves performance for neo-pronouns, but only to 64.7% even with 20 shots. We release the full dataset, code, and demo at https://tamannahossainkay.github.io/misgendered/Comment: Accepted at ACL 2023 as a long pape

THE STUDENTS’ ABILITY IN USING PERSONAL PRONOUNS IN NARRATIVE PARAGRAPH AT THE SECOND YEAR OF SMA N 1 DURI

The title of this thesis is “the students’ ability in using personal pronouns in narrative paragraph at the second year of SMA N 1 Duri”. The subject of this study is the second year of SMA N 1 Duri and the object of this study is the students’ ability in using personal pronouns in narrative paragraph. The problem found by the writer is about the students’ ability in using personal pronouns in narrative paragraph. This research has two research questions. The first question is how is the students’ ability in using personal pronouns in narrative paragraph and the second question is factors influence the students’ ability in using personal pronouns in narrative paragraph. In this research, the writer just took 15% of 319 students as sample. It was 48 students. The techniques of data collection are test and interview. Test was used to know the students’ ability in using personal pronouns in narrative paragraph and the data on factors influence the students’ ability in using personal pronouns in narrative paragraph. To know the percentage, the writer used formula as follows: F P x= %100 N Based on the analyzed data, the writer can make conclude that the students’ ability in using personal pronouns in narrative paragraph is in good categories and many factors influence the students’ ability in using personal pronouns in narrative paragraph such as interest, motivation, adequate time, facilities and teacher’s motivation

Pronoun development in bilingual Mandarin-English children

textEnglish and Chinese monolingual children differ in pronoun interpretation: English-speaking monolingual children display Delay of Principle B (DPBE) and Quantificational Asymmetry (QA), whereas Mandarin-speaking monolingual children do not. However, it was yet unknown what bilingual Mandarin-English children's pronoun interpretation will be. The following study investigated pronoun interpretation in 13 Mandarin-English bilingual children under English and Mandarin reflexive, referential, and quantificational conditions. The specific aims were to examine 1) if bilingual Mandarin-English children displayed DPBE and QA in both English and Chinese, 2) if there were any cross-linguistic transfer of pronoun interpretation between the two languages, and 3) if there were any correlations between age and language use and the accuracy of pronoun interpretation. The results showed that Mandarin-English children did indeed display DPBE in English and no DPBE and no QA in Mandarin. There was evidence of cross-linguistic transfer in English and Mandarin Reflexive condition. Finally, age but not language use was correlated with pronoun interpretation accuracy. The study provided insight into pronoun interpretation for bilingual Mandarin-English children and may prove to be helpful for developing an English-Mandarin language ability test.Communication Sciences and Disorder

THE VISUALIZATION METACOGNITIVE STRATEGY TO IMPROVE READING COMPREHENSION

Improving the reading comprehension of the students requires creativity in finding the strategy and motivation to read. Therefore, this research aims to implement visualization metacognitive strategy to improve the students’ reading comprehension. The researcher focuses on the implementation of visualization metacognitive strategy in improving the reading comprehension by using recount text. Then she also limits the scope on reading sub-skills that consists of main idea, supporting details, inference, reference, and vocabulary.  This research used tests which are divided to pretest and post-test and also questionnaire as the instruments. The samples were the students from SMAN 4 Banda Aceh, which consisted of 66 students and divided into experimental class and control class. The scores were tabulated statistically by using t-test. Then, the researcher also analyzed the students’ visual representation. The result of hypothesis testing of this research showed that students’ reading comprehension is significantly improved. This strategy developed students’ knowledge on reading sub-skills. Furthermore, the result of the students’ responses toward the strategy was responded positively by the students as it shows the result of 97% at the questionnaire. Therefore, it is suggested to implement visualization metacognitive strategy in improving reading comprehension. In conclusion, this research has demonstrated that the treatment activities have improved the students’ reading comprehension as they enjoy  competition and fun in learning English

An Analysis of Students’ Ability Using Personal Pronoun in Writing Descriptive Text at SMK PGRI Pekanbaru

ABSTRACT Gina Eka Putri (2019): An Analysis of Students’ Ability Using Personal Pronoun in Writing Descriptive Text at SMK PGRI Pekanbaru. The purpose of this research is to determine the students’ ability using personal pronoun in writing descriptive text at SMK PGRI Pekanbaru. There were 34 students as samples by using random sampling technique. This research approach was descriptive quantitative method. In collecting the data, the researcher used distributed the test to the respondents to assess the students’ ability in understanding the use of personal pronoun which consist of 24 questions. The researcher used SPSS 16.0 Program to analyze the data. The result of this research showed that the students’ ability using personal pronoun is less, because the mean score is 65.07. Key word: Analysis, Ability, Personal Pronou