Effective and Efficient Transfer Learning in the Era of Large Language Models

Substantial progress has been made in the field of natural language processing (NLP) due to the advent of large language models (LLMs)—deep neural networks with millions or billions of parameters pre-trained on large amounts of unlabeled data. However, these models have common weaknesses, including degenerate performance in data-scarce scenarios, and substantial computational resource requirements. This thesis aims to develop methods to address these limitations for improved applicability and performance of LLMs in resource-constrained settings with limited data and/or computational resources. To address the need for labeled data in data-scarce scenarios, I present two methods, in Chapter 2 and Chapter 3, respectively. The first method leverages beneficial relationships between NLP tasks for transfer learning, while the second method combines data augmentation and self-training to boost few-shot learning performance—the ability to perform novel tasks from only a few labeled examples. Additionally, in Chapter 4, I introduce a novel parameter-efficient transfer learning approach that reuses a single frozen model for all tasks while only learning minimal task-specific parameters (soft/continuous prompts) to represent tasks and transfer knowledge. Our method can match or outperform fine-tuning task-specific models (training the whole model on each task). In Chapter 5, I demonstrate the benefits of parameter-efficient transfer learning in a cross-lingual transfer setting. Finally, I conclude the thesis in Chapter 6 by outlining potential avenues for future research that aim to advance NLP through large-scale multi-task learning using multilingual and multimodal data

Acculturation of Russian Refugee Adolescents: The Life Domain of Peer Relationships

The purpose of this qualitative study was to uncover the meaning of acculturation as experienced by the Russian refugee adolescents in the domain of peer relationships. This qualitative study implemented a purposeful sampling strategy. In-depth interviews were conducted with 12 refugee adolescents from Russia (3 ethnic Russians and 9 Meskhetian Turks), male and female, aged 15-18, who resided in Denver, Colorado. Applying Moustakas\u27s (1994) phenomenology method of analysis, 8 main themes emerged. The essence of the phenomenon can be described in terms of the refugee adolescents\u27 need for self-worth and belongingness. Belongingness is understood as identification with and acceptance by peers of the culture of origin and host culture in a culture-contact situation upon immigration. It was found that exclusion by peers pushed refugee adolescents to search for restoration of dignity, acceptance, and identification with other culture(s) available to them. The findings revealed important factors moderating acculturation to the American culture in the peer relationships domain: refugee adolescents\u27 perception of pre-migration experience, a limited pool of potential partners, deprivation of the adult status, perceived discrepancy in the level of maturity between themselves and their American peers, cultural discrepancies in understanding friendship and ways of courtship, English language competence/use/preference, and perceived discrimination/negative treatment by part of the American peers from the dominant culture. In particular, negative treatment received from members of the dominant American culture was identified as a factor moderating acculturation to non-dominant American cultures in the peer relationships domain, and along with other factors, served as a basis for identification of the participants with the heterogeneous group of immigrant/minority youth from different countries who perceived similar negative treatment from their American counterparts. Perceived negative treatment was found to be a powerful risk factor, creating conditions for reactive identity formation towards the American culture, idealization of an oppositional culture, and self-radicalization. Implications for social work education, research, and practice, as well as future research opportunities are suggested

Ontology model for zakat hadith knowledge based on causal relationship, semantic relatedness and suggestion extraction

Hadith is the second most important source used by all Muslims. However, semantic ambiguity in the hadith raises issues such as misinterpretation, misunderstanding, and misjudgement of the hadith’s content. How to tackle the semantic ambiguity will be focused on this research (RQ). The Zakat hadith data should be expressed semantically by changing the surface-level semantics to a deeper sense of the intended meaning. This can be achieved using an ontology model covering three main aspects (i.e., semantic relationship extraction, causal relationship representation, and suggestion extraction). This study aims to resolve the semantic ambiguity in hadith, particularly in the Zakat topic by proposing a semantic approach to resolve semantic ambiguity, representing causal relationships in the Zakat ontology model, proposing methods to extract suggestion polarity in hadith, and building the ontology model for Zakat topic. The selection of the Zakat topic is based on the survey findings that respondents still lack knowledge and understanding of the Zakat process. Four hadith book types (i.e., Sahih Bukhari, Sahih Muslim, Sunan Abu Dawud, and Sunan Ibn Majah) that was covering 334 concept words and 247 hadiths were analysed. The Zakat ontology modelling cover three phases which are Preliminary study, source selection and data collection, data pre-processing and analysis, and development and evaluation of ontology models. Domain experts in language, Zakat hadith, and ontology have evaluated the Zakat ontology and identified that 85% of Zakat concept was defined correctly. The Ontology Usability Scale was used to evaluate the final ontology model. An expert in ontology development evaluated the ontology that was developed in Protégé OWL, while 80 respondents evaluated the ontology concepts developed in PHP systems. The evaluation results show that the Zakat ontology has resolved the issue of ambiguity and misunderstanding of the Zakat process in the Zakat hadith. The Zakat ontology model also allows practitioners in Natural language processing (NLP), hadith, and ontology to extract Zakat hadith based on the representation of a reusable formal model, as well as causal relationships and the suggestion polarity of the Zakat hadith

The Normalization Process of Multimodal Composition: The Unseeing People of Color

This study attempts to identify normalization cues within multimodal scholarship to highlight moments of un-seeing multimodal composing practices and theoretical contributions from non-Western traditions. Advocates of this approach to teaching composition understand it as an effective way for incorporating other voices into the curricular structures of composition courses. However, the instructional resources do not include or cite research that does not lend itself easily to dominant views of composing within academia. I assert that academia must go further with how value is assessed. There is research that acknowledges the multiliteracies practices found within subcultures of America, and plenty of work that deems the communicative practices observed in these subcultural communities as valuable. However, it is more than just including and citing scholarship from and about people of color\u27s compositional practices, academia must also employ these ways of knowing and being to fully empower students and utilize the knowledge that the students bring with them to the FYC classroom. The dominant assignment genre in academia is the academic essay. Other dominant methods of communication and transferring scholarship are the journal article, annotated bibliography, proposal, and personal essay. Not to mention the many scholars who have critiqued academia for privileging print literacies, which although may be multimodal, promotes a multimodality of one culture and ideological standpoint. Although the seminal texts from the study offer exceptional multimodal composition research and classroom resources, if we can agree that the mission of education…is to ensure that all students benefit from learning in ways that allow them to participate fully in public, community, and economic life and that literacy pedagogy, essentially what the FYC course offers, is expected to play a particularly important role in fulfilling this mission, then failing to see the value and utilize the scholarship from and about people of color ensures those that are marginalized continue to be un-seen and students remain unprepared for the tasks of composing and communicating outside of school (New London Group 60)

European Distance and E-Learning Network (EDEN). Conference Proceedings

Erasmus+ Programme of the European UnionThe powerful combination of the information age and the consequent disruption caused by these unstable environments provides the impetus to look afresh and identify new models and approaches for education (e.g. OERs, MOOCs, PLEs, Learning Analytics etc.). For learners this has taken a fantastic leap into aggregating, curating and co-curating and co-producing outside the boundaries of formal learning environments – the networked learner is sharing voluntarily and for free, spontaneously with billions of people.Supported by Erasmus+ Programme of the European Unioninfo:eu-repo/semantics/publishedVersio