A Sequence-to-Sequence Approach for Arabic Pronoun Resolution

This paper proposes a sequence-to-sequence learning approach for Arabic pronoun resolution, which explores the effectiveness of using advanced natural language processing (NLP) techniques, specifically Bi-LSTM and the BERT pre-trained Language Model, in solving the pronoun resolution problem in Arabic. The proposed approach is evaluated on the AnATAr dataset, and its performance is compared to several baseline models, including traditional machine learning models and handcrafted feature-based models. Our results demonstrate that the proposed model outperforms the baseline models, which include KNN, logistic regression, and SVM, across all metrics. In addition, we explore the effectiveness of various modifications to the model, including concatenating the anaphor text beside the paragraph text as input, adding a mask to focus on candidate scores, and filtering candidates based on gender and number agreement with the anaphor. Our results show that these modifications significantly improve the model's performance, achieving up to 81% on MRR and 71% for F1 score while also demonstrating higher precision, recall, and accuracy. These findings suggest that the proposed model is an effective approach to Arabic pronoun resolution and highlights the potential benefits of leveraging advanced NLP neural models

Cross-Lingual and Low-Resource Sentiment Analysis

Identifying sentiment in a low-resource language is essential for understanding opinions internationally and for responding to the urgent needs of locals affected by disaster incidents in different world regions. While tools and resources for recognizing sentiment in high-resource languages are plentiful, determining the most effective methods for achieving this task in a low-resource language which lacks annotated data is still an open research question. Most existing approaches for cross-lingual sentiment analysis to date have relied on high-resource machine translation systems, large amounts of parallel data, or resources only available for Indo-European languages. This work presents methods, resources, and strategies for identifying sentiment cross-lingually in a low-resource language. We introduce a cross-lingual sentiment model which can be trained on a high-resource language and applied directly to a low-resource language. The model offers the feature of lexicalizing the training data using a bilingual dictionary, but can perform well without any translation into the target language. Through an extensive experimental analysis, evaluated on 17 target languages, we show that the model performs well with bilingual word vectors pre-trained on an appropriate translation corpus. We compare in-genre and in-domain parallel corpora, out-of-domain parallel corpora, in-domain comparable corpora, and monolingual corpora, and show that a relatively small, in-domain parallel corpus works best as a transfer medium if it is available. We describe the conditions under which other resources and embedding generation methods are successful, and these include our strategies for leveraging in-domain comparable corpora for cross-lingual sentiment analysis. To enhance the ability of the cross-lingual model to identify sentiment in the target language, we present new feature representations for sentiment analysis that are incorporated in the cross-lingual model: bilingual sentiment embeddings that are used to create bilingual sentiment scores, and a method for updating the sentiment embeddings during training by lexicalization of the target language. This feature configuration works best for the largest number of target languages in both untargeted and targeted cross-lingual sentiment experiments. The cross-lingual model is studied further by evaluating the role of the source language, which has traditionally been assumed to be English. We build cross-lingual models using 15 source languages, including two non-European and non-Indo-European source languages: Arabic and Chinese. We show that language families play an important role in the performance of the model, as does the morphological complexity of the source language. In the last part of the work, we focus on sentiment analysis towards targets. We study Arabic as a representative morphologically complex language and develop models and morphological representation features for identifying entity targets and sentiment expressed towards them in Arabic open-domain text. Finally, we adapt our cross-lingual sentiment models for the detection of sentiment towards targets. Through cross-lingual experiments on Arabic and English, we demonstrate that our findings regarding resources, features, and language also hold true for the transfer of targeted sentiment

Cybernationalism and cyberactivism in China

El nacionalismo en la era de Internet se está convirtiendo cada vez más en un factor esencial que influye en la agenda-setting de la sociedad china, así como en las relaciones de China con los países extranjeros, especialmente con Occidente. Para China, una mejor comprensión de la estructura teórica universal y de los patrones de comportamiento del nacionalismo facilitaría la articulación social general de esta tendencia y potenciaría su papel positivo en la agenda-setting social. Por otra parte, un estudio del cibernacionalismo chino basado en una perspectiva china en el mundo académico occidental es un intento de transculturación. Desde el punto de vista de las relaciones internacionales y la geopolítica actuales, que son bastante urgentes, este intento ayudaría a mejorar la compatibilidad de China con el actual orden mundial dominado por Occidente, a reducir la desinformación entre China y otros países y a sentar las bases culturales e ideológicas para otras colaboraciones internacionales. Teniendo en cuenta el estado actual de la investigación sobre el nacionalismo chino y la naturaleza participativa de las masas del cibernacionalismo, esta disertación se centra en el cibernacionalismo en las tres partes siguientes. El primero es un estudio de los orígenes históricos del cibernacionalismo chino. Esta sección incluye tanto una exploración del consenso social en la antigua China como un estudio de la influencia del nacionalismo en la historia china moderna. El estudio de los orígenes históricos no sólo nos muestra la secuencia cronológica de la experiencia del desarrollo y la evolución tanto del proto-nacionalismo como del nacionalismo en China, sino que también revela un impulso decisivo para las reivindicaciones y comportamientos actuales del cibernacionalismo. La segunda parte trata del proceso de formación y ascenso del cibernacionalismo desde el siglo XXI. El importante antecedente del paso del nacionalismo al cibernacionalismo es el proceso de informatización de la sociedad china. Una vez completado el estudio de la situación básica de la sociedad china de Internet, especialmente el estudio de los medios sociales como espacio público, podemos vincular Internet con el nacionalismo y examinar el nuevo desarrollo del nacionalismo en la era de la participación de masas. El objetivo final es conectar el proto-nacionalismo, el nacionalismo y el cibernacionalismo, y seguir construyendo una comprensión del cibernacionalismo que sea coherente tanto con los principios universales del nacionalismo como con el contexto chino. Por último, validamos los resultados derivados del estudio anterior a través de la realidad social, es decir, estudiando las prácticas de ciberactivismo del cibernacionalismo para juzgar su suficiencia general así como su validez. Llevaremos a cabo varios estudios de caso de natural language processing basados en big data para reproducir la lógica de comportamiento y el impacto real del ciberactivismo de la manera más cercana posible a la realidad de Internet, evitando al mismo tiempo los defectos de argumentación unilateral y de infrarrepresentación de los estudios de caso tradicionales.Nationalism in the Internet age is increasingly becoming an essential factor influencing agendasetting within Chinese society, as well as China’s relations with foreign countries, especially the West. For China, a better understanding of the universal theoretical structure and behavioral patterns of nationalism would facilitate the overall social articulation of this trend and enhance its positive role in social agenda setting. On the other hand, a study of Chinese cybernationalism based on a Chinese perspective in western academia is an attempt at transculturation. From the viewpoint of the current rather urgent international relations and geopolitics, such an attempt would help to enhance China’s compatibility with the current western-dominated world order, reduce misinformation between China and other countries, and lay the cultural and ideological groundwork for various other international collaborations. Considering the current state of Chinese nationalism research and the mass participatory nature of cybernationalism, this dissertation focuses on cybernationalism in the following three parts. The first is a study of the historical origins of Chinese cybernationalism. This section includes both an exploration of the social consensus in ancient China and a survey of the influence of nationalism in modern Chinese history. The historical origins study not only shows us the chronological sequence of experiencing the development and evolution of both proto-nationalism and nationalism in China, but also reveals a decisive impetus for the current claims and behaviors of cybernationalism. The second part deals with the process of formation and rise of cybernationalism since the 21st century. The important background for the move from nationalism to cybernationalism is the informatization process of Chinese society. After we have completed the study of the basic situation of Chinese Internet society, especially the study of social media as a public space, we can link the Internet with nationalism and examine the new development of nationalism in the era of mass participation. The ultimate goal is to connect the proto-nationalism, nationalism, cybernationalism, and furtherly construct an understanding of cybernationalism that is consistent with both the universal principles of nationalism and the Chinese context. Finally, we validate the results derived from the previous study through social reality, i.e., by studying the cyberactivism practices of cybernationalism to judge its general sufficiency as well as validity. We will conduct several natural language processing case studies based on big data to reproduce the behavioral logic and actual impact of cyberactivism in the closest possible way to the Internet reality while avoiding the unilateral argumentation and under-representation flaws of traditional case studies

Arabic named entity recognition

En esta tesis doctoral se describen las investigaciones realizadas con el objetivo de determinar las mejores tecnicas para construir un Reconocedor de Entidades Nombradas en Arabe. Tal sistema tendria la habilidad de identificar y clasificar las entidades nombradas que se encuentran en un texto arabe de dominio abierto. La tarea de Reconocimiento de Entidades Nombradas (REN) ayuda a otras tareas de Procesamiento del Lenguaje Natural (por ejemplo, la Recuperacion de Informacion, la Busqueda de Respuestas, la Traduccion Automatica, etc.) a lograr mejores resultados gracias al enriquecimiento que a~nade al texto. En la literatura existen diversos trabajos que investigan la tarea de REN para un idioma especifico o desde una perspectiva independiente del lenguaje. Sin embargo, hasta el momento, se han publicado muy pocos trabajos que estudien dicha tarea para el arabe. El arabe tiene una ortografia especial y una morfologia compleja, estos aspectos aportan nuevos desafios para la investigacion en la tarea de REN. Una investigacion completa del REN para elarabe no solo aportaria las tecnicas necesarias para conseguir un alto rendimiento, sino que tambien proporcionara un analisis de los errores y una discusion sobre los resultados que benefician a la comunidad de investigadores del REN. El objetivo principal de esta tesis es satisfacer esa necesidad. Para ello hemos: 1. Elaborado un estudio de los diferentes aspectos del arabe relacionados con dicha tarea; 2. Analizado el estado del arte del REN; 3. Llevado a cabo una comparativa de los resultados obtenidos por diferentes tecnicas de aprendizaje automatico; 4. Desarrollado un metodo basado en la combinacion de diferentes clasificadores, donde cada clasificador trata con una sola clase de entidades nombradas y emplea el conjunto de caracteristicas y la tecnica de aprendizaje automatico mas adecuados para la clase de entidades nombradas en cuestion. Nuestros experimentos han sido evaluados sobre nueve conjuntos de test.Benajiba, Y. (2009). Arabic named entity recognition [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/8318Palanci

Re-constructing the nation: struggles in portraying minority ethnic groups in Chinese mainstream history textbooks

This thesis examines the changes to the portrayal of minority ethnic groups in Chinese history textbooks since the establishment of the People’s Republic of China in 1949. It finds that ideological shifts in Beijing have led to minority ethnic groups being portrayed in changing and even contradictory ways in school textbooks. In the history textbooks of the 1950s, the Chinese nation was largely defined as a Han nation-state, and other ethnic groups were generally represented as non-Chinese who had historically been ‘threats’ or ‘enemies’ of the Han/Chinese. It was not until the reform era from the late 1970s that a more inclusive and multi-ethnic conception of the Chinese nationhood was adopted, with ‘minority’ ethnic groups incorporated into the Chinese historical narrative and portrayed more positively. However, as the Communist Party took an increasingly nationalist turn from the 1990s, simultaneously downplaying messages of socialist internationalism, Han ethno-centrism became more apparent once again in textbook narratives, with minority ethnic groups correspondingly marginalised. This thesis also finds that, although non-Han groups were portrayed very differently in history textbooks to match shifting political ideologies, what remained unchanged throughout PRC history was the representation of the backwardness of the non-Han in relation to the Han who were always portrayed as advanced. Based on this examination, this thesis argues that while history education has always been used by the Communist Party to inculcate a highly state-centred vision of national identity, underlying conceptions of the Chinese nationhood have been rather fluid, and there has been no consistent progress towards a more inclusive notion of ‘Chineseness’. Instead, different visions have co-existed and competed, reflecting tensions inherent in the project of constructing modern national consciousness: China has struggled (and is still struggling) to stretch the short, tight skin of the nation over the gigantic body of its empire

Inductive Bias and Modular Design for Sample-Efficient Neural Language Learning

Most of the world's languages suffer from the paucity of annotated data. This curbs the effectiveness of supervised learning, the most widespread approach to modelling language. Instead, an alternative paradigm could take inspiration from the propensity of children to acquire language from limited stimuli, in order to enable machines to learn any new language from a few examples. The abstract mechanisms underpinning this ability include 1) a set of in-born inductive biases and 2) the deep entrenchment of language in other perceptual and cognitive faculties, combined with the ability to transfer and recombine knowledge across these domains. The main contribution of my thesis is giving concrete form to both these intuitions. Firstly, I argue that endowing a neural network with the correct inductive biases is equivalent to constructing a prior distribution over its weights and its architecture (including connectivity patterns and non-linear activations). This prior is inferred by "reverse-engineering" a representative set of observed languages and harnessing typological features documented by linguists. Thus, I provide a unified framework for cross-lingual transfer and architecture search by recasting them as hierarchical Bayesian neural models. Secondly, the skills relevant to different language varieties and different tasks in natural language processing are deeply intertwined. Hence, the neural weights modelling the data for each of their combinations can be imagined as lying in a structured space. I introduce a Bayesian generative model of this space, which is factorised into latent variables representing each language and each task. By virtue of this modular design, predictions can generalise to unseen combinations by extrapolating from the data of observed combinations. The proposed models are empirically validated on a spectrum of language-related tasks (character-level language modelling, part-of-speech tagging, named entity recognition, and common-sense reasoning) and a typologically diverse sample of about a hundred languages. Compared to a series of competitive baselines, they achieve better performances in new languages in zero-shot and few-shot learning settings. In general, they hold promise to extend state-of-the-art language technology to under-resourced languages by means of sample efficiency and robustness to the cross-lingual variation.ERC (Consolidator Grant 648909) Lexical Google Research Faculty Award 201