Reflection - quantifying a rare good

Based on a literature review, reflections in written text are rare. The reported proportions of reflection are based on different baselines, making comparisons difficult. In contrast, this research reports on the proportion of occurrences of elements of reflection based on sentence level. This metric allows to compare proportions of elements of reflection. Previous studies are based on courses tailored to foster reflection. The reported proportions represent more the success of a specific instruction than informing about proportions of reflections occurring in student writings in general. This study is based on a large sample of course forum posts of a virtual learning environment. In total 1000 sentences were randomly selected and manually classified according to six elements of reflection. Five raters rated each sentence. Agreement was calculated based on a majority vote. The proportions of elements of reflection are reported and its potential application for course analytics demonstrated. The results indicate that reflections in text are indeed rare, and that there are differences within elements of reflection

Automated detection of reflection in texts. A machine learning based approach

Promoting reflective thinking is an important educational goal. A common educational practice is to provide opportunities for learners to express their reflective thoughts in writing. The analysis of such text with regard to reflection is mainly a manual task that employs the principles of content analysis. Considering the amount of text produced by online learning systems, tools that automatically analyse text with regard to reflection would greatly benefit research and practice. Previous research has explored the potential of dictionary-based approaches that automatically map keywords to categories associated with reflection. Other automated methods use manually constructed rules to gauge insight from text. Machine learning has shown potential for classifying text with regard to reflection-related constructs. However, not much is known of whether machine learning can be used to reliably analyse text with regard to the categories of reflective writing models. This thesis investigates the reliability of machine learning algorithms to detect reflective thinking in text. In particular, it studies whether text segments from student writings can be analysed automatically to detect the presence (or absence) of reflective writing model categories. A synthesis of the models of reflective writing is performed to determine the categories frequently used to analyse reflective writing. For each of these categories, several machine learning algorithms are evaluated with regard to their ability to reliably detect reflective writing categories. The evaluation finds that many of the categories can be predicted reliably. The automated method, however, does not achieve the same level of reliability as humans do

HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models

Fairness has become a trending topic in natural language processing (NLP), which addresses biases targeting certain social groups such as genders and religions. However, regional bias in language models (LMs), a long-standing global discrimination problem, still remains unexplored. This paper bridges the gap by analysing the regional bias learned by the pre-trained language models that are broadly used in NLP tasks. In addition to verifying the existence of regional bias in LMs, we find that the biases on regional groups can be strongly influenced by the geographical clustering of the groups. We accordingly propose a HiErarchical Regional Bias evaluation method (HERB) utilising the information from the sub-region clusters to quantify the bias in pre-trained LMs. Experiments show that our hierarchical metric can effectively evaluate the regional bias with respect to comprehensive topics and measure the potential regional bias that can be propagated to downstream tasks. Our codes are available at https://github.com/Bernard-Yang/HERB.Comment: Accepted at AACL 2022 as Long Finding

Proceedings of the 5th Workshop on Awareness and Reflection in Technology Enhanced Learning

Awareness and reflection are viewed differently across the disciplines informing Technology Enhanced Learning (CSCW, psychology, educational sciences, computer science and others). The ARTEL workshop series brings together researchers and professionals from different backgrounds to provide a forum for discussing the multi-faceted area of awareness and reflection. Through the last ARTEL workshops at EC-TEL the addressed topics are converging towards the usage of awareness and reflection in practice, its implementation in modern organisations, its impact on learners and questions of feasibility and sustainability for awareness and reflection in education and work. To reflect the growing maturity of research in ARTEL over the years the workshop particularly invited contributions that dealt with the application of awareness and reflection in practice. This is encapsulated in the workshop motto: 'Awareness and Reflection in Practice: How can awareness and reflection technology become common in work practice and how does it change work practices?

Translatorische und außertranslatorische Automatizität: Eine integrative Darstellung

The paper examines empirically a subset of cognitive processes in trainee translators with the objective of gaining an insight into their decision-making. Specifically, we are interested in the nature and role of automated processing – above all, how pronounced it can be and how it influences the quality of decisions. The paper’s objective is then to come up with an integrative view of the relationship between translatorial automaticity and cognitive automaticity in general, viz. that not associated with translation. This could help us better capture some of the characteristics of translator behaviour and supplement our understanding of translation competence. Results from experiments with trainees reported in the paper show no correlation between the two dimensions of automated processing, and indicate that translatorial automaticity could be harder to override than its more general counterpart.Zweck dieser Abhandlung ist die empirische Untersuchung einer Teilmenge kognitiver Prozesse bei Übersetzern in der Ausbildung mit dem Ziel, einen Einblick in deren Entscheidungsfindung zu gewinnen. Dabei ist insbesondere die Art und Funktion der automatisierten Verarbeitung von Interesse – v. a. wie ausgeprägt diese sein kann und wie sie die Qualität von Entscheidungen beeinflusst. Ein weiteres Ziel der Abhandlung ist es, zu einer integrativen Sicht auf die Beziehung zwischen translatorischer Automatizität und kognitiver Automatizität allgemein, d. h. nicht im Zusammenhang mit Übersetzung, zu gelangen. Dies könnte zur besseren Erfassung bestimmter Merkmale des Übersetzerverhaltens beitragen und unser Verständnis der Übersetzungskompetenz verbessern. Die in der Abhandlung erläuterten Ergebnisse aus Experimenten mit Übersetzern in der Ausbildung weisen auf keinen Zusammenhang zwischen den beiden Dimensionen der automatisierten Verarbeitung hin und zeigen, dass sich die Ausschaltung der translatorischen Automatizität im Gegensatz zu der Ausschaltung ihres allgemeineren Gegenstücks als schwieriger erweisen könnte