Online peer assessment in higher education: a systematic review of literature in educational practices

Neste artigo procedemos a uma revisão sistemática da literatura sobre Online Peer Assessment (OPA) no Ensino Superior, mediadas por Tecnologias Digitais de Informação e Comunicação (TIC). Identificamos suas características, centrais e opcionais e mapeamos exemplos práticos (procedimentos e TIC) de OPA que poderão ser transversais, adaptáveis e aplicáveis em diversas unidades curriculares e regimes educacionais. Os resultados apontam para a utilização da OPA como uma estratégia que potencia a “avaliação para a aprendizagem”. Os referenciais teóricos subjacentes, os métodos de avaliação e os tipos de TIC utilizadas indicam seu direcionamento para um maior envolvimento e responsabilidade do aluno na sua aprendizagem. Evidenciamos na literatura que se busca desenvolver essas competências, dando ao aluno oportunidades com alguma regularidade para autoavaliar-se e avaliar seus pares por meio de feedback construtivo. Há também evidências de que dar ou produzir feedback é mais benéfico para a aprendizagem do que apenas recebê-lo, como também é cognitivamente mais exigente e envolve os alunos de forma mais ativa e os direciona ao pensamento crítico e a processos metacognitivos. A partir dos resultados alcançados, consideramos que a OPA, enquanto ferramenta cognitiva, contribui para a construção do conhecimento e para a reflexão sobre a aprendizagem. Um desafio que se coloca diz respeito ao desenvolvimento criativo com foco na diversificação e na inovação das práticas de avaliação no sentido de potenciar aprendizagens e resultados acadêmicos, em atenção às necessidades de aprendizagem que se manifestam frente às expectativas da educação atual e futura e às exigências da sociedade.In this research a systematic review of literature about Online Peer Assessment (OPA) in higher education, mediated by Information and Communication Technology (ICT), was carried out. Optional and central characteristics and its practices (procedures and ICT) were identified and mapping. These procedures and practices may be transversal, adaptable and applied in several curriculum units and educational regimes. The results point to the use of OPA as a strategy that enhances “assessment for learning”. The theoretical framework, the methods of assessment and the kinds of ICT used indicate directions for greater involvement and responsibility from the part of the student in his/her learning. Theoretical framework identifies the need to develop students’ skills, providing them with opportunities for self-assessment and peer assessment on a regular basis through constructive feedback. There is also evidence that to give or receive feedback is more beneficial for learning than just receiving it. It is also cognitively more demanding and directs the students to critical thinking and metacognitive processes. From the results obtained, OPA is seen as a cognitive tool, contributing to the building of knowledge and to reflection about learning. Issues for further reflection are also identified such as the creative development of assessment methods with a focus on diversity and innovation in order to enhance students’ learning and academic results, taking into account their learning needs in face of the expectations of current and future education and the demands of society.CIEC - Centro de Investigação em Estudos da Criança, IE, UMinho (UI 317 da FCT), PortugalFundos Nacionais através da FCT (Fundação para a Ciência e a Tecnologia) e cofinanciado pelo Fundo Europeu de Desenvolvimento Regional (FEDER) através do COMPETE 2020 – Programa Operacional Competitividade e Internacionalização (POCI) com a referência POCI-01-0145-FEDER-007562info:eu-repo/semantics/publishedVersio

Assessing a Collaborative Online Environment for Music Composition

The current pilot study tested the effectiveness of an e-learning environment built to enable students to compose music collaboratively. The participants interacted online by using synchronous and asynchronous resources to develop a project in which they composed a new music piece in collaboration. After the learning sessions, individual semi-structured interviews with the participants were conducted to analyze the participants\u2019 perspectives regarding the e-learning environment\u2019s functionality, the resources of the e-learning platform, and their overall experience with the e-learning process. Qualitative analyses of forum discussions with respect to metacognitive dimensions, and semi-structured interview transcriptions were performed. The findings showed that the participants successfully completed the composition task in the virtual environment, and that they demonstrated the use of metacognitive processes. Moreover, four themes were apparent in the semi-structured interview transcriptions: Teamwork, the platform, face-to-face/online differences, and strengths/weaknesses. Overall, the participants exhibited an awareness of the potential of the online tools, and the task performed. The results are discussed in consideration of metacognitive processes, and the following aspects that rendered virtual activity effective for learning: The learning environment, the platform, the technological resources, the level of challenge, and the nature of the activity. The possible implications of the findings for research on online collaborative composition are also considered

Uma análise do uso de avaliações por pares e rubricas na promoção de interação entre alunos em uma disciplina de graduação em engenharia de computação

Este artigo apresenta uma análise sobre o uso da combinação das técnicas deavaliação por pares e de rubricas na promoção de interação entre alunos em práticas de ensino baseadas em problemas numa disciplina de graduação em Engenharia de Computação. A metodologia de ensino adotada na disciplina se estrutura no preenchimento de fichas de avaliação pelos alunos de acordo com rubricas previamente disponibilizadas pelo instrutor. Essa metodologia foi aplicada para uma turma de 25 alunos, tendo sido analisados os comentários produzidos nas fichas de avaliação e os trabalhos desenvolvidos ao longo do semestre. Como resultado, conclui-se que o emprego dessas técnicas tem mostrado a efetivação de um ambiente educacional favorável à interação, verificando-se, inclusive, a implementação de itens não obrigatórios nos trabalhos desenvolvidos pelos alunos

Fuzzy model for peer assessment

Dentro del proceso de enseñanza-aprendizaje se tiene la evaluación que implica diversos mecanismos generados por el docente para su ejecución, como identificar los elementos a evaluar, definir el proceso donde se va a llevar a cabo, recopilar la información y finalmente obtener la valoración de los elementos. El uso de rúbricas en el proceso de evaluación por pares, permite valorar los logros alcanzados por los estudiantes en las diferentes actividades planteadas, pero éstas deben garantizar su validez y fiabilidad para poder ser consideradas como herramientas de apoyo al docente. Esta investigación abordó el tema de un “Modelo difuso para la evaluación por pares”, con el objetivo principal de desarrollar un modelo de clasificación difuso de validación de rúbricas en el proceso de evaluación por pares; a fin de facilitar la toma de decisiones en la selección adecuada de rúbricas en los procesos de enseñanza aprendizaje de la Educación Superior, mediante la utilización de técnicas avanzadas de minería de datos. Se planteo una metodología con el enfoque investigación-acción (ADR), la cual se realizó en tres iteraciones. En la primera iteración se analizó las tendencias en la selección de rúbricas en el contexto de evaluación por pares, participantes, dominios de aplicación y ambientes de aprendizaje con el fin de analizar el uso de las rúbricas en la evaluación de pares dentro del proceso de aprendizaje. En la segunda iteración se diseñaron los artefactos del modelo de clasificación difuso en la evaluación por pares, para lo cual se analizaron los pasos para el diseño de rúbricas, con el proceso de validez y fiabilidad de la rúbrica. Obteniendo una rúbrica evaluada que reúne los criterios que sustentan la validez de su contenido, la validez de comprensión y la fiabilidad de su consistencia interna. En la tercera iteración se diseñó el modelo de clasificación difuso utilizando minería de datos educativa y aprendizaje automático, mediante la metodología de agrupamiento para definir los clústeres. Luego, mediante la lógica difusa C-means, se determina la selección de rúbrica para evaluación por pares, ayudando al docente en su selección. Para validar la metodología difusa se utilizó datos generados en laboratorio con valores aleatorios, se aplicaron los algoritmos C-means, K-means y Jerárquico. Para comprobar la selección adecuada de los clústeres y la validación del algoritmo difuso, se lo realizó mediante la evaluación interna de los clústeres con el Coeficiente de Silueta (0.2904) y el Coeficiente de Dunn (0.24127). Mediante el índice de Dunn se demostró que el mejor algoritmo es C-means y con el índice de Silueta, que el mejor es K-means. Finalmente, en la aplicación de los tres algoritmos se obtuvieron resultados de similitud estadísticamente significativos para la selección de rúbricas. De acuerdo con el análisis realizado, se concluye que es una herramienta muy importante en la toma de decisiones el uso de las rúbricas en el proceso de evaluación por pares mediante la utilización de la tecnología y TIC. Las rúbricas deben ser de calidad demostrando su validez a través de la correspondencia entre las categorías que la conforman y demostrando su confiabilidad mediante la concordancia entre las calificaciones de los estudiantes y del docente (alfa de Cronbach>0.8). El modelo difuso desarrollado permitió la selección adecuada de los agrupamientos, la selección del clúster donde se encuentran las rúbricas recomendadas, quedando en evidencia que la lógica difusa resulta adecuada en los procesos de selección de rúbricas para evaluación por pares, de igual manera que los algoritmos K-means y Jeráquica. Se espera que este estudio contribuya a la selección de la rúbrica adecuada en el proceso de evaluación por pares, que realiza el docente universitario dentro de su proceso de enseñanza aprendizaje.Within the teaching-learning process there is the evaluation that implies various mechanisms generated by the teacher for its execution, such as identifying the elements to be evaluated, the process where it is going to be carried out, collecting the information and finally obtaining the evaluation of the results items. The use of rubrics in the peer assessment process allows the assessment of the achievements of the students in the different activities proposed, but they must guarantee their validity and reliability in order to be considered as support tools for the teacher. This research addressed the issue of a "Fuzzy Model for Peer Assessment", with the main objective of developing a fuzzy classification model for rubric validation in the peer assessment process; in order to facilitate decision-making in the appropriate selection of rubrics in the teaching-learning processes of Higher Education, through the use of advanced data mining techniques. A methodology was proposed with the action research (ADR) approach, which was carried out in three iterations. In the first iteration, trends in the selection of rubrics were analyzed in the context of peer assessment, participants, application domains, and learning environments in order to analyze the use of rubrics in peer assessment within the learning process. . In the second iteration, the artifacts of the fuzzy classification model in the peer evaluation were designed, for which the steps for the design of rubrics were analyzed, with the process of validity and reliability of the rubric. Obtaining an evaluated rubric that meets the criteria that support the validity of its content, the validity of comprehension and the reliability of its internal consistency. In the third iteration, the fuzzy classification model was designed using educational data mining and machine learning, using the clustering methodology to define the clusters. Then, using C-means fuzzy logic, the selection of the rubric for peer evaluation is determined, helping the teacher in their selection. To validate the fuzzy methodology, data generated in the laboratory with random values was used, the C-means, K-means and Hierarchical algorithms were applied. In order to verify the adequate selection of the clusters and the validation of the fuzzy algorithm, it was carried out through the internal evaluation of the clusters with the Silhouette Coefficient (0.2904) and the Dunn Coefficient (0.24127). Using the Dunn index, it was shown that the best algorithm is C-means and with the Silhouette index, that the best is K-means. Finally, in the application of the three algorithms, statistically significant similarity results were obtained for the selection of rubrics. According to the analysis carried out, it is concluded that the use of rubrics in the peer evaluation process through the use of technology and ICT is a very important tool in decisión making. The rubrics must be of quality, demonstrating their validity through the correspondence between the categories that comprise them and demonstrating their reliability through the concordance between the student's and the teacher's qualifications (Cronbach's alpha>0.8). The fuzzy model developed allowed the adequate selection of the groupings, the selection of the cluster where the recommended rubrics are located, making it evident that the fuzzy logic is adequate in the rubric selection processes for peer evaluation, in the same way that the algorithms K-means and Hierarchical. It is expected that this study contributes to the selection of the appropriate rubric in the peer assessment process carried out by the university professor within his teaching-learning process

A ferramenta Wiki como veículo potencializador da avaliação formativa

Tese de mestrado, Educação (Área de especialidade em Educação e Tecnologias Digitais), Universidade de Lisboa, Instituto de Educação, 2015A geração atual de alunos tem crescido num mundo imerso em tecnologia e esta situação irá, sem dúvida, perdurar no futuro.A ferramenta wiki é uma ferramenta digital colaborativa, acessível a partir de um browser, que é possível utilizar sem necessidade de conhecimentos de programação. Os wikis constituem-se como espaços digitais que favorecem a aprendizagem por interação, discussão de ideias e construção conjunta de conhecimento. A investigação desenvolvida tem como finalidade conhecer como o uso da ferramenta wiki para a promoção da avaliação formativa, pode potencializar a construção de conhecimento de alunos do 9.º ano, na temática eletricidade. No âmbito desta problemática foram identificadas as seguintes questões orientadoras: conhecer como é que a interação entre pares potencializa a construção do conhecimento dos alunos numa ferramenta colaborativa como o wiki; conhecer como é que o feedback do professor potencializa a construção do conhecimento dos alunos numa ferramenta colaborativa como o wiki e que potencialidades e limitações os alunos reconhecem sobre a importância do feedback e interação entre pares para a promoção da construção do seu conhecimento nesta ferramenta. Neste contexto, desenvolveu-se um processo de investigação sobre a utilização da ferramenta wiki para a promoção de mobilização de conhecimento científico, através da interação entre pares e feedback do professor. Esta investigação foi efetuada no ano letivo de 2014/2015, numa escola do 3.º ciclo do ensino básico do ensino público, com duas turmas do 9.º ano, envolvendo um total de 51 alunos, através do desenvolvimento de tarefas sobre eletricidade, apresentadas na forma de desafios. Como instrumentos de recolha de dados recorreu-se às produções escritas dos alunos e feedback escrito da professora no wiki, e a duas entrevistas em grupo focado. Os resultados sugerem que o processo de avaliação formativo adequa-se a este tipo de ambientes, pois permite uma harmonização às necessidades de ritmos e níveis de aprendizagem distintos recorrentes da individualidade de cada aluno. Preconiza-se também que este ambiente digital facilita a partilha, a negociação, a troca de ideias e a avaliação de pares. A ferramenta permite ainda um feedback do professor dinâmico e interativo, facilitando a incorporação, ao longo do processo de aprendizagem, de situações de avaliação formativa. Os alunos reconhecem que a ferramenta proporciona um ambiente transparente, onde podem comentar e orientar os trabalhos dos colegas, e receber orientação tanto dos colegas como do professor. Parece-nos, assim, ser um trabalho a incentivar, quer porque fomenta a autonomia, a colaboração e a autorregulação, quer porque possibilita desenvolver competências contextualizadas, passíveis de serem monitorizadas pelo professor.The current generation of students has grown in a world immersed in technology and this situation will undoubtedly continue in the future. Wikis are collaborative digital tools, accessible from a browser, which you can use without the need for programming knowledge. Wikis are digital spaces that promote learning through interaction, discussion of ideas and joint construction of knowledge. The developed research aims to understand how the use of a wiki for the promotion of formative assessment, can enhance the construction of knowledge of students in the 9th grade, under the theme electricity. Under this problem, the following guiding questions were identified: how peer interaction enhances the construction of students' knowledge in a collaborative tool like a wiki; how teacher feedback enhances the construction of students' knowledge in a collaborative tool like a wiki and potentialities and limitations recognize by the students about the importance of feedback and peer interaction to promote the construction of knowledge in this tool. In this context, it was developed a process of research using the tool wiki for the promotion of mobilization of scientific knowledge through peer interaction and teacher feedback. This research was conducted in the school year 2014/2015, in a school of the 3rd cycle of basic education of public school, with two classes of the 9th year, involving a total of 51 students, through the development of tasks concerning electricity, and presented as challenges. The data collection instruments used were the written productions of the students and written feedback from the teacher on the wiki, and two focus group interviews. The results suggest that the formative evaluation process is suitable for this type of environment, since it allows harmonizing the needs of different rhythms and recurrent learning levels. It also indicates that this type of digital environment facilitates sharing, negotiation, exchange of ideas and peer assessment. The tool also allows dynamic and interactive feedback from the teacher, facilitating the incorporation, in the process of learning, of formative assessment situations. Students recognize that the tool provides a transparent environment, where they can review and guide the work of colleagues, and receive guidance form colleagues and the teacher. It seems, therefore, to be a work to encourage since it fosters autonomy, collaboration and self-regulation, as well as it enables the development of contextualized skills that can be monitored by the teacher

A case study of floods in Sri Lanka

학위논문(박사)--서울대학교 대학원 :사회과학대학 지리학과,2020. 2. 이건학.본 연구는 사회·경제적 네트워크에 대한 주요 견해와 스리랑카 농촌 및 도시 지역에서 홍수 범람의 취약성을 개선에 대한 영향을 다루고 있다. 남아사이 중산층 개발도상국에 속하는 스리랑카는 수십 년 전부터 폭우와 이로 인한 홍수로 인해 여러 부작용을 겪어 왔다. 이에 스리랑카 사회에서는 강력한 사회경제적 네트워크와 상호 호혜적인 인력 동원 및 협력과 이타적 성격으로 홍수로 인한 여러 부작용을 이겨 내었다. 이러한 맥락에서 본 연구는 스리랑카의 홍수 범람에 대한 사회경제적 네트워크의 지리와 관련된 사회적 자본 행위자에 대해 파악하고자 한다. 이에 대한 실증적 연구를 진행하기 위해 스리랑카의 21개의 행정 단위(Grama Niladhari Division, GND)에서 거주하고 있는 405개의 침수 가구를 대상으로 설문 조사를 진행하였다. 설문 조사 대상은 크게 농촌 지역과 도시 지역으로 나눌 수 있으며, 농촌 지역의 경우 15개 GND에 소속된 327 가구, 도시 지역의 경우 6개 GND에 속한 78가구가 이에 해당한다. 또한 추가적인 데이터 수집을 위해 13개월에 걸친 가계 설문 조사, 비공식 인터뷰, 포커스 그룹 토론, 현장 관찰을 수행하였다. 주요 연구 방법으로는 사회적 네트워크 분석 기법, 사회적 취약성 다변량 지수를 사용하였으며, 일부 질적 연구 방법을 추가적으로 적용하였다. 연구 결과 네트워크의 성격과 이에 대한 측정치는 시간에 따라 서로 다른 방향 및 크기로 변화 하였으며, 시간과 지리적 공간에 따른 네트워크 연결의 진화적 변화도 관찰할 수 있었다. 특히, 모든 시골 지역 GND와 관련하여 밀도, 근접성, 중심성과 같은 주요 네트워크 측정치들은 홍수 발생 전 단계 및 발생 단계에서는 감소세를 보이다가, 홍수 발생 이후에는 증가세를 보였다. 이와는 대조적으로 도시에서는 홍수 발생 이전과 발생 단계에서는 주요 네트워크 측정치가 증가세를 보였지만, 발생 이후에는 감소하는 경향을 보였다. 또한, 네트워크 클러스터는 시골 및 도시 지역 모두 홍수 발생 이전 시점과 발생 시점에서 보여지고 있으며 발생 이후에는 감소하는 것으로 나타났다. 네트워크 구조는 대부분의 시골 지역에서 홍수 발생 이후에 지원 네트워크의 조직화가 더욱 짜임새 있는 형태로 전개되었다. 반대로 모든 네트워크의 구조적인 변화는 상호 지원 유대의 행동과 특징에만 기초하고 있다. 사회적 네트워크는 홍수 발생의 이전과 밣생 및 이후의 시기에 있어 홍수에 대응하는데 중심 역할을 해왔다. 특히 정보, 음식, 물 및 기타 생필품의 제공, 살림살이에 대한 청소와 이동, 대피소 제공, 살림살이를 다시 집으로 옮기는 과정, 오염된 주거 환경과 공공 장소에 대한 청소, 각종 재정적 지원의 경우 홍수로 입은 피해를 복구하는데 큰 도움이 된 것으로 나타났다. 더욱 중요한 것은 지원 네트워크 행동에 있어 주요한 차이점이 도시와 지방 그리고 작은 규모의 시골 지역들 사이에서 관찰된다는 점이다. 즉, 홍수 사건에서의 지역 지원 네트워크 행동은 도시 네트워크 행동이 다양한 방식으로 다양화되는 것과는 극명한 대조를 이룬다. 즉, 도시에 비해 농촌에서 더 깊은 유대를 바탕으로 한 지원이 이루어지고 있었다. 또한 본 연구는 마을 주민들이 홍수에 대응하기 위하여 사회경제적 네트워크를 강하게 구축하였고, 이에 대한 사회자본의 역사적 배경이 매우 깊다는 것을 관찰하였다. 홍수 범람에 대한 사회적 취약성 점수는 매우 다양한 분포를 보였다. Kuruwita의 가구 표본은 Elapatha, Colombo의 가구 표본에 비하여 상당히 낮은 수준의 취약성을 보였다. 구체적으로 Kuruwita, Elapatha, Colombo DSD에서 평균 취약성 점수는 0.39(최소: 0.01, 최대: 0.875), 0.48(최소: 0.113, 최대: 0.996). 0.56(최소: 0.211, 최대: 0.9999)으로 각각 관찰되었다. 또한 취약성 점수에 대한 지역적 다양성도 관찰된다. 취약성 분포의 공간적 패턴에 대한 차이 또한 취약성 지도(가구 및 GND 수준)를 통해 확인할 수 있었다.This dissertation coupled together with the considering dominating ideas of socio-economic networks, related social capital and their influences on the ameliorating social vulnerability to flood inundation in rural and urban areas of Sri Lanka. Sri Lanka as a South Asian middle-income developing country has been experiencing adverse effects and consequences from torrential rains and related flood disasters since decades. In particular, in Sri Lankan society, strong socioeconomic networks and reciprocal resource sharing and mobilizations, their collaborations, and altruistic nature of helping others (this is such a de-facto tradition) have affectively been forestalled the adverse effects and consequences of mass flooding events. On this context, this research seeks to examine the key research puzzle of What are the roles, efficacies and the Geographies of socioeconomic networks and related social capital behaviors in the ameliorating social vulnerability to flood inundation in Sri Lankan soil? For the empirical study, 405 flood-inundated households were selected prior to their consent for household questionnaire survey covering 21 local administrative divisions (e.g. GNDs) in Sri Lanka. The study areas comprised 21 Grama Niladhari Divisions (GNDs) which are belonged to rural (15 GNDs covering 327 households) and urban (6 GNDs covering 78 households) geographical settings. For empirical data collection, household questionnaire survey, informal interviews, focus group discussions, and field observations were carried out during 13 months (from January, 2018 to January, 2019). The study mainly used Social Networks Analysis (SNA) methods, quantification of social vulnerability to flood inundation by applying Multi Facets Composite Social Vulnerability Index (MFCSVI), and some of qualitative methods in accordance with the mixed research method approach. The empirical findings revealed that network characteristics and measures have changed over time (at before, during, and after phases) in different magnitudes and also observed evolutionary changes of network ties over time and among different geographical settings. In particular, related to all the rural GNDs, the key network measures (e.g. degree density, closeness, and betweenness centralities) are decreased from before flood inundation phase to during phase and then increased at after phase. On the contrary, in urban networks, those measures have increased from before phase to during phase and then decreased at the after phase. And also, network clusters are observed at both before and during phases similarly in rural and urban areas. Network structures are became more distributed forms at after phase in almost all the rural support networks. By contrast, all the structural changes of networks solely depend on the behaviors and characteristics of reciprocal support ties and their resource mobilizations. Social networks have been played a pivotal role in flood disaster responding at before, during, and after flooding events. Particularly, provision of information, food, water and other basic needs; evacuation and moving out belongings; provision of shelters; moving in belongings back; cleaning up contaminated households and public places; provisions of emotional and financial supports are strongly helped and mattered for securing and reviving flood-affected livelihoods. More importantly, major differences of support networks behaviors are observed in related to the urban-rural dichotomy and also among rural areas at lesser magnitudes. In other words, the regional support network behaviors in the flooding events are in stark contrast to the urban networks behaviors is diversified a plenty of ways. Much dense support ties are exemplified in rural areas compared to the urban contexts. Study also revealed that the villagers have very strong historical background of socio-economic networks and social capital in response to flooding events by the ways in which identified it as Traditional Social Capital (TSC) in this study. The scores of social vulnerability to flood inundation are varied between sectors as well as within sectors. Kuruwita sampled households exemplified with fairly low scores of vulnerability compared to Elapatha and Colombo sampled households. For examples, in Kuruwita, Elapatha, and Colombo DSDs, the average vulnerability scores observed as 0.39 (with Min 0.01 and Max 0.875), 0.48 (with Min 0.113 and Max 0.996), and 0.56 (with Min 0.211 and Max 0.999) respectively. Regional diversities of vulnerability scores also are identified. Vulnerability mapping (household level and GND level) also revealed the different spatial patterns of social vulnerability distribution. Results show that MFCSVI is a suitable and sophisticated empirical application for regional level social vulnerability assessment in reference to the key vulnerability components. Most notably, this study made some of contributions and theoretical implications to the existing body of literatures of related disciplines, for instances, analyzing of spatiotemporal dynamics of socio-economic networks of different flood-inundated geographical settings, the examining of the traditional social capital perspective of affected villages, and the empirical application of MFCSVI approach for social vulnerability quantification.Chapter 1. Introduction 1 1.1. Background and research problem 1 1.2. Research questions and objectives 9 1.3. Relevance and significance 11 1.4. Scope and theoretical background 13 1.5. Organization of chapters 14 Chapter 2. Literature review and conceptual background 16 2.1. Social networks 16 2.1.1. Conceptual background of social networks 17 2.1.2. Measurement and representation of social networks 19 2.1.3. Social capital and social networks 25 2.1.4. Social networks and social capital in the context of disaster preparedness and recovery 29 2.2. Social vulnerability in the context of natural disasters 33 2.2.1. Vulnerability in different perspectives and notions 36 2.2.2. Determinants of social vulnerability 41 2.2.3. Exposure, coping capacity, adaptive capacity, and resilience of social vulnerability 44 2.2.4. Social vulnerability assessment and key applications 49 Chapter 3. Materials and methodologies 56 3.1. Study areas 56 3.1.1. Kuruwita DSD 57 3.1.2. Elapatha DSD 60 3.1.3. Colombo DSD 63 3.2. Research methods and empirical applications 69 3.2.1. Data collection 69 3.2.2. Quantitative and qualitative analysis methods and measures 76 Chapter 4. Spatio-temporal dynamics of socio-economic networks for flood disaster preparedness and recovery 91 4.1. Spatio-temporal evolutionary dynamics of reciprocal supports, network measures, and network graphs 91 4.1.1. Kuruwita DSD – Reciprocal supports and socio-economic networks 92 4.1.2. Elapatha DSD – Reciprocal supports and socio-economic networks 102 4.1.3. Colombo DSD – Reciprocal supports and socio-economic networks 112 4.1.4. Overall perspectives of empirical findings 121 4.1.5. Organizational networks behaviors on rural-urban flooding events 147 Chapter 5. Social capital legacies for flood disaster preparedness and recovery 152 5.1. Implications of Social capital legacies 152 5.1.1. Socio-economic associations and groups 153 5.1.2. Feeling trust and solidarity among respondents 157 5.1.3. Collective actions and cooperation in daily life 160 5.1.4. Information and communication among villagers 162 5.1.5. Social cohesion and inclusion among respondents 163 5.1.6. Towards empowering people and political actions 165 5.2. Traditional social capital (TSC) in the flood inundation events 168 5.2.1. Past experiences of social capital 168 5.2.2. Legacies of traditional Native-ethos and floods preventive apparatuses 171 Chapter 6. Characteristics and geographies of social vulnerability to flood inundation 180 6.1. Measurement of multi-facets composite social vulnerability to flood inundation 181 6.1.1. Social vulnerability to flood inundation - Kuruwita DSD 181 6.1.2. Social vulnerability to flood inundation - Elapatha DSD 187 6.1.3. Social vulnerability to flood inundation - Colombo DSD 192 6.2. Spatial distribution of social vulnerability to the flood inundation 198 6.2.1. Spatial patterns of social vulnerability at the household Level 198 6.2.2. Spatial patterns of social vulnerability for the GNDs 207 6.3. Geographies of social vulnerability to the flood inundation and rural-urban dichotomy 216 Chapter 7. Conclusion and discussion 229 7.1. Key findings 229 7.2. The Research gaps, empirical contributions and theoretical implications 230 7.3. Policy implications and recommendations 234 7.4. Future research foci 237 7.5. Limitations 238 Bibliography 240 Abstract in Korean 259 Appendixes 261Docto

Peer assessment model with the sentiment análisis approach

La evaluación de los trabajos de respuesta abierta es una tarea que debe ser realizada por un experto; sin embargo, suponen una importante carga de trabajo de corrección para el docente. En este contexto, la evaluación entre pares se ha considerado como un enfoque alternativo para abordar el problema. Este tipo de evaluación no solo proporciona la reducción en la carga de trabajo de corrección, sino que también aporta beneficios adicionales, como la posibilidad de que el estudiante verifique diferentes soluciones para un mismo problema y la provisión de retroalimentaciones útiles. Los cambios en los paradigmas educativos han promovido la integración de métodos de evaluación que pretenden ir más allá de la evaluación de conocimientos (sumativa), que estén más integrados en el proceso de formación y aprendizaje (formativa). La evaluación formativa contribuye significativamente en la calidad de aprendizaje que los estudiantes obtienen al dar y recibir retroalimentación, y en el acceso inmediato que los docentes pueden tener sobre el progreso de la clase. Las instituciones educativas actualmente buscan obtener el conocimiento inmerso de estos textos no estructurado. Por lo tanto, el objetivo general de esta tesis ha sido diseñar un modelo de evaluación entre pares, que coadyuve a los docentes a mejorar sus procesos de enseñanza-aprendizaje mediante métodos de análisis de sentimiento. Se aplicó la metodología de diseño investigación-acción, en primer lugar, se realizó el estado del arte sobre evaluación entre pares, minería de texto y técnicas de computación blanda. Subsecuentemente, se diseñó un modelo que combina la evaluación entre pares con el aprendizaje colaborativo y el método calibrado en varias fases: a) se formó grupos de estudiantes que participan en realizar el trabajo de manera colaborativa, con la finalidad de tener grupos similares, pero tener diferencias individuales en el proceso de evaluación entre pares para beneficiarse de la colaboración entre estudiantes; b) se diseñó una rúbrica para la recolección de datos, donde los evaluadores evaluaron aspectos específicos del trabajo, proporcionando por cada criterio una puntuación numérica y retroalimentación textual; c) los evaluados evaluaron la calidad de evaluación de la tarea (evaluación inversa) para obtener el rating de confianza del evaluador; d) los grupos corrigieron el trabajo basándose en las retroalimentaciones dadas por los evaluadores en la primera ronda (evaluación en dos rondas); e) la puntuación de evaluación de tarea se calibró en función del rendimiento e índice (rating) de confianza del evaluador. Durante el desarrollo del modelo se obtuvo dos conjuntos de datos en español, uno de evaluación de tarea y otro de evaluación de calidad de la evaluación. Se aplicó el enfoque de aprendizaje automático supervisado para obtener una puntuación de sentimiento correspondiente a una retroalimentación textual específica. Se analizó distintas técnicas de minería de texto y procesamiento de lenguaje natural sobre la tarea de clasificación de sentimiento como Bag of Words, combinaciones de (N-Grams + Term Frequency-Inverse Document Frequency + Stop- Words), y Word2Vec/Glove pre-entrenados para formar los distintos vocabularios. Se evaluó algoritmos de aprendizaje automático clásico (Naïve Bayes, Multinomial Naive Bayes, Support Vector Machine, Logistic Regression, Random Forest, Decision Trees), de aprendizaje automático moderno (Vote Ensemble), y de aprendizaje profundo (Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM)). Se obtuvo dos modelos predictivos con mejor rendimiento. Un modelo con Bi-LSTM utilizando representación de Glove, para predecir la puntuación de sentimiento de la retroalimentación textual de evaluación de tarea; y un modelo con LSTM utilizando representación de Glove, para predecir la puntuación de sentimiento de la retroalimentación textual de evaluación de calidad de la evaluación. Seguidamente, se obtuvo un modelo de cálculo que contribuyó a mejorar la confiabilidad del proceso de evaluación entre pares. La puntuación de cada criterio de evaluación de tarea y evaluación de calidad de evaluación se generó con la técnica computacional de lógica difusa correlacionando puntuación numérica y sentimiento, determinando que los métodos de defuzzificación (máximo más chico, media de máximo y máximo más grande) fueron los más apropiados para este estudio. La puntuación individual de cada evaluador se obtuvo con cálculos de media de todos los criterios. La puntuación del colectivo de evaluación de tarea y rating de confianza del evaluador se obtuvo con cálculos de media/mediana del conjunto de puntuaciones individuales, determinando que la mediana tiene el mejor ajuste para generar una puntuación del colectivo confiable. Se probó la validez del modelo propuesto en 3 escenarios de educación superior: virtual asincrónico, virtual sincrónico y presencial. Se correlacionó mediante Pearson la puntación que recibe el estudiante del colectivo con la puntuación que proporciona el docente, obteniendo similaridad fuerte en el 8% de las actividades en virtual asincrónico (r=0.718-0.790), en el 25% de las actividades en virtual sincrónico (r=0.741 a 0.971) y en el 40% de las actividades en presencial (r=0.780 a 0.951), determinando que el modelo se puede aplicar en todos los escenarios de educación evaluados, y con mayor efectividad en el presencial. Finalmente, se obtuvo un modelo de calibración que contribuyó a mejorar la fiabilidad en el proceso de evaluación entre pares, ya que, mediante el ajuste de la puntuación individual de cada tarea en función del rendimiento y índice (rating) de confianza del evaluador, se logró que la relación entre la puntación del colectivo y puntuación que proporciona el docente tendiera a subir el 46% de las actividades en escenario virtual asincrónico, 69% en virtual sincrónico y 60% en presencial. Además, se evaluó si existe mejora del rendimiento estudiantil en la segunda ronda aplicando el modelo en el proceso de evaluación entre pares, mediante la prueba t de Student, se determinó que el 100% de las actividades evaluadas obtuvieron la puntuación media en la segunda ronda mayor que la primera ronda con un valor de significancia menor a 0.05, el incremento en la segunda ronda del rendimiento del estudiante en virtual asincrónico fue de 3%- 12%, en virtual sincrónico de 7%-22%, y en presencial de 15%-34%. En tal sentido, el modelo de evaluación entre pares basado en análisis de sentimiento podría implementarse como una herramienta pedagógica para apoyar al docente en enriquecer el proceso de enseñanza aprendizaje, ya que los estudiantes dieron y recibieron retroalimentaciones detalladas sobre lo correcto o incorrecto de un trabajo específico, y pudieron refutar sobre las retroalimentaciones dadas; lo que a su vez indujo que mejoraran el trabajo y el rendimiento en la segunda rondaThe assessment of open response work is a task that must be carried out by an expert; however, they represent a significant correction workload for the teacher. In this context, peer assessment has been considered as an alternative approach to address the problem. This type of assessment not only reduces the correction workload but also brings additional benefits, such as the possibility for the student to verify different solutions for the same problem and the provision of useful feedback. Changes in educational paradigms have promoted the integration of assessment methods that aim to go beyond (summative) knowledge assessment, which is more integrated into the training and learning process (formative). Formative assessment contributes significantly to the quality of learning students gain from giving and receiving feedback, and the immediate Access teachers can have to class progress. Educational institutions currently seek to gain immersed knowledge from these unstructured texts. Therefore, the general objective of this thesis has been to design a peer assessment model that helped teachers improve their teaching-learning processes through sentiment analysis methods. The research-action design methodology was applied, firstly, the state of the art on peer assessment, text mining, and computational techniques was carried out. Subsequently, a model was designed that combines peer assessment with collaborative learning and the calibrated method in several phases: a) groups of students were formed to participate in carrying out the work collaboratively, to have similar groups, but have individual differences in the peer assessment process to benefit from collaboration among students; b) a rubric was designed for data collection, where the evaluators evaluated specific aspects of the work, providing a numerical score and textual feedback for each criterion; c) the evaluators evaluated the quality of the task assessment (inverse assessment) to obtain the evaluator's confidence rating; d) the groups corrected the work based on the feedback given by the evaluators in the first round (evaluation in two rounds); e) the task assessment score was calibrated based on the performance and confidence rating of the evaluator. During the development of the model, two sets of data were obtained in Spanish, one for task assessment and the other for assessment of the quality of the assessment. The supervised machine learning approach was applied to obtain a sentiment score corresponding to specific textual feedback. Different text mining and natural language processing techniques were analyzed on the sentiment classification task, such as Bag of Words, combinations of (N-Grams+Term Frequency-Inverse Document Frequency+Stop-Words), and Word2Vec/Glove pre-trained to form the different vocabularies. Algorithms were evaluated of classic machine learning (Naïve Bayes, Multinomial Naïve Bayes, Support Vector Machine, Logistic Regression, Random Forest, Decision Trees), modern machine learning (Vote Ensemble), and deep learning (Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM)). Two predictive models with better performance were obtained. A model with Bi-LSTM using Glove's representation, to predict the sentiment score of task assessment textual feedback; and a model with LSTM using Glove's representation, to predict the sentiment score of the assessment quality assessment textual feedback. Thereafter, a calculation model was obtained that contributed to improving the reliability of the peer assessment process. The score for each task assessment criterion and quality assessment was generated with the fuzzy logic computational technique correlating numerical score and sentiment, determining that the (Smallest of Maximum, Middle of Maximum, and Largest of Maximum) defuzzification methods were the most appropriate for this study. The individual score of each evaluator was obtained with mean calculations of all the criteria. The task assessment collective score and the evaluator confidence rating were obtained with mean/median calculations of the set of individual scores, determining that the median has the best fit to generate a reliable collective score. The validity of the proposed model was tested in 3 higher education scenarios: virtual asynchronous, virtual synchronous, and face-to-face. Using Pearson, the score received by the student from the group was correlated with the score provided by the teacher, obtaining strong similarity in 8% of the activities in virtual asynchronous (r=0.718-0.790), 25% of the activities in virtual synchronous (r=0.741 to 0.971) and 40% of the activities in face-to-face (r=0.780 to 0.951), determining that the model can be applied in all the education scenarios evaluated, and with greater effectiveness in face-to-face. Finally, a calibration model was obtained that contributed to improving the reliability of the peer assessment process, since, by adjusting the individual score of each task based on the performance and confidence rating of the evaluator, it was achieved that the relationship between the score of the group and the score provided by the teacher would tend to increase in 46% of the activities in asynchronous virtual settings, 69% in synchronous virtual settings and 60% in faceto- face settings. In addition, it was evaluated if there is an improvement in student performance in the second round by applying the model in the peer assessment process, using the student's t-test, it was determined that 100% of the activities evaluated obtained the average score in the second round greater than the first round with a significance value of less than 0.05, the increase in the second round of student performance in asynchronous virtual was 3%-12%, in synchronous virtual 7%-22%, and in face-to-face 15 %-3. 4%. In this sense, the peer assessment model based on sentiment analysis could be implemented as a pedagogical tool to support the teacher in enriching the teaching-learning process, since the students gave and received detailed feedback on the correct or incorrect of a specific work, and they were able to refute the feedback given; which in turn induced improved work and performance in the second round

Collective knowledge advancement as a pedagogical practice in teacher education. An explorative case study of student group work with wiki assignments in the interplay between an offline and a global online setting

ENGELSK: The aim of this dissertation was to describe and analyze collective knowledge advancement (CKA) as a pedagogical practice in teacher education. The background is that Internet permits new types of authentic knowledge production that make it possible for anyone to make contributions (e.g., Wikipedia). In the future, it is expected that schools and teacher education institutions will let students make contributions in these online settings to a greater degree. The present study explored how student groups worked with different wiki assignments in one specific teacher education course. A range of different types of data (group interviews, video data, screen capture data, and wiki log data) were collected. By utilizing two theoretical concepts (germ cell and contradictions) from cultural-historical activity theory, help was identified as the singular entity that exhibited the simplest possible characteristics of CKA as a pedagogical practice. On the basis of these findings, it was concluded that three different types of help are particularly important. These include help as informal peer feedback, which relies on spontaneous verbalization of ongoing thoughts, and help that is provided through the open publication of student work in the online setting. The third type of help requires that all students be assigned as helpers for each other so they can share the workload more equally. This dissertation contributes to educational research in five ways. First, the findings indicate that different types of informal teaching are important components of CKA as a pedagogical practice. Second, the results also describe new types of authentic learning that build on interactions between students and outsiders in the online setting. Third, the findings suggest that it is possible to turn campus-based teaching into a “practicum period” by letting students experiment with new types of collaboration. Fourth, the findings show that a polycontextual pedagogical practice emerges in the interplay between an offline setting and several different online settings. Finally, the analysis demonstrated that the notion of a germ cell provides a powerful means of studying different types of pedagogical practice. NORSK: Denne avhandlingen har som formål å beskrive og analysere kollektiv kunnskapsutvikling som en pedagogisk praksis i lærerutdanningen. Bakgrunnen er at internett nå muliggjør mange nye typer autentisk kunnskapsproduksjon der hvem som helst kan være med å bidra (for eksempel Wikipedia). Man vil kunne forvente at både lærerutdanning og skoler i større grad vil la elever være aktive bidragsytere i slike online-miljø. Denne forskningsstudien utforsker hvordan studenter samarbeider når de løser ulike wikioppgaver på et studium i lærerutdanningen. Flere ulike typer data har blitt samlet inn (gruppeintervju, videoopptak av studentsamarbeid, skjermaktivitet på datamaskin og wikilogg). Ved å ta i bruk to teoretiske begreper (kimcelle og kontradiksjoner) fra kulturhistorisk aktivitetsteori, så blir hjelp identifisert som den minste meningsfulle enheten som beskriver hva som karakteriserer kollektiv kunnskapsutvikling som en pedagogisk praksis. Studien viser at særlig tre spesifikke typer hjelp er viktig. Den første typen er hjelp som skjer i form av uformelle tilbakemeldinger mellom medstudenter. Her oppstår den spontant i verbale samtaler. Den andre typen er hjelp som blir gitt ved åpen publisering av studentarbeid i ulike online-miljø. Det tredje typen hjelp består i at alle studenter bør ha en rolle som hjelpere for hverandre i undervisningen slik at arbeidsbyrden blir fordelt mer rettferdig. Denne avhandlingen bidrar til utdanningsforskningen på fem ulike måter. For det første viser den at ulike typer uformell undervisning er sentrale i en pedagogisk praksis som bygger på kollektiv kunnskapsutvikling. For det andre beskriver funnene en ny type autentisk læring som bygger på kommunikasjon mellom studenter og utenforstående i et online-miljø. For det tredje viser funnene at lærerstudentene kan få en form for “praksistrening” på campus ved å eksperimentere med nye samarbeidsformer. For det fjerde viser funnene at den pedagogiske praksis kan analyseres som polykontekstuell fordi studentene deltar i både et lokalt offline-miljø og flere online-miljø. For det femte viser analysen at begrepet kimcelle kan brukes for å prøve å få en dypere forståelse for hva som kjennetegner ulike pedagogiske praksiser.publishedVersio