SUFAT: An analytics tool for gaining insights from student feedback comments

Singapore National Research Foundation under International Research Centres in Singapore Funding Initiative; Singapore MOE Academic Research Tier

Formative analytics: apoio ao processo de aprendizagem

The increasing digitalization of the education system is prompting a different approach to more individualize feedback, especially in the context of large groups. Learning Management Systems are equipped with global statistical analysis modules, which are expensive to license on their own, providing analyses for the different teaching activities carried out on the platform, including personalize alert systems to identify gaps in compliance with scheduled activities. Other educational activities that take place outside the platform generate data that may not be recorded there. Therefore, the processing of data from various sources can be developed using analytics tools, facilitating the construction of interactive dashboards for teachers and students to consult. This results in data that allows students to be informed of their performance on a regular basis and in a personalize way. This study presents architecture designed to accompany a 1st year and 1st semester course in a management degree program with over 300 students. This architecture is made up of three management tools: an APP for teachers, an interactive dashboard for teachers and students and an individualize weekly report sent by email to students. This personalized report compares individual performance with the class and explains the results of the weekly mini-tests carried out in lectures and workshops. It also includes periodic assessment tests and activities carried out on the platform. The report concludes with a text appropriate to the week’s performance. The study is completed with an analysis of student satisfaction with the information made available to them, concluding that it generates a positive perception of their motivation and self-esteem that access to the dashboard and individualize report provides.A crescente digitalização do sistema educativo favorece uma diferente abordagem para um feedback mais individualizado, sobretudo em contexto de grandes grupos. Os sistemas de gestão da aprendizagem são dotados de módulos de análise estatística globais, com dispendioso licenciamento autónomo, disponibilizando análises para as diferentes atividades pedagógicas desenvolvidas na plataforma, incluindo sistemas personalizados de alerta para identificar lacunas no cumprimento das atividades calendarizadas. Outras ações pedagógicas que se desenvolvem fora da plataforma geram dados que podem não ser ali registados. Assim sendo, o tratamento de dados de diversas fontes pode ser desenvolvido através de ferramentas de Analytics facilitando a construção de dashboards interativos de consulta para professores e estudantes. Daqui resultam dados que permitem informar periodicamente e de forma personalizada os estudantes acerca do seu desempenho. O presente estudo apresenta a arquitetura concebida para acompanhar uma disciplina do 1.º semestre do 1.º ano, de um curso superior de Gestão, com mais de 300 estudantes. Desta arquitetura de suporte ao sistema de informação resultam três instrumentos de gestão: uma aplicação (APP) para os docentes, um dashboard interativo para docentes e estudantes e um relatório individual, semanal, enviado por email para os estudantes. Este relatório personalizado que confronta o desempenho individual com o da turma, explicita os resultados dos minitestes semanais, realizados em aulas teórico-práticas e práticas, testes periódicos de avaliação e atividades realizadas na plataforma. O relatório completa-se com um texto adequado ao desempenho na semana. O estudo termina com uma análise à satisfação dos estudantes de acordo com a informação que lhes é disponibilizada, concluindo gerar-se uma perceção positiva na sua motivação e autoestima que o acesso ao dashboard e ao relatório individualizado proporciona

Course evaluation for low pass rate improvement in Engineering education

Abstract : A course evaluation is a process that includes evaluations of lecturers’ teaching performances and their course material moderations. These two procedures are usually implemented, whether officially by the faculty of engineering or by lecturers’ own initiatives, to help identify lecturers’ strengths and weaknesses and the ways forward to improve their performances and their qualities of teaching. This paper presents different ways of implementing these two criteria from students’ and professionals’ perspectives. Official questionnaires from the faculty of engineering, personal questionnaires using Google surveys, Moodle and special designed forms have been used for moderation and evaluations. The process of evaluation is the core of a feedback procedure followed by universities in order for them to monitor the teaching quality of their staff. Satisfactory results show that such a process can improve the lecturers’ teaching performances, courses material quality, students’ satisfaction and performances, and finally the pass rate of the class

Teaching analytics and teacher dashboards to visualise SET data: Implication to theory and practice

Teaching Analytics (TA) is an emergent theoretical approach that combines teaching expertise, visual analytics, and design-based research to support teachers' diagnostic pedagogical ability to use data as evidence to improve teaching quality. The thesis is focused on designing dashboards to help teachers visualise Student Evaluation of Teaching (SET) data as a form of TA for improving the quality of teaching. The research examined the role of TA by deploying customisable dashboards to support teachers in using data to design and facilitate learning. The researcher carried out an integrated literature review to explore the notion of TA and SET data. Moreover, a Data Science Life Cycle model was proposed to guide teachers and researchers using SET data to improve learning and teaching quality. The research comprised several phases. In phase I, a simulated data technique was used to generate SET scores that informed the development of a preliminary teacher dashboard. Phase II surveyed teachers' use of SET data. The survey results indicated that more than half of the participants used SET for improving teaching practice. The research also showed that participants valued the free-text qualitative comments in SET data. Hence, phase III collected real free-text qualitative comments in SET data on students' perceptions of a previously tutored course. The survey results further indicated that although teachers were unaware of a dashboard's value in presenting data, they wanted to visualise SET data using dashboards. Phase IV redesigned the preliminary dashboards to present the real SET data and the simulated SET scores. Finally, phase V carried out usability testing to evaluate teachers' perceptions of usability and usefulness of the teacher's dashboards. Overall, the result of the usability study indicated the perceived value of the teacher's dashboards