Visual Task Classification using Classic Machine Learning and CNNs

Our eyes actively perform tasks including, but not limited to, searching, comparing, and counting. This includes tasks in front of a computer, whether it be trivial activities like reading email, or video gaming, or more serious activities like drone management, or flight simulation. Understanding what type of visual task is being performed is important to develop intelligent user interfaces. In this work, we investigated standard machine and deep learning methods to identify the task type using eye-tracking data-including both raw numerical data and the visual representations of the user gaze scan paths and pupil size. To this end, we experimented with computer vision algorithms such as Convolutional Neural Networks (CNNs) and compared the results to classic machine learning algorithms. We found that Machine learning-based methods performed with high accuracy classifying tasks that involve minimal visual search, while CNNs techniques do better in situations where visual search task is included

Task Classification During Visual Search Using Classic Machine Learning and Deep Learning

In an average human life, the eyes not only passively scan visual scenes, but most times end up actively performing tasks including, but not limited to, searching, comparing, and counting. As a result of the advances in technology, we are observing a boost in the average screen time. Humans are now looking at an increasing number of screens and in turn images and videos. Understanding what scene a user is looking at and what type of visual task is being performed can be useful in developing intelligent user interfaces, and in virtual reality and augmented reality devices. In this research, we run machine learning and deep learning algorithms to identify the task type from eye-tracking data. In addition to looking at raw numerical data, we take a “visual” approach by experimenting on variations of Computer Vision algorithms like Convolutional Neural Networks on the visual representations of the user gaze scan paths. We compare the results of our visual approach to the classic algorithm of random forests

Wandering eyes: Eye movements during mind wandering in video lectures

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/154300/1/acp3632_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/154300/2/acp3632.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/154300/3/acp3632-sup-0001-Suppinfo.pd

Automated Gaze-Based Mind Wandering Detection during Computerized Learning in Classrooms

We investigate the use of commercial off-the-shelf (COTS) eye-trackers to automatically detect mind wandering—a phenomenon involving a shift in attention from task-related to task-unrelated thoughts—during computerized learning. Study 1 (N = 135 high-school students) tested the feasibility of COTS eye tracking while students learn biology with an intelligent tutoring system called GuruTutor in their classroom. We could successfully track eye gaze in 75% (both eyes tracked) and 95% (one eye tracked) of the cases for 85% of the sessions where gaze was successfully recorded. In Study 2, we used this data to build automated student-independent detectors of mind wandering, obtaining accuracies (mind wandering F1 = 0.59) substantially better than chance (F1 = 0.24). Study 3 investigated context-generalizability of mind wandering detectors, finding that models trained on data collected in a controlled laboratory more successfully generalized to the classroom than the reverse. Study 4 investigated gaze- and video- based mind wandering detection, finding that gaze-based detection was superior and multimodal detection yielded an improvement in limited circumstances. We tested live mind wandering detection on a new sample of 39 students in Study 5 and found that detection accuracy (mind wandering F1 = 0.40) was considerably above chance (F1 = 0.24), albeit lower than offline detection accuracy from Study 1 (F1 = 0.59), a finding attributable to handling of missing data. We discuss our next steps towards developing gaze-based attention-aware learning technologies to increase engagement and learning by combating mind wandering in classroom contexts

O quanto eu quero este certificado? caçadores de certificados no Lúmina

Com o grande aumento do número de alunos inscritos em Massive Open Online Courses (MOOCs), e o crescimento na quantidade de plataformas de distribuição, a oferta deste tipo de curso está cada vez maior, pois representam uma possibilidade de disseminação de conhecimento especializado, de forma flexível e aberta. Todavia, com o aumento do uso de plataformas on-line de aprendizagem o estudo da desonestidade acadêmica se torna relevante, neste contexto, pois estes cursos podem ser mais facilmente burlados do que cursos presenciais. Por isso, o objetivo geral desta Tese é identificar parâmetros de configuração de MOOCs que desestimulem estudantes que têm comportamentos de “caçadores de certificados” a obter certificações, ao mesmo tempo em que não desestimulem estudantes engajados, na Plataforma de MOOCs da Universidade Federal do Rio Grande do Sul (UFRGS), o Lúmina. Para tanto, é preciso identificar e caracterizar o perfil dos “caçadores de certificados”, estudantes que buscam explorar características da plataforma e dos cursos, para obter um certificado sem se dedicar a sua aprendizagem. Nesta tese, levantou-se a hipótese de haver um “comportamento de caçador” (independente do aluno) e um perfil que pode ser chamado de “estudante-caçador” (um indivíduo que sempre exibe este comportamento). Para caracterizar o comportamento de caçador e dos estudantes-caçadores foi desenvolvido um processo metodológico iterativo com as seguintes etapas: Seleção e Processamento dos dados; Aplicação de Técnicas de Mineração de Dados Educacionais; e Considerações sobre o Processo. Como técnica de Mineração de Dados Educacionais para identificar este comportamento e estes estudantes, foram utilizados algoritmos de Aprendizagem de Máquina não supervisionados, mais especificamente algoritmos de agrupamento hierárquico. Em relação à identificação do "comportamento de caçador", o algoritmo de agrupamento não foi capaz de identificar características que permitam identificar os usuários, pois os grupos formados apresentam níveis parecidos na maioria das variáveis utilizadas, à exceção das variáveis "curso tem mais de 10 questões", que é um indicador de dificuldade do curso. Em relação à identificação de estudantes caçadores, entende-se que a obtenção de pelo menos 3 certificados em menos de 35 dias é um bom indicador para classificar um estudante como caçador de certificados. Em relação ao modelo que ajusta a presença de caçadores às configurações dos cursos, conclui-se que não há indícios suficientes para indicar que as restrições nas configurações sejam eficazes para inibir caçadores de certificados.With the large increase in the number of students enrolled in Massive Open Online Courses (MOOCs), and the growth in the number of distribution platforms, the offer of this type of course is increasing, as they represent a possibility of disseminating specialized knowledge, of flexible and open way. However, with the increased use of online learning platforms, the study of academic dishonesty becomes relevant in this context, as these courses can be more easily circumvented than face-to-face courses. Therefore, the general objective of this Thesis is to identify MOOCs configuration parameters that discourage students who have “certificate hunter” behaviors to obtain certifications, while not discouraging engaged students, in the MOOCs Platform of the Federal University of Rio de Janeiro. Grande do Sul (UFRGS), the Lúmina. To do so, it is necessary to identify and characterize the profile of “certificate hunters”, students who seek to explore features of the platform and courses, in order to obtain a certificate without dedicating themselves to learning. In this thesis, it was hypothesized that there is a “hunter behavior” (independent of the student) and a profile that can be called “student-hunter” (an individual who always exhibits this behavior). To characterize the behavior of hunters and student-hunters, an iterative methodological process was developed with the following steps: Selection and Processing of data; Application of Educational Data Mining Techniques; and Process Considerations. As an Educational Data Mining technique to identify this behavior and these students, unsupervised Machine Learning algorithms were used, more specifically hierarchical clustering algorithms. Regarding the identification of "hunter behavior", the grouping algorithm was not able to identify characteristics that allow identifying users, since the groups formed have similar levels in most of the variables used, with the exception of the variables "course has more than 10 questions", which is an indicator of course difficulty. Regarding the identification of student hunters, it is understood that obtaining at least 3 certificates in less than 35 days is a good indicator to classify a student as a certificate hunter. Regarding the model that adjusts the presence of hunters to the course settings, it is concluded that there is not enough evidence to indicate that the restrictions in the settings are effective in inhibiting certificate hunters