Intelligent Authoring of Gamified Intelligent Tutoring Systems

ABSTRACT Intelligent Tutoring Systems can successfully complement and substitute other instructional models in many contexts. However, it is very common to students to become bored or disengaged using ITS. The inclusion of gamification capabilities (e.g., level, points and so on) in ITS design aims to engage students and to drive desired learning behaviors. Researchers have been noting that teachers are increasingly demanding to act as active users of systems with such features. In this context, the main challenge of this project is contributing to the actively participation (i.e., design) of teachers in the use of gamified intelligent tutoring systems. This challenge leads to the following research questions: (i) "how could we enable teachers to customize the construction of gamified ITS in a simple way and without requiring technical capabilities from them?"; and (ii) "how could we also provide good design principles in order to aid teachers in the customization of gamified ITS?". Thus, our aim is to develop an intelligent authoring platform to enable teachers for customizing gamified ITSs. In this way, we describe in this text a set of specific objectives that must be completed to achieve this general aim

Authoring gamified intelligent tutoring systems.

Sistemas Tutores Inteligentes (STIs) têm recibo a atenção de acadêmicos e profissionais desde da década de 70. Tem havido um grande número de estudos recentes em apoio da efetividade de STIs. Entretanto, é muito comum que estudantes fiquem desengajados ou entediados durante o processo de aprendizagem usando STIs. Para considerar explicitamente os aspectos motivacionais de estudantes, pesquisadores estão cada vez mais interessados em usar gamificação em conjunto com STIs. Contudo, apesar de prover tutoria individualizada para estudantes e algum tipo de suporte para professores, estes usuários não têm recebido alta prioridade no desenvolvimento destes tipos de sistemas. De forma a contribuir para o uso ativo e personalizado de STIs gamificados por professores, três problemas técnicos devem ser considerados. Primeiro, projetar STI é muito complexo (deve-se considerar diferentes teorias, componentes e partes interessadas) e incluir gamificação pode aumentar significativamente tal complexidade e variabilidade. Segundo, as funcionalidades de STIs gamificados podem ser usadas de acordo com vários elementos (ex.: nível educacional, domínio de conhecimento, teorias de gamificaçãoe STI, etc). Desta forma, é imprescindível tirar proveito das teorias e práticas de ambos os tópicos para reduzir o espaço de design destes sistemas. Terceiro, para efetivamente auxiliar professores a usarem ativamente estes sistemas, faz-se necessário prover uma solução simples e usável para eles. Para lidar com estes problemas, o principal objetivo desta tese é projetar uma solução computacional de autoria para fornecer aos professores uma forma de personalizar as funcionalidades de STIs gamificados gerenciando a alta variabilidade destes sistemas e considerando as teorias/práticas de gamificação e STI. Visando alcançar este objetivo, nós identificamos o espaço de variabilidade e o representamos por meio do uso de uma abordagem de modelagem de features baseada em ontologias (OntoSPL). Desenvolvemos um modelo ontológico integrado (Ontologia de tutoria gamificada ou Gamified tutoring ontology) que conecta elementos de design de jogos apoiados por evidências no domínio de e-learning, além de teorias e frameworks de gamificação aos conceitos de STI. Finalmente, desenvolvemos uma solução de autoria (chamada AGITS) que leva em consideração tais ontologias para auxiliar professores na personalização de funcionalidades de STIs gamificados. As contribuições deste trabalho são avaliadas por meio da condução de quatro estudos empíricos: (1) conduzimos um experimento controlado para comparar a OntoSPL com uma abordagem de modelagem de features bem conhecida na literatura. Os resultados sugerem que esta abordagem é mais flexível e requer menos tempo para mudar; (2) avaliamos o modelo ontológico integrado usando um método de avaliação de ontologias (FOCA) com especialistas tanto de contexto acadêmico quanto industrial. Os resultados sugerem que as ontologias estão atendendo adequadamente os papeis de representação do conhecimento; (3) avaliamos versões não-interativas da solução de autoria desenvolvida com 59 participantes. Os resultados indicam uma atitude favorável ao uso da solução de autoria projetada,nos quais os participantes concordaram que a solução é fácil de usar, usável, simples, esteticamente atraente,tem um suporte bem percebido e alta credibilidade; e (4) avaliamos, por fim,versões interativas (do zero e usando um modelo) da solução de autoria com 41 professores. Os resultados sugerem que professores podem usar e reusar, com um alto nível de aceitação, uma solução de autoria que inclui toda a complexidade de projetar STI gamificado.Intelligent Tutoring Systems (ITSs) have been drawing the attention of academics and practitioners since early 70’s. There have been a number of recent studies in support of the effectiveness of ITSs. However, it is very common that students become disengaged or bored during the learning process by using ITSs. To explicitly consider students’ motivational aspects, researchers are increasingly interested in using gamification along with ITS.However, despite providing individualized tutoring to students and some kind of support for teachers, teachers have been not considered as first-class citizens in the development of these kinds of systems. In order to contribute to the active and customized use of gamified ITS by teachers, three technical problems should be considered. First, designing ITS is very complex (i.e., take into account different theories, components, and stahekolders) and including gamification may significantly increase such complexity and variability. Second, gamified ITS features can be used depending on several elements (e.g., educational level, knowledge domain, gamification and ITS theories, etc). Thus, it is imperative to take advantage of theories and practices from both topics to reduce the design space of these systems. Third, in order to effectively aid teachers to actively use such systems, it is needed to provide a simple and usable solution for them. To deal with these problems, the main objective of this thesis is to design an authoring computational solution to provide for teachers a way to customize gamified ITS features managing the high variability of these systems and considering gamification and ITS theories/practices. To achieve this objective, we identify the variability space and represent it using an ontology-based feature modeling approach (OntoSPL). We develop an integrated ontological model (Gamified tutoring ontology) that connects evidence-supported game design elements in the e-learning domain as well as gamification theories and frameworks to existing ITS concepts. Finally, we develop an authoring solution (named AGITS) that takes into account these ontologies to aid teachers in the customization of gamified ITS features. We evaluate our contributions by conducting four empirical studies: (1) we perform a controlled experiment to compare OntoSPL against a well-known ontology-based feature modeling approach. The results suggest that our approach is more flexible and requires less time to change; (2) we evaluate the ontological integrated model by using an ontology evaluation method (FOCA) with experts from academic and industrial settings. The results suggest that our ontologies are properly targeting the knowledge representation roles; (3) we evaluate non-interactive versions of the designed authoring solution with 59 participants. The results indicate a positive attitude towards the use of the designed authoring solutions, in which participants agreed that they are ease to use, usable, simple, aesthetically appealing, have a well-perceived system support and high credibility; and (4) we also evaluate interactive versions (scratch and template) of our authoring solution with 41 teachers. The results suggest that teachers can use and reuse, with a high acceptance level, an authoring solution that includes all the complexity to design gamified ITS

Ontology-based feature modeling: an empirical study in changing scenarios

A software product line (SPL) is a set of software systems that have a particular set of common features\ud and that satisfy the needs of a particular market segment or mission. Feature modeling is one of the key\ud activities involved in the design of SPLs. The feature diagram produced in this activity captures the commonalities\ud and variabilities of SPLs. In some complex domains (e.g., ubiquitous computing, autonomic\ud systems and context-aware computing), it is difficult to foresee all functionalities and variabilities a\ud specific SPL may require. Thus, Dynamic Software Product Lines (DSPLs) bind variation points at runtime\ud to adapt to fluctuations in user needs as well as to adapt to changes in the environment. In this context,\ud relying on formal representations of feature models is important to allow them to be automatically analyzed\ud during system execution. Among the mechanisms used for representing and analyzing feature\ud models, description logic (DL) based approaches demand to be better investigated in DSPLs since it provides\ud capabilities, such as automated inconsistency detection, reasoning efficiency, scalability and\ud expressivity. Ontology is the most common way to represent feature models knowledge based on DL reasoners.\ud Previous works conceived ontologies for feature modeling either based on OWL classes and properties\ud or based on OWL individuals. However, considering change or evolution scenarios of feature\ud models, we need to compare whether a class-based or an individual-based feature modeling style is\ud recommended to describe feature models to support SPLs, and especially its capabilities to deal with\ud changes in feature models, as required by DSPLs. In this paper, we conduct a controlled experiment to\ud empirically compare two approaches based on each one of these modeling styles in several changing scenarios\ud (e.g., add/remove mandatory feature, add/remove optional feature and so on). We measure time to\ud perform changes, structural impact of changes (flexibility) and correctness for performing changes in our\ud experiment. Our results indicate that using OWL individuals requires less time to change and is more\ud flexible than using OWL classes and properties. These results provide insightful assumptions towards\ud the definition of an approach relying on reasoning capabilities of ontologies that can effectively support\ud products reconfiguration in the context of DSPL.CNPqCAPE

25 years of Requirements Engineering in Brazil: a systematic mapping

Abstract. The celebration of 25th anniversary of the Brazilian Symposium of Software Engineering (SBES) as well as the forthcoming Requirements Engineering Conference to be held in Brazil for the first time, has led us to have a closer look at the local Requirements Engineering (RE) Community. A systematic mapping was performed in order to find out the main Brazilian research groups, authors as well as their topics of interest and publications with greatest impact. This information may be useful for those that do not know well the local requirements engineering community, such as local newcomers or foreign researchers. It may also help to identify potential groups for collaboration. Similarly, it may provide valuable information to assist local agencies when granting research funds

A systematic review on multi-device inclusive environments

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)The challenge of designing universal access to knowledge demands considerations on multi-device interaction. A systematic review of inclusive environments built from multiple devices was conducted based on studies published during the period of 2002-2013. The search strategy combined manual and automatic searches from which 8889 studies were identified; 34 studies were found proposing software tools for building multi-device inclusive environments (0.38 % of the original sample). Thus, this study analyzes the ways academic and industrial communities have developed tools for building inclusive environments. The main findings of this review are: (1) an urgent need for the recognition of accessibility as an important non-functional requirement; (2) a need for taking into account the social conditions of users, such as illiteracy and people living in underserved communities; and (3) the identification of new research questions in the context of multi-device inclusive environments.The challenge of designing universal access to knowledge demands considerations on multi-device interaction. A systematic review of inclusive environments built from multiple devices was conducted based on studies published during the period of 2002-2013.154737772CNPQ - CONSELHO NACIONAL DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICOConselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)150113/2013-7; 560044/2010-0; 308618/2014-

Interpretable Knowledge Gain Prediction for Vocational Preparatory E-Learnings

Paaßen B, Dywel M, Fleckenstein M, Pinkwart N. Interpretable Knowledge Gain Prediction for Vocational Preparatory E-Learnings. In: DeFalco JA, Matos DDM da C, Blanc B, Reichow I, eds. Proceedings of the 23rd International Conference on Artificial Intelligence in Education (AIED 2022) Practitioner’s Track. 2022: 132–137.Vocational further education typically builds upon prior knowledge. For learners who lack this prior knowledge, preparatory e-learnings may help. Therefore, we wish to identify students who would profit from such an e-learning. We consider the example of a math e-learning for the Bachelor Professional of Chemical Production and Management (CCI). To estimate whether the e-learning would help, we employ a predictive model. Developing such a model in a real-world scenario confronted us with a range of challenges, such as small sample sizes, overfitting, or implausible model parameters. We describe how we addressed these challenges such that other practitioners can learn from our case study of employing data mining in vocational training

Evaluation of a hybrid AI-human recommender for CS1 instructors in a real educational scenario

Automatic code graders, also called Programming Online Judges (OJ), can support students and instructors in introduction to programming courses (CS1). Using OJs in CS1, instructors select problems to compose assignment lists, whereas students submit their code solutions and receive instantaneous feedback. Whilst this process reduces the instructors’ workload in evaluating students’ code, selecting problems to compose assignments is arduous. Recently, recommender systems have been proposed by the literature to support OJ users. Nonetheless, there is a lack of recommenders fitted for CS1 courses and the ones found in the literature have not been assessed by the target users in a real educational scenario. It is worth noting that hybrid human/AI systems are claimed to potentially surpass isolated human or AI. In this study, we adapted and evaluated a state-of-the-art hybrid human/AI recommender to support CS1 instructors in selecting problems to compose variations of CS1 assignments. We compared data-driven measures (e.g., time students take to solve problems, number of logical lines of code, and hit rate) extracted from student logs whilst solving programming problems from assignments created by instructors versus when solving assignments in collaboration with an adaptation of cutting-edge hybrid/AI method. As a result, employing a data analysis comparing experimental and control conditions using multi-level regressions, we observed that the recommender provided problems compatible with human-selected in all data-driven measures tested

An ontology-driven software product line architecture for developing gamified intelligent tutoring systems

Intelligent tutoring systems (ITSs) are effective to provide instruction for students in several situations. Many works have been using gamification by adding game elements to learning contexts aiming to engage students and to drive desired learning behaviours. However, the design of gamified ITS should deal with a huge variability. Software product lines (SPLs) promise to offer rapid product development and more affordable development costs to build software from the same family. A key factor to successfully implement a product-line approach is to structure commonalities and variabilities into a product line architecture (PLA). In this paper, we propose a PLA for developing gamified ITSs that uses an ontology-driven feature modelling strategy. We illustrate how our architecture could be applied to instantiate a product on the basic math domain. We also discuss a set of implications of using it as well as how it could support the evolution/changing of gamified ITSs