Modelling human teaching tactics and strategies for tutoring systems

One of the promises of ITSs and ILEs is that they will teach and assist learning in an intelligent manner. Historically this has tended to mean concentrating on the interface, on the representation of the domain and on the representation of the student’s knowledge. So systems have attempted to provide students with reifications both of what is to be learned and of the learning process, as well as optimally sequencing and adjusting activities, problems and feedback to best help them learn that domain. We now have embodied (and disembodied) teaching agents and computer-based peers, and the field demonstrates a much greater interest in metacognition and in collaborative activities and tools to support that collaboration. Nevertheless the issue of the teaching competence of ITSs and ILEs is still important, as well as the more specific question as to whether systems can and should mimic human teachers. Indeed increasing interest in embodied agents has thrown the spotlight back on how such agents should behave with respect to learners. In the mid 1980s Ohlsson and others offered critiques of ITSs and ILEs in terms of the limited range and adaptability of their teaching actions as compared to the wealth of tactics and strategies employed by human expert teachers. So are we in any better position in modelling teaching than we were in the 80s? Are these criticisms still as valid today as they were then? This paper reviews progress in understanding certain aspects of human expert teaching and in developing tutoring systems that implement those human teaching strategies and tactics. It concentrates particularly on how systems have dealt with student answers and how they have dealt with motivational issues, referring particularly to work carried out at Sussex: for example, on responding effectively to the student’s motivational state, on contingent and Vygotskian inspired teaching strategies and on the plausibility problem. This latter is concerned with whether tactics that are effectively applied by human teachers can be as effective when embodied in machine teachers

HoloDetect: Few-Shot Learning for Error Detection

We introduce a few-shot learning framework for error detection. We show that data augmentation (a form of weak supervision) is key to training high-quality, ML-based error detection models that require minimal human involvement. Our framework consists of two parts: (1) an expressive model to learn rich representations that capture the inherent syntactic and semantic heterogeneity of errors; and (2) a data augmentation model that, given a small seed of clean records, uses dataset-specific transformations to automatically generate additional training data. Our key insight is to learn data augmentation policies from the noisy input dataset in a weakly supervised manner. We show that our framework detects errors with an average precision of ~94% and an average recall of ~93% across a diverse array of datasets that exhibit different types and amounts of errors. We compare our approach to a comprehensive collection of error detection methods, ranging from traditional rule-based methods to ensemble-based and active learning approaches. We show that data augmentation yields an average improvement of 20 F1 points while it requires access to 3x fewer labeled examples compared to other ML approaches.Comment: 18 pages

Warnings: Violation Symptoms Indicating Architecture Erosion

As a software system evolves, its architecture tends to degrade, and gradually impedes software maintenance and evolution activities and negatively impacts the quality attributes of the system. The main root cause behind architecture erosion phenomenon derives from violation symptoms (such as violations of architecture pattern). Previous studies focus on detecting violations in software systems using architecture conformance checking approaches. However, code review comments are also rich sources that may contain extensive discussions regarding architecture violations. In this work, we investigated the characteristics of architecture violation symptoms in code review comments from the developers' perspective. We employed a set of keywords related to violation symptoms to collect 606 (out of 21,583) code review comments from four popular OSS projects in the OpenStack and Qt communities. We manually analyzed the collected 606 review comments to provide the categories and linguistic patterns of violation symptoms, as well as the reactions how developers addressed them. Our findings show that: (1) 10 categories of violation symptoms are discussed by developers during the code review process; (2) The frequently-used terms of expressing violation symptoms are "inconsistent" and "violate", and the most frequently-used linguistic pattern is Problem Discovery; (3) Refactoring and removing code are the major measures (90%) to tackle violation symptoms, while a few violation symptoms were ignored by developers. Our findings suggest that the investigation of violation symptoms can help researchers better understand the characteristics of architecture erosion and facilitate the development and maintenance activities, and developers should explicitly manage violation symptoms, not only for addressing the existing architecture violations but also preventing future violations.Comment: Preprint accepted for publication in Information and Software Technology, 202

Understanding, Analysis, and Handling of Software Architecture Erosion

Architecture erosion occurs when a software system's implemented architecture diverges from the intended architecture over time. Studies show erosion impacts development, maintenance, and evolution since it accumulates imperceptibly. Identifying early symptoms like architectural smells enables managing erosion through refactoring. However, research lacks comprehensive understanding of erosion, unclear which symptoms are most common, and lacks detection methods. This thesis establishes an erosion landscape, investigates symptoms, and proposes identification approaches. A mapping study covers erosion definitions, symptoms, causes, and consequences. Key findings: 1) "Architecture erosion" is the most used term, with four perspectives on definitions and respective symptom types. 2) Technical and non-technical reasons contribute to erosion, negatively impacting quality attributes. Practitioners can advocate addressing erosion to prevent failures. 3) Detection and correction approaches are categorized, with consistency and evolution-based approaches commonly mentioned.An empirical study explores practitioner perspectives through communities, surveys, and interviews. Findings reveal associated practices like code review and tools identify symptoms, while collected measures address erosion during implementation. Studying code review comments analyzes erosion in practice. One study reveals architectural violations, duplicate functionality, and cyclic dependencies are most frequent. Symptoms decreased over time, indicating increased stability. Most were addressed after review. A second study explores violation symptoms in four projects, identifying 10 categories. Refactoring and removing code address most violations, while some are disregarded.Machine learning classifiers using pre-trained word embeddings identify violation symptoms from code reviews. Key findings: 1) SVM with word2vec achieved highest performance. 2) fastText embeddings worked well. 3) 200-dimensional embeddings outperformed 100/300-dimensional. 4) Ensemble classifier improved performance. 5) Practitioners found results valuable, confirming potential.An automated recommendation system identifies qualified reviewers for violations using similarity detection on file paths and comments. Experiments show common methods perform well, outperforming a baseline approach. Sampling techniques impact recommendation performance

A model repair application scenario with PROVA

Dissertação de mestrado em Engenharia InformáticaModel-Driven Engineering (MDE) is a well known approach to software development that promotes the use of structured specifications, referred to as models, as the primary development artifact. One of the main challenges in MDE is to deal with a wide diversity of evolving models, some of which developed and maintained in parallel. In this setting, a particular point of attention is to manage the model inconsistencies that become inevitable, since it is too easy to make contradictory design decisions and hard to recognise them. In fact, during the development process, user updates will undoubtedly produce inconsistencies which must eventually be repaired. Tool support for this task is then essential in order to automate model repair, so consistency can be easily recovered. However, one of the main challenges in this domain is that for any given set of inconsistencies, there exists an infinite number of possible ways of fixing it. While much of researchers recognise this fact, the way in which this problem should be resolved is far from being agreed upon, and methods on how to detect and fix inconsistencies vary widely. In this master dissertation a comparison between different approaches is done and an application scenario is explored in close collaboration with industry. An off-the-shelf model repair tool leveraging the power of satisfiability (SAT) solving is put to test, while an incremental technique of complex repair trees is implemented and evaluated as a promising, yet very distinctive competitor.A engenharia orientada aos modelos (MDE), uma abordagem bem conhecida no desenvolvimento de software, promove a utilização de especificações estruturadas, denominadas modelos, como o artefacto primário de desenvolvimento. Um dos principais desafios neste domínio é lidar com a grande diversidade de modelos em evolução, muitas vezes desenvolvidos e mantidos em paralelo. Neste cenário é essencial gerir as inconsistências dos modelos, que se tornam inevitáveis uma vez que facilmente se tomam decisões contraditórias e de difícil reconhecimento. De facto, durante o processo de modelação, atualizações aos modelos por parte do utilizador irão sem dúvida produzir inconsistências que devem ser reparadas. Ferramentas que suportem este processo tornam-se essenciais para automatizar a reparação dos modelos, por forma a que a consistência seja facilmente recuperada. No entanto, para qualquer conjunto de inconsistências existe um número potencialmente infinito de possíveis formas de o corrigir, facto que revela ser um dos principais problemas neste domínio. Embora grande parte dos investigadores reconheça este desafio, a forma como esta problemática deve ser abordada está longe de reunir consenso e as soluções propostas variam muito. Nesta dissertação de mestrado é feita uma comparação entre diferentes abordagens à técnica de reparação de modelos, e um cenário de aplicação é explorado em estreita colaboração com a indústria. Uma ferramenta pronta a usar e que aproveita o poder do SAT solving é posta à prova, enquanto que uma outra técnica, incremental e baseada em complexas árvores de reparação, é implementada e avaliada como uma abordagem concorrente promissora e bastante distinta

Constraint rule-based programming of norms for electronic institutions

Peer reviewedPostprin

Automatically correcting syntactic and semantic errors in ATL transformations using multi-objective optimization

L’ingénierie dirigée par les modèles (EDM) est un paradigme de développement logiciel qui promeut l’utilisation de modèles en tant qu’artefacts de première plan et de processus automatisés pour en dériver d’autres artefacts tels que le code, la documentation et les cas de test. La transformation de modèle est un élément important de l’EDM puisqu’elle permet de manipuler les représentations abstraites que sont les modèles. Les transformations de modèles, comme d’autres programmes, sont sujettes à la fois à des erreurs syntaxiques et sémantiques. La correction de ces erreurs est difficile et chronophage, car les transformations dépendent du langage de transformation comme ATL et des langages de modélisation dans lesquels sont exprimés les modèles en entrée et en sortie. Les travaux existants sur la réparation des transformations ciblent les erreurs syntaxiques ou sémantiques, une erreur à la fois, et définissent manuellement des patrons de correctifs. L’objectif principal de notre recherche est de proposer un cadre générique pour corriger automatiquement de multiples erreurs syntaxiques et sémantiques. Afin d’atteindre cet objectif, nous reformulons la réparation des transformations de modèles comme un problème d’optimisation multiobjectif et le résolvons au moyen d’algorithmes évolutionnaires. Pour adapter le cadre aux deux catégories d’erreurs, nous utilisons différents types d’objectifs et des stratégies sophistiquées pour guider l’exploration de l’espace des solutions.Model-driven engineering (MDE) is a software development paradigm that promotes the use of models as first-class artifacts and automated processes to derive other artefacts from them such as code, documentation and test cases. Model transformation is an important element of MDE since it allows to manipulate the abstract representations that are models. Model transformations, as other programs are subjects to both syntactic and semantic errors. Fixing those errors is difficult and time consuming as the transformations depend on the transformation language such as ATL, and modeling languages in which input and output models are expressed. Existing work on transformation repair targets either syntactic or semantic errors, one error at a time, and define patch templates manually. The main goal of our research is to propose a generic framework to fix multiple syntactic and semantic errors automatically. In order to achieve this goal, we reformulate the repair of model transformations as a multi-objective optimization problem and solve it by means of evolutionary algorithms. To adapt the framework to the two categories of errors, we use different types of objectives and sophisticated strategies to guide the search

A new machine learning approach to support asset management in water distribution networks

One of the main causes of the widespread problem of freshwater scarcity lies in unfruitful maintenance of distribution infrastructure, leading to failures with consequent waste of precious resources. It is estimated that more than 25% of the annual loss of water is due to poor conditions of the distribution networks and, in a scenario of continuously increasing demand for water, effects of such inefficiency might be even more dramatic, beyond the merely economic aspect. However, with the rise of data analysis, the awareness of the power of predictive technologies and machine learning techniques, the opportunity to make use of these tools to support decision making has become more than a hope. With this study, the author attempts to address the problem of usage of historical data of pipes and their failures in the Spanish city of Manresa to deduce conclusions on how to conduct maintenance interventions. After conducting an explorative study on how pipes intrinsic factors may have reflections on breakages, machine learning algorithms (Logistic Regression and Random Forest have been chosen in this thesis) are used to predict pipe failures over time. Lately, results from predictions will be used to take out conclusions from two different assessment models. The first method, given the structure of cost of a general distribution company, tries to establish the optimal ratio between sensitivity and sensibility of a predictive model to return the best economic benefit from the predictive maintenance. The second approach wants to assess how the uptime of the service level can be improved whether relying on prediction to replace pipes, given a certain agreed investment budget. In an old industry such as water distribution, difficulties come up not only during the development of predictive models but also during the reconstruction of the data on which training and testing models, since they can suffer from inconsistencies. Indeed, data gathering has not unique and standardized methodologies and time and people take-over have changed procedures during the data collection, making the whole work harde

Data Credence in IoT: Vision and Challenges

As the Internet of Things permeates every aspect of human life, assessing the credence or integrity of the data generated by "things" becomes a central exercise for making decisions or in auditing events. In this paper, we present a vision of this exercise that includes the notion of data credence, assessing data credence in an efficient manner, and the use of technologies that are on the horizon for the very large scale Internet of Things