Mechanix: An Intelligent Web Interface for Automatic Grading of Sketched Free-Body Diagrams

Sketching free body diagrams is an essential skill that students learn in introductory physics and engineering classes; however, university class sizes are growing and often have hundreds of students in a single class. This situation creates a grading challenge for instructors as there is simply not enough time nor resources to provide adequate feedback on every problem. We have developed a web-based application called Mechanix to provide automated real-time feedback on hand-drawn free body diagrams for students. The system is driven by novel sketch recognition algorithms developed for recognizing and comparing trusses, general shapes, and arrows in diagrams. We have discovered students perform as well as paper homework or other online homework systems which only check the final answer through deployment to five universities with 450 students completing homework on the system over the 2018 and 2019 school years. Mechanix has reduced the amount of manual grading required for instructors in those courses while ensuring students can correctly draw the free body diagram

An Exploration Of The Effects Of Enhanced Compiler Error Messages For Computer Programming Novices

Computer programming is an essential skill that all computing students must master and is increasingly important in many diverse disciplines. It is also difficult to learn. One of the many challenges novice programmers face from the start are notoriously cryptic compiler error messages. These report details on errors made by students and are essential as the primary source of information used to rectify those errors. However these difficult to understand messages are often a barrier to progress and a source of discouragement. A high number of student errors, and in particular a high frequency of repeated errors – when a student makes the same error consecutively – have been shown to be indicators of students who are struggling with learning to program. This instrumental case study research investigates the student experience with, and the effects of, software that has been specifically written to help students overcome their challenges with compiler error messages. This software provides help by enhancing error messages, presenting them in a straightforward, informative manner. Two cohorts of first year computing students at an Irish higher education institution participated over two academic years; a control group in 2014-15 that did not experience enhanced error messages, and an intervention group in 2013-14 that did. This thesis lays out a comprehensive view of the student experience starting with a quantitative analysis of the student errors themselves. It then views the students as groups, revealing interesting differences in error profiles. Following this, some individual student profiles and behaviours are investigated. Finally, the student experience is discovered through their own words and opinions by means of a survey that incorporated closed and open-ended questions. In addition to reductions in errors overall, errors per student, and the key metric of repeated error frequency, the intervention group is shown to behave more cohesively with fewer indications of struggling students. A positive learning experience using the software is reported by the students and the lecturer. These results are of interest to educators who have witnessed students struggle with learning to program, and who are looking to help remove the barrier presented by compiler error messages. This work is important for two reasons. First, the effects of error message enhancement have been debated in the literature – this work provides evidence that there can be positive effects. Second, these results should be generalisable at least in part, to other languages, students and institutions

Deployment and Evaluation of an AI-Augmented Tutoring Application in a Classroom Setting

Mechanix is an online application created by the sketch recognition lab at Texas A&M. The application assists student understanding of sketching Free Body Diagrams, through the use of an AI augmented drawing surface used to assist in automated tutoring and grading of subjects that employ FBDs. Deployed at 5 universities across the United states, the evaluation of the impacts of the application has been conducted by members of the iDREEM lab under the advisement of Dr. Julie Linsey. Through the course of the experiment, the application has been shown to have an equal effect as traditional homework methods on student understanding of most engineering concepts related to statics and Dynamics. It has also been shown that the application has a net positive effect on students application of difficult engineering concepts, especially related to open ended design problems.M.S

An Intelligent Debugging Tutor For Novice Computer Science Students

Debugging is a necessary aspect of computer science that can be difficult for novices and experienced programmers alike. This skill is mainly self-taught and is generally gained through trial and error, perhaps with some assistance from a professor or other expert figure. Novices encountering their first software defects may have few avenues open to them depending on the environment in which they are learning to program. The evident problem here is that the potential for a student to become stuck, frustrated, and/or losing confidence in their ability to pursue computer science is great. For a student to be successful when working professionally or progressing through academia they need to be able to function independently; trusting their own knowledge on par or above that of others so that their productivity does not rely on the knowledge of someone else. In order to solve this problem an Intelligent Tutoring System for teaching debugging skills to the novice utilizing Case Based Reasoning, Static Program Slicing, and the student\u27s preferred learning style was proposed. Case acquisition and automatic Exercise Generation were also explored. The system built for this research program was evaluated using novice students at the College and High School levels. Results of this evaluation produced statistically significant results at the p\u3c.05 and p\u3c.01 levels, with generated exercises exhibiting significance at the p\u3c.01 level. These results prove that the methodology chosen is a valid approach for the problem described, that the system does in fact teach students how to debug programs, and that the system is capable of successfully generating exercises on the fly

DVCL:A Distributed Virtual Computer Lab for Security and Network Education

Teaching networking and IT security in higher education requires a safeplayground for students, where they can safely carry out hands-on exercises.This safe playground is known as a computer lab. Universities have todesign and to provide such a lab with respect to certain criteria, e.g.technical opportunities, educational requirements and demands of thelearners. Since there is no one-size-fits-all lab, the labs will be designed tofit into a certain context and thus have own strengths and weaknesses.In this thesis, we investigate and work with two established labs, whichwere designed for hands-on experiences in networking and it security courses.These labs are predominantly different but have an essential overlap ineducational requirements.One lab is developed by the Open University. It is dedicated for distancelearning. It is based on virtualization and every student is able to startthis lab on his own computer. Students can work out exercises wheneverand wherever they want. A shortcoming however is that students have towork alone, (distant) group work is not possible due to the isolated labarchitecture. This lab is the technical base for our research.The other lab is developed by the Cologne University. It is a physicallab, dedicated for on-campus courses and thus it is not portable. Butstudents can meet in the lab, work in groups and are able to get supportfrom a course advisor, who is also able to verify exercises. A shortcominghowever is that students must be present (they have to travel to theuniversity) and they are dependent on the opening hours of the laboratoryand the availability of the course advisers.In two research parts, we show how such two different lab approachescan be combined and what can be achieved.The first research part is about design issues.Initially, we enable group work in our lab for distance education, sincegroup work is an essential part in on-campus classes. Also remote studentsshould be able to work together. Since the lab is designed as an isolatedsystem, the challenge is to connect two of them on the network level butwithout creating a potential bridge between the isolated and the outsideworld. We achieved this by adding a communication interface to the labarchitecture. This communication interface consists of a ghost host toextract and inject network packets, and a remote bridge endpoint, totransport these packets between remote ghost hosts across an intermediateconnection, e.g. the internet. The developed prototype is called DistributedVirtual Computer Lab (DVCL) and enables to connect two or more distantlabs while preserving the isolated character.The DVCL is then extended and improved by a central authority (CA).While the point-to-point connection of the communication interface canconnect two remote networks in a handy way, more connections requirecareful planning by the students. We show that a CA simplifies the usageof our DVCL for the students (and also for academic staff) and in additionto it avoids administrative configuration errors while connecting remotelabs, e.g. a circular flow which leads to an unusable lab.The first part is completed by two applicability enhancements. Thefirst enhancement covers and resolves security issues in order to pushour prototypical implementation of the DVCL and the CA closer to aproductive learning environment. The second enhancement introduces aGraphical User Interface to increase the usability of the DVCL.The second research part is about educational aspects.In the first part, we assume that working independent from a physicalon-campus lab as well as group work is essential for our students. Ourevaluation of more than 200 students participating in an on-campus networkingcourse shows, that nearly half of the students actually say, thatthey would like to work independently from the university at least partiallyand they would welcome the introduction of an e-learning system. Inaddition, a predominant majority think of working in groups as well asreceiving guidance and feedback as crucial to their learning success. Thisresult justifies and confirms our research and also reveals an additional requirement.The challenge is to provide feedback and guidance to a student, who isworking on an exercise and a human course advisor is not available. Thisis e.g. when students use the DVCL at home in the evening hours. Weshow, that captured network traffic of a lab can give some indication ofwhat a student has already configured according to a certain exercise. Weuse this insight to develop an Electronic Exercise Assistant. This softwareprogram is able to recognize the progress of an exercise and can provideappropriate feedback and support, based on preloaded rules and conditions.This significantly improves the learning situation for students workingremotely in lab. Besides this automatic support, the exercise assistant canverify intermediate and complete solutions of an exercise.The second part is completed by an educational enhancement. Ourevaluation and also own observations show, that a lab is more than aroom with computer and network facilities. Rather it is a social placewhere students e.g. meet, form learning groups, talk and discuss. Weuse these insights and enhance the DVCL to support social interactions.Based on our on-campus lab as source, we model a set of communicational,organizational as well as educational activities and implement them in ourDVCL. The result shows, that our DVCL prototype is no longer a technicalplatform but a virtual place, where students can meet, communicate,arrange learning groups, exchange experiences and work on exercises.This thesis shows that aspects of our two different lab environments canbe combined. Our resulting Distributed Virtual Computer Lab incorporatesstrengths of each source lab. It is a gain for distance teaching as well as foron-campus classes. Remote students are now able to utilize the lab being avirtual classroom, where they can learn in groups, assisted by an electronicadvisor and without the need for a face-to-face meeting. On-campus classescan offer students a new learning environment, where they can learn in aclassroom character without the need to travel to the university

An automated marking system for graphical user interfaces

This research investigates the feasibility and effectiveness of assessing students programming solutions to Graphical User Interface exercises in an automated fashion. Automated marking systems ease the burden on the staff involved in running a course and allow students to get results and feedback in a timely fashion. Several automated marking systems exist but are currently unable to mark GUIs. The inherent complexity of GUIs and the need for aesthetic analysis has rendered GUIs beyond the scope of most marking systems. The marking approach described in this thesis implements a number of novel concepts. By exploiting language design properties such as the hierarchical relationship between components, it was possible to develop a framework capable of testing and marking students' GUI programs. Introspectively analysing the interface enables the marking system to obtain access to the intrinsic elements contained within the GUI. Once access has been obtained, the tests can be performed on the actual interface components themselves rather than a mere representation. GUI assessment is more than functional testing, aesthetics play a major role in the creation of an interface. Existing aesthetic metrics do not provide the analytical capabilities required due to their failure to include colour. The distractive effects that colours have were quantified and incorporated into the metrics. The results of the dynamic and aesthetic testing show that through the implementation of the novel components detailed, the creation of a GUI marking system is feasible and its marking both consistent and effective. The design enables the system to return results in a timely fashion and the effects that colour has can be seen in the results of basic aesthetic testing

An automated marking system for graphical user interfaces

This research investigates the feasibility and effectiveness of assessing students programming solutions to Graphical User Interface exercises in an automated fashion. Automated marking systems ease the burden on the staff involved in running a course and allow students to get results and feedback in a timely fashion. Several automated marking systems exist but are currently unable to mark GUIs. The inherent complexity of GUIs and the need for aesthetic analysis has rendered GUIs beyond the scope of most marking systems. The marking approach described in this thesis implements a number of novel concepts. By exploiting language design properties such as the hierarchical relationship between components, it was possible to develop a framework capable of testing and marking students' GUI programs. Introspectively analysing the interface enables the marking system to obtain access to the intrinsic elements contained within the GUI. Once access has been obtained, the tests can be performed on the actual interface components themselves rather than a mere representation. GUI assessment is more than functional testing, aesthetics play a major role in the creation of an interface. Existing aesthetic metrics do not provide the analytical capabilities required due to their failure to include colour. The distractive effects that colours have were quantified and incorporated into the metrics. The results of the dynamic and aesthetic testing show that through the implementation of the novel components detailed, the creation of a GUI marking system is feasible and its marking both consistent and effective. The design enables the system to return results in a timely fashion and the effects that colour has can be seen in the results of basic aesthetic testing