Tableau-based decision procedure for linear time temporal logic: implementation, testing, performance analysis and optimisation

This thesis reports on the implementation and experimental analysis of an incremental multi-pass tableau-based procedure a la Wolper for testing satis- ability in the linear time temporal logic LTL, based on a breadth- rst search strategy. I describe the implementation and discuss the performance of the tool on several series of pattern formulae, as well as on some random test sets, and compare its performance with an implementation of Schwendimann's one- pass tableaux by Widmann and Gor e on several representative series of pattern formulae, including eventualities and safety patterns. The experiments have established that Schwendimann's algorithm consistently, and sometimes dra- matically, outperforms the incremental tableaux, despite the fact that the the- oretical worst-case upper-bound of Schwendimann's algorithm, 2EXPTIME, is worse than that of Wolper's algorithm, which is EXPTIME. This shows, once again, that theoretically established worst-case complexity results do not always re ect truly the practical e ciency, at least when comparing decision procedures

Academic Placement Data and Analysis: 2016 Final Report

Academic Placement Data and Analysis (APDA), a project funded by the American Philosophical Association (APA) and headed by Carolyn Dicey Jennings (UC Merced), aims “to make information on academic job placement useful to prospective graduate students in philosophy.” The project has just been updated to include new data, which Professor Jennings describes in a post at New APPS. She also announces a new interactive data tool with which one can sift through and sort information. (from Daily Nous

Using Case-Based Reasoning to Improve the Quality of Feedback Provided by Automated Assessment Systems for Programming Exercises

Information technology is now ubiquitous in higher education institutions worldwide. More than 85% of American universities use e-learning systems to supplement traditional classroom activities. An obvious benefit of these online tools is their ability to automatically grade exercises submitted by students and provide immediate feedback. Most of these systems, however, provide binary (correct/incorrect) feedback to students. While some educators find such feedback is useful, we have found that binary instant feedback causes plagiarism and disengagement from the exercises as some students may need additional guidance in order to successfully overcome obstacles to understanding. In an effort to address the shortcomings of binary feedback, we designed a Case-Based Reasoning (CBR) framework for generating detailed feedback on programming exercises by reusing existing knowledge provided by human instructors. A crucial component of the CBR framework is the ability to recognize incorrectness similarity between programs. Two programs are considered to be similarly incorrect, if they contain similar bugs, which ensures that corrective feedback generated for one program, is equally appropriate for the other.We investigated several approaches for computing incorrectness similarity, including static analysis of source code, execution traces of running programs, and comparing outputs from test cases. We found that, given the kind of errors committed by our students, the dynamic approach of comparing outputs from test cases proved to be the most accurate method of computing incorrectness similarity.We built an e-learning system, called Compass, on top of the CBR platform that we developed. Compass was deployed in a live classroom environment at the University of California, Merced, in the Spring 2017 semester. We compared data collected from this class to data from previous instances of the course, where students were completing the same exercises but received binary instant feedback.We found that the introduction of Compass, and the detailed feedback it is able to generate on programming exercises, led to a statistically significant decrease in plagiarism and disengagement rates. In addition, we found that students were able to complete exercises faster, with fewer errors. All these factors are associated with improved student learning.Another significant aspect of Compass is that it scales well to large class sizes. This is because the number of different mistakes made by students is relatively small and the number of students making the same mistake as other students is large. These two conditions enable the CBR engine of Compass to handle a large number of students with minimal instructor intervention.Work is currently underway to incorporate Compass into other undergraduate courses at the University of California, Merced. As future work, we are planning to investigate the effects of Compass on underrepresented student populations. We have reasons to believe that Compass can provide much needed help to students who may lack confidence to seek such assistance on their own

A Case-Based Reasoning Approach to Providing High-Quality Feedback on Computer Programming Exercises

Automated assessment and immediate feedback are staple features of modern e-learning systems. In the case of programming exercises, most systems only provide binary (correct/incorrect) feedback, which is often inadequate for students struggling with the material, as they may need expert guidance in order to successfully overcome obstacles to understanding. We propose a Case-Based Reasoning (CBR) approach to improve the quality of feedback on programming exercises. CBR is a machine learning technique that solves problems based on previous experiences (cases). Every time the instructor provides feedback to a student on a particular exercise, the information is stored in a database as a past case. When students experience similar problems in the future, knowledge contained in past cases is used to guide the students to a solution. While the system will provide detailed feedback automatically, this feedback will have been previously crafted by human instructors, leveraging their pedagogical expertise

Tableau tool for testing satisfiability in LTL: Implementation and experimental analysis

AbstractWe report on the implementation and experimental analysis of an incremental multi-pass tableau-based procedure à la Wolper for testing satisfiability in the linear time temporal logic LTL, based on a breadth-first search strategy. We describe the implementation and discuss the performance of the tool on several series of pattern formulae, as well as on some random test sets, and compare its performance with an implementation of Schwendimann's one-pass tableaux by Widmann and Goré on several representative series of pattern formulae, including eventualities and safety patterns. Our experiments have established that Schwendimann's algorithm consistently, and sometimes dramatically, outperforms the incremental tableaux, despite the fact that the theoretical worst-case upper-bound of Schwendimann's algorithm, 2EXPTIME, is worse than that of Wolper's algorithm, which is EXPTIME. This shows, once again, that theoretically established worst-case complexity results do not always reflect truly the practical efficiency, at least when comparing decision procedures