Workflow Discovery with Semantic Constraints: The SAT-Based Implementation of APE

Science today is increasingly computational, and many researchers regularly face the need of creating purpose-specific computational pipelines for their specific data analysis problems. The manual composition and implementation of such workflows regularly costs valuable research time. Hence, many scientists wish for a system that would only require an abstract description of their intended data analysis process, and from there automatically compose and implement suitable workflows. In this paper we describe APE (the Automated Pipeline Explorer), a new implementation of a synthesis-based workflow discovery framework that aims to accomplish such automated composition. The framework captures the required technical domain knowledge in the form of tool and type taxonomies and functional tool annotations. Based on this semantic domain model, the framework allows users to specify their intents about workflows at an abstract, conceptual level in the form of natural-language templates. Internally, APE maps them to a temporal logic and translates them into a propositional logic instance of the problem that can be solved by an off-the-shelf SAT solver. From the solutions provided by the solver, APE then constructs executable workflow implementations. First applications of APE on realistic scientific workflow scenarios have shown that it is able to efficiently synthesize meaningful workflows. We use an example from the geospatial application domain as a running example in this paper

Dynamic estimation in the extended marginal Rasch model with an application to mathematical computer-adaptive practice

We introduce a general response model that allows for several simple restrictions, resulting in other models such as the extended Rasch model. For the extended Rasch model, a dynamic Bayesian estimation procedure is provided, which is able to deal with data sets that change over time, and possibly include many missing values. To ensure comparability over time, a data augmentation method is used, which provides an augmented person-by-item data matrix and reproduces the sufficient statistics of the complete data matrix. Hence, longitudinal comparisons can be easily made based on simple summaries, such as proportion correct, sum score, etc. As an illustration of the method, an example is provided using data from a computer-adaptive practice mathematical environment

Knowledge models from PDF textbooks

Textbooks are educational documents created, structured and formatted by domain experts with the primary purpose to explain the knowledge in the domain to a novice. Authors use their understanding of the domain when structuring and formatting the content of a textbook to facilitate this explanation. As a result, the formatting and structural elements of textbooks carry the elements of domain knowledge implicitly encoded by their authors. Our paper presents an extensible approach towards automated extraction of knowledge models from textbooks and enrichment of their content with additional links (both internal and external). The textbooks themselves essentially become hypertext documents where individual pages are annotated with important concepts in the domain. The evaluation experiments examine several aspects and stages of the approach, including the accuracy of model extraction, the pragmatic quality of extracted models using one of their possible applications— semantic linking of textbooks in the same domain, the accuracy of linking models to external knowledge sources and the effect of integration of multiple textbooks from the same domain. The results indicate high accuracy of model extraction on symbolic, syntactic and structural levels across textbooks and domains, and demonstrate the added value of the extracted models on the semantic level

Order out of Chaos: Construction of Knowledge Models from PDF Textbooks

Textbooks are educational documents created, structured and formatted by domain experts with the main purpose to explain the knowledge in the domain to a novice. Authors use their understanding of the domain when structuring and formatting the content of a textbook to facilitate this explanation. As a result, the formatting and structural elements of textbooks carry the elements of domain knowledge implicitly encoded by their authors. Our paper presents an extendable approach towards automated extraction of this knowledge from textbooks taking into account their formatting rules and internal structure. We focus on PDF as the most common textbook representation format; however, the overall method is applicable to other formats as well. The evaluation experiments examine the accuracy of the approach, as well as the pragmatic quality of the obtained knowledge models using one of their possible applications - semantic linking of textbooks in the same domain. The results indicate high accuracy of model construction on symbolic, syntactic and structural levels across textbooks and domains, and demonstrate the added value of the extracted models on the semantic level

Interlingua: Linking textbooks across different languages

Increasing numbers of students enrol in formal and informal courses taught in a foreign language. Studying a course from an unfamiliar university/program is difficult enough, but the difficulties multiply when the transition to new course requirements is exacerbated by the necessity to learn course material in a foreign language. This paper describes Interlingua a platform where students can study textbooks in a foreign language supported by on-demand access to relevant reading material in their mother tongue. Interlingua automatically recognises important terminology within textbooks content, extracts structural models of textbooks and links sections and subsections across textbooks in different languages covering the same academic subject. The interface and architecture of Interlingua as well as the technologies underlying the platform are described

Knowledge models from PDF textbooks

Textbooks are educational documents created, structured and formatted by domain experts with the primary purpose to explain the knowledge in the domain to a novice. Authors use their understanding of the domain when structuring and formatting the content of a textbook to facilitate this explanation. As a result, the formatting and structural elements of textbooks carry the elements of domain knowledge implicitly encoded by their authors. Our paper presents an extensible approach towards automated extraction of knowledge models from textbooks and enrichment of their content with additional links (both internal and external). The textbooks themselves essentially become hypertext documents where individual pages are annotated with important concepts in the domain. The evaluation experiments examine several aspects and stages of the approach, including the accuracy of model extraction, the pragmatic quality of extracted models using one of their possible applications— semantic linking of textbooks in the same domain, the accuracy of linking models to external knowledge sources and the effect of integration of multiple textbooks from the same domain. The results indicate high accuracy of model extraction on symbolic, syntactic and structural levels across textbooks and domains, and demonstrate the added value of the extracted models on the semantic level

TopHat: A formal foundation for task-oriented programming

Software that models how people work is omnipresent in today's society. Current languages and frameworks often focus on usability by non-programmers, sacrificing flexibility and high level abstraction. Task-oriented programming (TOP) is a programming paradigm that aims to provide the desired level of abstraction while still being expressive enough to describe real world collaboration. It prescribes a declarative programming style to specify multi-user workflows. Workflows can be higher-order. They communicate through typed values on a local and global level. Such specifications can be turned into interactive applications for different platforms, supporting collaboration during execution. TOP has been around for more than a decade, in the forms of iTasks and mTasks, which are tailored for real-world usability. So far, it has not been given a formalisation which is suitable for formal reasoning. In this paper we give a description of the TOP paradigm and then decompose its rich features into elementary language elements, which makes them suitable for formal treatment. We use the simply typed lambda-calculus, extended with pairs and references, as a base language. On top of this language, we develop TopHat, a language for modular interactive workflows. We describe TopHat by means of a layered semantics. These layers consist of multiple big-step evaluations on expressions, and two labelled transition systems, handling user inputs. With TopHat we prepare a way to formally reason about TOP languages and programs. This approach allows for comparison with other work in the field. We have implemented the semantic rules of TopHat in Haskell, and the task layer on top of the iTasks framework. This shows that our approach is feasible, and lets us demonstrate the concepts by means of illustrative case studies. TOP has been applied in projects with the Dutch coast guard, tax office, and navy. Our work matters because formal program verification is important for mission-critical software, especially for systems with concurrency

A domain reasoner for propositional logic

An important topic in courses in propositional logic is rewriting propositional formulae with standard equivalences. This paper analyses what kind of feedback is offered by the various learning environments for rewriting propositional logic formulae, and discusses how we can provide these kinds of feedback in a learning environment. To give feedback and feed forward, we define solution strategies for several classes of exercises. We offer an extensive description of the knowledge necessary to support solving this kind of propositional logic exercises in a learning environment

Building a Generic Feedback System for Rule-Based Problems

We present a generic framework that provides hints on how to achieve a goal to users of software supporting rule-based problem solving from different domains. Our approach consists of two parts. First, we present a DSL that relates and unifies different rule-based problems. Second, we use generic search algorithms to solve various kinds of problems. This solution can then be used to calculate a hint for the user. We present three rule-based problem frameworks to illustrate our approach: the Ideas framework, PuzzleScript and iTasks. By taking real world examples from these three example frameworks and instantiating feedback systems for them, we validate our approach

Generation of assessment questions from textbooks enriched with knowledge models

Augmenting digital textbooks with assessment material improves their effectiveness as learning tools. It can be a laborious task requiring considerable amount of time and expertise. This paper presents an automated assessment generation tool that works as a component of the Intextbooks platform. Intextbooks extracts fine-grained knowledge models from PDF textbooks and converts them into semantically annotated learning resources. With the help of the developed assessment components, these textbooks become interactive educational tools capable to assess students' knowledge of relevant concepts. The results of an expert-based pilot evaluation show that generated questions are properly worded and have a good range in term of difficulty. From the point of assessment value, some generated questions types fall behind manually constructed assessment, while others obtain comparable results