Supporting mediated peer-evaluation to grade answers to open-ended questions

We show an approach to semi-automatic grading of answers given by students to open ended questions (open answers). We use both peer-evaluation and teacher evaluation. A learner is modeled by her Knowledge and her assessments quality (Judgment). The data generated by the peer- and teacher- evaluations, and by the learner models is represented by a Bayesian Network, in which the grades of the answers, and the elements of the learner models, are variables, with values in a probability distribution. The initial state of the network is determined by the peer-assessment data. Then, each teacher’s grading of an answer triggers evidence propagation in the network. The framework is implemented in a web-based system. We present also an experimental activity, set to verify the effectiveness of the approach, in terms of correctness of system grading, amount of required teacher's work, and correlation of system outputs with teacher’s grades and student’s final exam grade

A Life Cycle Software Quality Model Using Bayesian Belief Networks

Software practitioners lack a consistent approach to assessing and predicting quality within their products. This research proposes a software quality model that accounts for the influences of development team skill/experience, process maturity, and problem complexity throughout the software engineering life cycle. The model is structured using Bayesian Belief Networks and, unlike previous efforts, uses widely-accepted software engineering standards and in-use industry techniques to quantify the indicators and measures of software quality. Data from 28 software engineering projects was acquired for this study, and was used for validation and comparison of the presented software quality models. Three Bayesian model structures are explored and the structure with the highest performance in terms of accuracy of fit and predictive validity is reported. In addition, the Bayesian Belief Networks are compared to both Least Squares Regression and Neural Networks in order to identify the technique is best suited to modeling software product quality. The results indicate that Bayesian Belief Networks outperform both Least Squares Regression and Neural Networks in terms of producing modeled software quality variables that fit the distribution of actual software quality values, and in accurately forecasting 25 different indicators of software quality. Between the Bayesian model structures, the simplest structure, which relates software quality variables to their correlated causal factors, was found to be the most effective in modeling software quality. In addition, the results reveal that the collective skill and experience of the development team, over process maturity or problem complexity, has the most significant impact on the quality of software products

Multi-criteria decision analysis in Bayesian networks-Diagnosing ecosystem service trade-offs in a hydropower regulated river

The paper demonstrates the use of Bayesian networks in multicriteria decision analysis (MCDA) of environmental design alternatives for environmental flows (eflows) and physical habitat remediation measures in the Mandalselva River in Norway. We demonstrate how MCDA using multi-attribute value functions can be implemented in a Bayesian network with decision and utility nodes. An object-oriented Bayesian network is used to integrate impacts computed in quantitative sub-models of hydropower revenues and Atlantic salmon smolt production and qualitative judgement models of mesohabitat fishability and riverscape aesthetics. We show how conditional probability tables are useful for modelling uncertainty in value scaling functions, and variance in criteria weights due to different stakeholder preferences. While the paper demonstrates the technical feasibility of MCDA in a BN, we also discuss the challenge

Optimising ITS behaviour with Bayesian networks and decision theory

We propose and demonstrate a methodology for building tractable normative intelligent tutoring systems (ITSs). A normative ITS uses a Bayesian network for long-term student modelling and decision theory to select the next tutorial action. Because normative theories are a general framework for rational behaviour, they can be used to both define and apply learning theories in a rational, and therefore optimal, way. This contrasts to the more traditional approach of using an ad-hoc scheme to implement the learning theory. A key step of the methodology is the induction and the continual adaptation of the Bayesian network student model from student performance data, a step that is distinct from other recent Bayesian net approaches in which the network structure and probabilities are either chosen beforehand by an expert, or by efficiency considerations. The methodology is demonstrated by a description and evaluation of CAPIT, a normative constraint-based tutor for English capitalisation and punctuation. Our evaluation results show that a class using the full normative version of CAPIT learned the domain rules at a faster rate than the class that used a non-normative version of the same system

Bayesian Networks with Expert Elicitation as Applicable to Student Retention in Institutional Research

The application of Bayesian networks within the field of institutional research is explored through the development of a Bayesian network used to predict first- to second-year retention of undergraduates. A hybrid approach to model development is employed, in which formal elicitation of subject-matter expertise is combined with machine learning in designing model structure and specification of model parameters. Subject-matter experts include two academic advisors at a small, private liberal arts college in the southeast, and the data used in machine learning include six years of historical student-related information (i.e., demographic, admissions, academic, and financial) on 1,438 first-year students. Netica 5.12, a software package designed for constructing Bayesian networks, is used for building and validating the model. Evaluation of the resulting model’s predictive capabilities is examined, as well as analyses of sensitivity, internal validity, and model complexity. Additionally, the utility of using Bayesian networks within institutional research and higher education is discussed. The importance of comprehensive evaluation is highlighted, due to the study’s inclusion of an unbalanced data set. Best practices and experiences with expert elicitation are also noted, including recommendations for use of formal elicitation frameworks and careful consideration of operating definitions. Academic preparation and financial need risk profile are identified as key variables related to retention, and the need for enhanced data collection surrounding such variables is also revealed. For example, the experts emphasize study skills as an important predictor of retention while noting the absence of collection of quantitative data related to measuring students’ study skills. Finally, the importance and value of the model development process is stressed, as stakeholders are required to articulate, define, discuss, and evaluate model components, assumptions, and results

A canonical theory of dynamic decision-making

Decision-making behavior is studied in many very different fields, from medicine and eco- nomics to psychology and neuroscience, with major contributions from mathematics and statistics, computer science, AI, and other technical disciplines. However the conceptual- ization of what decision-making is and methods for studying it vary greatly and this has resulted in fragmentation of the field. A theory that can accommodate various perspectives may facilitate interdisciplinary working. We present such a theory in which decision-making is articulated as a set of canonical functions that are sufficiently general to accommodate diverse viewpoints, yet sufficiently precise that they can be instantiated in different ways for specific theoretical or practical purposes. The canons cover the whole decision cycle, from the framing of a decision based on the goals, beliefs, and background knowledge of the decision-maker to the formulation of decision options, establishing preferences over them, and making commitments. Commitments can lead to the initiation of new decisions and any step in the cycle can incorporate reasoning about previous decisions and the rationales for them, and lead to revising or abandoning existing commitments. The theory situates decision-making with respect to other high-level cognitive capabilities like problem solving, planning, and collaborative decision-making. The canonical approach is assessed in three domains: cognitive and neuropsychology, artificial intelligence, and decision engineering

A Bayesian Belief Network learning tool integrates multi-scale effects of riparian buffers on stream invertebrates

Riparian forest buffers have multiple benefits for biodiversity and ecosystem services in both freshwater and terrestrial habitats but are rarely implemented in water ecosystem management, partly reflecting the lack of information on the effectiveness of this measure. In this context, social learning is valuable to inform stakeholders of the efficacy of riparian vegetation in mitigating stream degradation. We aim to develop a Bayesian belief network (BBN) model for application as a learning tool to simulate and assess the reach-and segment-scale effects of riparian vegetation properties and land use on instream invertebrates. We surveyed reach-scale riparian conditions, extracted segment-scale riparian and subcatchment land use information from geographic information system data, and collected macroinvertebrate samples from four catchments in Europe (Belgium, Norway, Romania, and Sweden). We modelled the ecological condition based on the Average Score Per Taxon (ASPT) index, a macroinvertebrate-based index widely used in European bioassessment, as a function of different riparian variables using the BBN modelling approach. The results of the model simulations provided insights into the usefulness of riparian vegetation attributes in enhancing the ecological condition, with reach-scale riparian vegetation quality associated with the strongest improvements in ecological status. Specifically, reach-scale buffer vegetation of score 3 (i.e. moderate quality) generally results in the highest probability of a good ASPT score (99-100%). In contrast, a site with a narrow width of riparian trees and a small area of trees with reach-scale buffer vegetation of score 1 (i.e. low quality) predicts a high probability of a bad ASPT score (74%). The strengths of the BBN model are the ease of interpretation, fast simulation, ability to explicitly indicate uncertainty in model outcomes, and interactivity. These merits point to the potential use of the BBN model in workshop activities to stimulate key learning processes that help inform the management of riparian zones