Benchmarking in cluster analysis: A white paper

To achieve scientific progress in terms of building a cumulative body of knowledge, careful attention to benchmarking is of the utmost importance. This means that proposals of new methods of data pre-processing, new data-analytic techniques, and new methods of output post-processing, should be extensively and carefully compared with existing alternatives, and that existing methods should be subjected to neutral comparison studies. To date, benchmarking and recommendations for benchmarking have been frequently seen in the context of supervised learning. Unfortunately, there has been a dearth of guidelines for benchmarking in an unsupervised setting, with the area of clustering as an important subdomain. To address this problem, discussion is given to the theoretical conceptual underpinnings of benchmarking in the field of cluster analysis by means of simulated as well as empirical data. Subsequently, the practicalities of how to address benchmarking questions in clustering are dealt with, and foundational recommendations are made

Criterion-referenced measurement: Its main applications, problems and findings

The need for criterion-referenced measurements has mainly arisen from the introduction of instructional programs organized according to modern principles from educational technology. Some of these programs are discussed, and it is indicated for what purposes criterion-referenced measurements are used. Three main problems of criterion-referenced measurement are distinguished: The problem of criterion-referenced scoring and score interpretation, the problem of criterion-referenced item and test analysis, and the problem of mastery testing. For each of these problems a variety of solutions of the paper to provide an overview of these and to introduce the reader to the original literature

A new adaptive response surface method for reliability analysis

Response surface method is a convenient tool to assess reliability for a wide range of structural mechanical problems. More specifically, adaptive schemes which consist in iteratively refine the experimental design close to the limit state have received much attention. However, it is generally difficult to take into account a lot of variables and to well handle approximation error. The method, proposed in this paper, addresses these points using sparse response surface and a relevant criterion for results accuracy. For this purpose, a response surface is built from an initial Latin Hypercube Sampling (LHS) where the most significant terms are chosen from statistical criteria and cross-validation method. At each step, LHS is refined in a region of interest defined with respect to an importance level on probability density in the design point. Two convergence criteria are used in the procedure: The first one concerns localization of the region and the second one the response surface quality. Finally, a bootstrap method is used to determine the influence of the response error on the estimated probability of failure. This method is applied to several examples and results are discussed

Inverse Uncertainty Quantification using the Modular Bayesian Approach based on Gaussian Process, Part 2: Application to TRACE

Inverse Uncertainty Quantification (UQ) is a process to quantify the uncertainties in random input parameters while achieving consistency between code simulations and physical observations. In this paper, we performed inverse UQ using an improved modular Bayesian approach based on Gaussian Process (GP) for TRACE physical model parameters using the BWR Full-size Fine-Mesh Bundle Tests (BFBT) benchmark steady-state void fraction data. The model discrepancy is described with a GP emulator. Numerical tests have demonstrated that such treatment of model discrepancy can avoid over-fitting. Furthermore, we constructed a fast-running and accurate GP emulator to replace TRACE full model during Markov Chain Monte Carlo (MCMC) sampling. The computational cost was demonstrated to be reduced by several orders of magnitude. A sequential approach was also developed for efficient test source allocation (TSA) for inverse UQ and validation. This sequential TSA methodology first selects experimental tests for validation that has a full coverage of the test domain to avoid extrapolation of model discrepancy term when evaluated at input setting of tests for inverse UQ. Then it selects tests that tend to reside in the unfilled zones of the test domain for inverse UQ, so that one can extract the most information for posterior probability distributions of calibration parameters using only a relatively small number of tests. This research addresses the "lack of input uncertainty information" issue for TRACE physical input parameters, which was usually ignored or described using expert opinion or user self-assessment in previous work. The resulting posterior probability distributions of TRACE parameters can be used in future uncertainty, sensitivity and validation studies of TRACE code for nuclear reactor system design and safety analysis

Statistical extraction of process zones and representative subspaces in fracture of random composite

We propose to identify process zones in heterogeneous materials by tailored statistical tools. The process zone is redefined as the part of the structure where the random process cannot be correctly approximated in a low-dimensional deterministic space. Such a low-dimensional space is obtained by a spectral analysis performed on pre-computed solution samples. A greedy algorithm is proposed to identify both process zone and low-dimensional representative subspace for the solution in the complementary region. In addition to the novelty of the tools proposed in this paper for the analysis of localised phenomena, we show that the reduced space generated by the method is a valid basis for the construction of a reduced order model.Comment: Submitted for publication in International Journal for Multiscale Computational Engineerin

Layered evaluation of interactive adaptive systems : framework and formative methods

Peer reviewedPostprin