Exploring the Usefulness of a Non-Random Holdout Sample for Model Validation: Welfare Effects on Female Behavior

Opportunities for external validation of behavioral models in the social sciences that are based on randomized social experiments or on large regime shifts, that can be treated as experiments for the purpose of model validation, are extremely rare. In this paper, we consider an alternative approach, namely mimicking the essential element of regime change by non-randomly holding out from estimation a portion of the sample that faces a significantly different policy regime. The non-random holdout sample is used for model validation/selection. We illustrate the non-random holdout sample approach to model validation in the context of a model of welfare program participation. The policy heterogeneity that we exploit to generate a non-random holdout sample takes advantage of the wide variation across states that has existed in welfare policy.Model validation, Hold-out sample, Public welfare

Context discovery using attenuated Bloom filters in ad-hoc networks

A novel approach to performing context discovery in ad-hoc networks based on the use of attenuated Bloom filters is proposed in this paper. In order to investigate the performance of this approach, a model has been developed. This document describes the model and its validation. The model has been implemented in Matlab, and some results are also shown in this document. Attenuated Bloom filters appear to be a very promising approach for context discovery in ad hoc networks

Deep Bilevel Learning

We present a novel regularization approach to train neural networks that enjoys better generalization and test error than standard stochastic gradient descent. Our approach is based on the principles of cross-validation, where a validation set is used to limit the model overfitting. We formulate such principles as a bilevel optimization problem. This formulation allows us to define the optimization of a cost on the validation set subject to another optimization on the training set. The overfitting is controlled by introducing weights on each mini-batch in the training set and by choosing their values so that they minimize the error on the validation set. In practice, these weights define mini-batch learning rates in a gradient descent update equation that favor gradients with better generalization capabilities. Because of its simplicity, this approach can be integrated with other regularization methods and training schemes. We evaluate extensively our proposed algorithm on several neural network architectures and datasets, and find that it consistently improves the generalization of the model, especially when labels are noisy.Comment: ECCV 201

Model-Based Proactive Read-Validation in Transaction Processing Systems

Concurrency control protocols based on read-validation schemes allow transactions which are doomed to abort to still run until a subsequent validation check reveals them as invalid. These late aborts do not favor the reduction of wasted computation and can penalize performance. To counteract this problem, we present an analytical model that predicts the abort probability of transactions handled via read-validation schemes. Our goal is to determine what are the suited points-along a transaction lifetime-to carry out a validation check. This may lead to early aborting doomed transactions, thus saving CPU time. We show how to exploit the abort probability predictions returned by the model in combination with a threshold-based scheme to trigger read-validations. We also show how this approach can definitely improve performance-leading up to 14 % better turnaround-as demonstrated by some experiments carried out with a port of the TPC-C benchmark to Software Transactional Memory

The role of functional prototyping within the KADS methodology : a thesis presented in partial fulfilment of the requirements for the degree of Master of Science in Computer Science at Massey University

Knowledge-based systems have until recent times lacked a clear and complete methodology for their construction. KADS was the result of the early 1980's project (ESPRIT-I P1098) which had the aim of developing a comprehensive, commercially viable methodology for knowledge-based system construction. KADS has subsequently proved to be one of the more popular approaches, focusing on the modelling approach to knowledge based system development. One area of the KADS methodology that has not been examined to any great depth is that of model validation. Model validation is the process of ensuring that a derived model is an accurate representation of the domain from which it has been derived from. The two approaches which have been suggested for this purpose within the KADS framework are that of protocol analysis and functional prototyping. This project seeks to apply the second of these choices, that of functional prototyping, to the model of expertise created by da Silva (1994) for model validation purposes. The problem domain is that of farm management, under an joint program of research between the Computer Science, Information Systems and Agricultural Management departments of Massey University. The project took the model of expertise and created a knowledge representation model in compliance with the selected object-oriented paradigm. After this the creation of a functional prototype in a Microsoft Windows based PC environment took place, using Kappa-PC as the application development tool. The validation took place through a demonstration session to a number of domain experts. Conclusions drawn from the experience gained through the creation and use of the prototype are presented, outlining the reasons why functional prototyping was deemed to be an appropriate method for model validation