A theory of cross-validation error

This paper presents a theory of error in cross-validation testing of algorithms for predicting real-valued attributes. The theory justifies the claim that predicting real-valued attributes requires balancing the conflicting demands of simplicity and accuracy. Furthermore, the theory indicates precisely how these conflicting demands must be balanced, in order to minimize cross-validation error. A general theory is presented, then it is developed in detail for linear regression and instance-based learning

Theoretical analyses of cross-validation error and voting in instance-based learning

This paper begins with a general theory of error in cross-validation testing of algorithms for supervised learning from examples. It is assumed that the examples are described by attribute-value pairs, where the values are symbolic. Cross-validation requires a set of training examples and a set of testing examples. The value of the attribute that is to be predicted is known to the learner in the training set, but unknown in the testing set. The theory demonstrates that cross-validation error has two components: error on the training set (inaccuracy) and sensitivity to noise (instability). This general theory is then applied to voting in instance-based learning. Given an example in the testing set, a typical instance-based learning algorithm predicts the designated attribute by voting among the k nearest neighbors (the k most similar examples) to the testing example in the training set. Voting is intended to increase the stability (resistance to noise) of instance-based learning, but a theoretical analysis shows that there are circumstances in which voting can be destabilizing. The theory suggests ways to minimize cross-validation error, by insuring that voting is stable and does not adversely affect accuracy

A comparison of different approaches to target differentiation with sonar

Ankara : The Department of Electrical and Electronics Engineering and the Institute of Engineering and Science of Bilkent University, 2001.Thesis (Ph.D.) -- Bilkent University, 2001.Includes bibliographical references leaves 180-197This study compares the performances of di erent classication schemes and fusion techniques for target di erentiation and localization of commonly encountered features in indoor robot environments using sonar sensing Di erentiation of such features is of interest for intelligent systems in a variety of applications such as system control based on acoustic signal detection and identication map building navigation obstacle avoidance and target tracking The classication schemes employed include the target di erentiation algorithm developed by Ayrulu and Barshan statistical pattern recognition techniques fuzzy c means clustering algorithm and articial neural networks The fusion techniques used are Dempster Shafer evidential reasoning and di erent voting schemes To solve the consistency problem arising in simple ma jority voting di erent voting schemes including preference ordering and reliability measures are proposed and veried experimentally To improve the performance of neural network classiers di erent input signal representations two di erent training algorithms and both modular and non modular network structures are considered The best classication and localization scheme is found to be the neural network classier trained with the wavelet transform of the sonar signals This method is applied to map building in mobile robot environments Physically di erent sensors such as infrared sensors and structured light systems besides sonar sensors are also considered to improve the performance in target classication and localization.Ayrulu (Erdem), BirselPh.D