Validation procedures in radiological diagnostic models. Neural network and logistic regression

The objective of this paper is to compare the performance of two predictive radiological models, logistic regression (LR) and neural network (NN), with five different resampling methods. One hundred and sixty-seven patients with proven calvarial lesions as the only known disease were enrolled. Clinical and CT data were used for LR and NN models. Both models were developed with cross validation, leave-one-out and three different bootstrap algorithms. The final results of each model were compared with error rate and the area under receiver operating characteristic curves (Az). The neural network obtained statistically higher Az than LR with cross validation. The remaining resampling validation methods did not reveal statistically significant differences between LR and NN rules. The neural network classifier performs better than the one based on logistic regression. This advantage is well detected by three-fold cross-validation, but remains unnoticed when leave-one-out or bootstrap algorithms are used.Skull, neoplasms, logistic regression, neural networks, receiver operating characteristic curve, statistics, resampling

Regression Error Characteristic Optimisation of Non-Linear Models.

Copyright © 2006 Springer-Verlag Berlin Heidelberg. The final publication is available at link.springer.comBook title: Multi-Objective Machine LearningIn this chapter recent research in the area of multi-objective optimisation of regression models is presented and combined. Evolutionary multi-objective optimisation techniques are described for training a population of regression models to optimise the recently defined Regression Error Characteristic Curves (REC). A method which meaningfully compares across regressors and against benchmark models (i.e. ‘random walk’ and maximum a posteriori approaches) for varying error rates. Through bootstrapping training data, degrees of confident out-performance are also highlighted

ROC curve regression analysis: the use of ordinal regression models for diagnostic test assessment.

Diagnostic tests commonly are characterized by their true positive (sensitivity) and true negative (specificity) classification rates, which rely on a single decision threshold to classify a test result as positive. A more complete description of test accuracy is given by the receiver operating characteristic (ROC) curve, a graph of the false positive and true positive rates obtained as the decision threshold is varied. A generalized regression methodology, which uses a class of ordinal regression models to estimate smoothed ROC curves has been described. Data from a multi-institutional study comparing the accuracy of magnetic resonance (MR) imaging with computed tomography (CT) in detecting liver metastases, which are ideally suited for ROC regression analysis, are described. The general regression model is introduced and an estimate for the area under the ROC curve and its standard error using parameters of the ordinal regression model is given. An analysis of the liver data that highlights the utility of the methodology in parsimoniously adjusting comparisons for covariates is presented

Predicting Landslides Using Locally Aligned Convolutional Neural Networks

Landslides, movement of soil and rock under the influence of gravity, are common phenomena that cause significant human and economic losses every year. Experts use heterogeneous features such as slope, elevation, land cover, lithology, rock age, and rock family to predict landslides. To work with such features, we adapted convolutional neural networks to consider relative spatial information for the prediction task. Traditional filters in these networks either have a fixed orientation or are rotationally invariant. Intuitively, the filters should orient uphill, but there is not enough data to learn the concept of uphill; instead, it can be provided as prior knowledge. We propose a model called Locally Aligned Convolutional Neural Network, LACNN, that follows the ground surface at multiple scales to predict possible landslide occurrence for a single point. To validate our method, we created a standardized dataset of georeferenced images consisting of the heterogeneous features as inputs, and compared our method to several baselines, including linear regression, a neural network, and a convolutional network, using log-likelihood error and Receiver Operating Characteristic curves on the test set. Our model achieves 2-7% improvement in terms of accuracy and 2-15% boost in terms of log likelihood compared to the other proposed baselines.Comment: Published in IJCAI 202

Learning curves for Gaussian process regression: Approximations and bounds

We consider the problem of calculating learning curves (i.e., average generalization performance) of Gaussian processes used for regression. On the basis of a simple expression for the generalization error, in terms of the eigenvalue decomposition of the covariance function, we derive a number of approximation schemes. We identify where these become exact, and compare with existing bounds on learning curves; the new approximations, which can be used for any input space dimension, generally get substantially closer to the truth. We also study possible improvements to our approximations. Finally, we use a simple exactly solvable learning scenario to show that there are limits of principle on the quality of approximations and bounds expressible solely in terms of the eigenvalue spectrum of the covariance function.Comment: 25 pages, 10 figure

EUCLIA - Exploring the UV/optical continuum lag in active galactic nuclei. I. a model without light echoing

The tight inter-band correlation and the lag-wavelength relation among UV/optical continua of active galactic nuclei have been firmly established. They are usually understood within the widespread reprocessing scenario, however, the implied inter-band lags are generally too small. Furthermore, it is challenged by new evidences, such as the X-ray reprocessing yields too much high frequency UV/optical variations as well as it fails to reproduce the observed timescale-dependent color variations among {\it Swift} lightcurves of NGC 5548. In a different manner, we demonstrate that an upgraded inhomogeneous accretion disk model, whose local {\it independent} temperature fluctuations are subject to a speculated {\it common} large-scale temperature fluctuation, can intrinsically generate the tight inter-band correlation and lag across UV/optical, and be in nice agreement with several observational properties of NGC 5548, including the timescale-dependent color variation. The emergent lag is a result of the {\it differential regression capability} of local temperature fluctuations when responding to the large-scale fluctuation. An average speed of propagations as large as $\gtrsim 15\%$ of the speed of light may be required by this common fluctuation. Several potential physical mechanisms for such propagations are discussed. Our interesting phenomenological scenario may shed new light on comprehending the UV/optical continuum variations of active galactic nuclei.Comment: 18 pages, 8 figures. ApJ accepted. Further comments are very welcome