15 research outputs found

    Validating Pareto Optimal Operation Parameters of Polyp Detection Algorithms for CT Colonography

    Get PDF
    We evaluated a Pareto front-based multi-objective evolutionary algorithm for optimizing our CT colonography (CTC) computer-aided detection (CAD) system. The system identifies colonic polyps based on curvature and volumetric based features, where a set of thresholds for these features was optimized by the evolutionary algorithm. We utilized a two-fold cross-validation (CV) method to test if the optimized thresholds can be generalized to new data sets. We performed the CV method on 133 patients; each patient had a prone and a supine scan. There were 103 colonoscopically confirmed polyps resulting in 188 positive detections in CTC reading from either the prone or the supine scan or both. In the two-fold CV, we randomly divided the 133 patients into two cohorts. Each cohort was used to obtain the Pareto front by a multi-objective genetic algorithm, where a set of optimized thresholds was applied on the test cohort to get test results. This process was repeated twice so that each cohort was used in the training and testing process once. We averaged the two training Pareto fronts as our final training Pareto front and averaged the test results from the two runs in the CV as our final test results. Our experiments demonstrated that the averaged testing results were close to the mean Pareto front determined from the training process. We conclude that the Pareto front-based algorithm appears to be generalizable to new test data

    Parameter Optimization for Image Denoising Based on Block Matching and 3D Collaborative Filtering

    Get PDF
    Clinical MRI images are generally corrupted by random noise during acquisition with blurred subtle structure features. Many denoising methods have been proposed to remove noise from corrupted images at the expense of distorted structure features. Therefore, there is always compromise between removing noise and preserving structure information for denoising methods. For a specific denoising method, it is crucial to tune it so that the best tradeoff can be obtained. In this paper, we define several cost functions to assess the quality of noise removal and that of structure information preserved in the denoised image. Strength Pareto Evolutionary Algorithm 2 (SPEA2) is utilized to simultaneously optimize the cost functions by modifying parameters associated with the denoising methods. The effectiveness of the algorithm is demonstrated by applying the proposed optimization procedure to enhance the image denoising results using block matching and 3D collaborative filtering. Experimental results show that the proposed optimization algorithm can significantly improve the performance of image denoising methods in terms of noise removal and structure information preservation

    ROC Optimisation of Safety Related Systems

    Get PDF
    1st Workshop on ROC Analysis in Artificial Intelligence (ROCAI 2004), part of the 16th European Conference on Artificial Intelligence, Valencia, Spain, 22-27 August 2004Many safety related and critical systems warn of potentially dangerous events; for example the Short Term Conflict Alert (STCA) system warns of airspace infractions between aircraft. Although installed with current technology such critical systems may become out of date due to changes in the circumstances in which they function, operational procedures and the regulatory environment. Current practice is to ‘tune’ by hand the many parameters governing the system in order to optimise the operating point in terms of the true positive and false positive rates, which are frequently associated with highly imbalanced costs. In this paper we cast the tuning of critical systems as a multiobjective optimisation problem. We show how a region of the optimal receiver operating characteristic (ROC) curve may be obtained, permitting the system operators to select the operating point. We apply this methodology to the STCA system, using a multi-objective (1 + 1)-evolution strategy, showing that we can improve upon the current hand-tuned operating point as well as providing the salient ROC curve describing the true-positive versus false-positive tradeoff

    Multi-objective optimisation of safety related systems: An application to Short Term Conflict Alert.

    Get PDF
    Copyright © 2006 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Notes: In this paper multi-objective optimisation is used for the first time to adjust the 1500 parameters of Short-Term Conflict Alert systems to optimise the Receiver Operating Characteristic (ROC) by simultaneously reducing the false positive rate and increasing the true positive alert rate, something that previous work by other researchers had not succeeded in doing. Importantly for such safety-critical systems, the method also yields an assessment of the confidence that may be placed in the optimised ROC curves. The paper results from a collaboration with NATS and a current KTP project, also with NATS, is deploying the methods in air-traffic control centres nationwide.Many safety related and critical systems warn of potentially dangerous events; for example, the short term conflict alert (STCA) system warns of airspace infractions between aircraft. Although installed with current technology, such critical systems may become out of date due to changes in the circumstances in which they function, operational procedures, and the regulatory environment. Current practice is to "tune," by hand, the many parameters governing the system in order to optimize the operating point in terms of the true positive and false positive rates, which are frequently associated with highly imbalanced costs. We cast the tuning of critical systems as a multiobjective optimization problem. We show how a region of the optimal receiver operating characteristic (ROC) curve may be obtained, permitting the system operators to select the operating point. We apply this methodology to the STCA system, using a multiobjective (1+1) evolution strategy, showing that we can improve upon the current hand-tuned operating point, as well as providing the salient ROC curve describing the true positive versus false positive tradeoff. We also provide results for three-objective optimization of the alert response time in addition to the true and false positive rates. Additionally, we illustrate the use of bootstrapping for representing evaluation uncertainty on estimated Pareto fronts, where the evaluation of a system is based upon a finite set of representative data

    Multi-objective optimisation for receiver operating characteristic analysis

    Get PDF
    Copyright © 2006 Springer-Verlag Berlin Heidelberg. The final publication is available at link.springer.comBook title: Multi-Objective Machine LearningSummary Receiver operating characteristic (ROC) analysis is now a standard tool for the comparison of binary classifiers and the selection operating parameters when the costs of misclassification are unknown. This chapter outlines the use of evolutionary multi-objective optimisation techniques for ROC analysis, in both its traditional binary classification setting, and in the novel multi-class ROC situation. Methods for comparing classifier performance in the multi-class case, based on an analogue of the Gini coefficient, are described, which leads to a natural method of selecting the classifier operating point. Illustrations are given concerning synthetic data and an application to Short Term Conflict Alert

    Multi-class ROC analysis from a multi-objective optimisation perspective

    Get PDF
    Copyright © 2006 Elsevier. NOTICE: this is the author’s version of a work that was accepted for publication in Pattern Recognition Letters . Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Pattern Recognition Letters, Vol. 27 Issue 8 (2006), DOI: 10.1016/j.patrec.2005.10.016Notes: Receiver operating characteristics (ROC) are traditionally used for assessing and tuning classifiers discriminating between two classes. This paper is the first to set ROC analysis in a multi-objective optimisation framework and thus generalise ROC curves to any number of classes, showing how multi-objective optimisation may be used to optimise classifier performance. An important new result is that the appropriate measure for assessing overall classifier quality is the Gini coefficient, rather than the volume under the ROC surface as previously thought. The method is currently being exploited in a KTP project with AI Corporation on detecting credit card fraud.The receiver operating characteristic (ROC) has become a standard tool for the analysis and comparison of classifiers when the costs of misclassification are unknown. There has been relatively little work, however, examining ROC for more than two classes. Here we discuss and present an extension to the standard two-class ROC for multi-class problems. We define the ROC surface for the Q-class problem in terms of a multi-objective optimisation problem in which the goal is to simultaneously minimise the Q(Q − 1) misclassification rates, when the misclassification costs and parameters governing the classifier’s behaviour are unknown. We present an evolutionary algorithm to locate the Pareto front—the optimal trade-off surface between misclassifications of different types. The use of the Pareto optimal surface to compare classifiers is discussed and we present a straightforward multi-class analogue of the Gini coefficient. The performance of the evolutionary algorithm is illustrated on a synthetic three class problem, for both k-nearest neighbour and multi-layer perceptron classifiers

    The 7th Conference of PhD Students in Computer Science

    Get PDF

    A Model Based Approach to the Analysis of Intersection Conflicts and Collision Avoidance Systems

    Full text link
    This dissertation studies the viability of driver assistance systems to improve the safety of „unprotected‟ left turns at signalized intersections. To achieve this, modeling and simulation have been conducted, including a driver model, with calibration and validation based on naturalistic driving data. A detailed analysis of the driving data has been conducted to reconstruct the vehicle trajectories in an automated manner. Particular challenges for this analysis include the development of automated detection of relevant events in a large database, automated estimation of sensor latencies, and the multiple application of Kalman filtering to fuse motion variables. A conflict analysis has been conducted to estimate the actual and predicted available gaps using the reconstructed vehicle trajectories. Monte Carlo simulations were conducted to create a large number of free left turn events in order to simulate a proposed driver assistance system and optimize safety performance. Optimization was conducted using multiobjective techniques which balance performance in terms of the rates of correct detections of conflicts, false alarms, and successful braking under the condition of correct detections based on Pareto optimality criteria. In this study, data to support the analysis was obtained from onboard instrumentation, where it was found essential to include detailed estimation of latencies between various sensors; after this, data fusion can be performed. It was found that high fidelity modeling of longitudinal control is critical to the safety system analysis. Also, it was found necessary to represent multiple levels of control, including visual preview and acceleration feedback. For the speed control reference, it was found that an “anticipated acceleration” can be used to define both straight braking events and free left turns; the driver may keep both options available during the intersection approach up to a critical decision point where the two references are equal. It was critical to the parametric optimization of the driver assistance system to take account of the need for warnings to be issued sufficiently early for the driver to respond; multiobjective design optimization was found to be an appropriate tool to include this requirement, as well as more typical requirements for involving false warnings.Ph.D.Mechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/89791/1/knobukaw_1.pd

    Control of solution MMA polymerization in a CSTR

    Get PDF

    Intelligent computing applications based on eye gaze : their role in mammographic interpretation training

    Get PDF
    Early breast cancer in women is best identified through high quality mammographic screening. This is achieved by well trained health professionals and appropriate imaging. Traditionally this has used X-ray film but is rapidly changing to utilise digital imaging with the resultant mammograms visually examined on high resolution clinical workstations. These digital images can also be viewed on a range of display devices, such as standard computer monitors or PDAs. In this thesis the potential of using such non-clinical workstation display devices for training purposes in breast screening has been investigated. The research introduces and reviews breast screening both in the UK and internationally where it concentrates upon China which is beginning screening. Various imaging technologies used to examine the breast are described, concentrating upon the move from using X-ray film to digital mammograms. Training in screening in the UK is detailed and it is argued that there is a need to extend this. Initially, a national survey of all UK mammography screeners within the National Health Breast Screening Programme (NHSBSP) was undertaken. This highlighted the current main difficulties of mammographic (film) interpretation training being tied to the device for inspecting these images. The screeners perceived the need for future digital imaging training that could be outside the breast screening centre; namely 3W training (Whatever training required, Whenever and Wherever). This is largely because the clinical workstations would logistically not be available for training purposes due to the daily screening demand. Whilst these workstations must be used for screening and diagnostic purposes to allow visualisation of very small detail in the images, it is argued here that training to identify such features can be undertaken on other devices where there is not the time constraints that exist during breast screening. A series of small pilot studies were then undertaken, trialling experienced radiologists with potential displays (PDAs and laptops) for mammographic image examination. These studies demonstrated that even on a PDA small mammographic features could be identified, albeit with difficulty, even with a very limited HCI manipulation tool. For training purposes the laptop, studied here with no HCI tool, was supported. Such promising results of display acceptability led to an investigation of mammographic inspection on displays of various sizes and resolutions. This study employed radiography students, potentially eventual screeners, who were eye tracked as they examined images on various sized displays. This showed that it could be possible to use a small PDA to deliver training. A detailed study then investigated whether aspects of an expert radiologist s visual inspection behaviour could be used to develop various training approaches. Four approaches were developed and examined using naĂŻve observers who were eye tracked as they were trained and tested. The approaches were found to be all feasible to implement but of variable usefulness for delivering mammographic interpretation training; this was confirmed by opinions from a focus group of screeners. On the basis of the previous studies, over a period of eight months, a large scale study involving 15 film readers from major breast screening centres was conducted where they examined series of digital mammograms on a clinical workstation, monitor and an iPhone. Overall results on individuals performance, image manipulation behaviour and visual search data indicated that a standard monitor could be employed successfully as an alternative for the digital workstation to deliver on-demand mammographic interpretation training using the full mammographic case images. The small iPhone, elicited poor performance, and was therefore judged not suitable for delivering training with the software employed here. However, future software developments may well overcome its shortcomings. The potential to implement training in China was examined by studying the current skill level of some practicing radiologists and an examination of how they responded to the developed training approaches. Results suggest that such an approach would be also applicable in other countries with different levels of screening skills. On-going further work is also discussed: the improvement of performance evaluation in mammography; new visual research on other breast imaging modalities and using visual search with computer aided detection to assist mammographic interpretation training.EThOS - Electronic Theses Online ServiceGBUnited Kingdo
    corecore