Effects of common image manipulations on diagnostic performance in digital pathology: human study

A very recent work of Ref.[1] studied the effects of image manipulation and image degradation on the perceived attributes of image quality (IQ) of digital pathology slides. However, before any conclusions and recommendations can be formulated regarding specific image manipulations (and IQ attributes), it is necessary to investigate their effects on the diagnostic performance of clinicians when interpreting these images. In this study, 6 expert pathologists interpreted digital images of H&E stained animal pathology samples in a free-response (FROC) experiment. Participants marked locations suspicious for viral inclusions (inclusion bodies) and rated them using a continuous scale from 0 (low confidence) to 100% (high confidence). The images were the same as in Ref.[1]: crops of digital pathology slides of 3 different animal tissue samples, all 1200×750 pixels in size. Each participant viewed a total of 72 images: 12 nonmanipulated (reference) images (4 of each tissue type), and 60 manipulated images (5 for each reference image). The extent of artificial manipulations was adjusted relative to the reference images using the HDR-VDP metric [2] in the luminance domain: added Gaussian blur (sb=3), decreased gamma (-5%), added white Gaussian noise (sn=10), decreased color saturation (-5%), and JPG compression (libjpeg 50). The images were displayed on a 3MP medical color LCD in a controlled viewing environment. Preliminary analysis assessing the change in the number of positive markings in the reference and manipulated images indicates that blurring and changes in gamma, followed by changes in color saturation, could have an effect on diagnostic performance. This largely coincides with the findings from Ref.[1], where IQ ratings appeared to be most affected by changes in color and gamma parameters. Importantly, diagnostic performance appears to be content dependent; it is different across tissue types. Further data analysis (including JAFROC) is ongoing and shall be reported in the conference talk

Influence of study design on digital pathology image quality evaluation : the need to define a clinical task

Despite the current rapid advance in technologies for whole slide imaging, there is still no scientific consensus on the recommended methodology for image quality assessment of digital pathology slides. For medical images in general, it has been recommended to assess image quality in terms of doctors’ success rates in performing a specific clinical task while using the images (clinical image quality, cIQ). However, digital pathology is a new modality, and already identifying the appropriate task is difficult. In an alternative common approach, humans are asked to do a simpler task such as rating overall image quality (perceived image quality, pIQ), but that involves the risk of nonclinically relevant findings due to an unknown relationship between the pIQ and cIQ. In this study, we explored three different experimental protocols: (1) conducting a clinical task (detecting inclusion bodies), (2) rating image similarity and preference, and (3) rating the overall image quality. Additionally, within protocol 1, overall quality ratings were also collected (task-aware pIQ). The experiments were done by diagnostic veterinary pathologists in the context of evaluating the quality of hematoxylin and eosin-stained digital pathology slides of animal tissue samples under several common image alterations: additive noise, blurring, change in gamma, change in color saturation, and JPG compression. While the size of our experiments was small and prevents drawing strong conclusions, the results suggest the need to define a clinical task. Importantly, the pIQ data collected under protocols 2 and 3 did not always rank the image alterations the same as their cIQ from protocol 1, warning against using conventional pIQ to predict cIQ. At the same time, there was a correlation between the cIQ and task-aware pIQ ratings from protocol 1, suggesting that the clinical experiment context (set by specifying the clinical task) may affect human visual attention and bring focus to their criteria of image quality. Further research is needed to assess whether and for which purposes (e.g., preclinical testing) task-aware pIQ ratings could substitute cIQ for a given clinical task

Eye Tracking Methods for Analysis of Visuo-Cognitive Behavior in Medical Imaging

Predictive modeling of human visual search behavior and the underlying metacognitive processes is now possible thanks to significant advances in bio-sensing device technology and machine intelligence. Eye tracking bio-sensors, for example, can measure psycho-physiological response through change events in configuration of the human eye. These events include positional changes such as visual fixation, saccadic movements, and scanpath, and non-positional changes such as blinks and pupil dilation and constriction. Using data from eye-tracking sensors, we can model human perception, cognitive processes, and responses to external stimuli. In this study, we investigated the visuo-cognitive behavior of clinicians during the diagnostic decision process for breast cancer screening under clinically equivalent experimental conditions involving multiple monitors and breast projection views. Using a head-mounted eye tracking device and a customized user interface, we recorded eye change events and diagnostic decisions from 10 clinicians (three breast-imaging radiologists and seven Radiology residents) for a corpus of 100 screening mammograms (comprising cases of varied pathology and breast parenchyma density). We proposed novel features and gaze analysis techniques, which help to encode discriminative pattern changes in positional and non-positional measures of eye events. These changes were shown to correlate with individual image readers' identity and experience level, mammographic case pathology and breast parenchyma density, and diagnostic decision. Furthermore, our results suggest that a combination of machine intelligence and bio-sensing modalities can provide adequate predictive capability for the characterization of a mammographic case and image readers diagnostic performance. Lastly, features characterizing eye movements can be utilized for biometric identification purposes. These findings are impactful in real-time performance monitoring and personalized intelligent training and evaluation systems in screening mammography. Further, the developed algorithms are applicable in other application domains involving high-risk visual tasks

Camera based Display Image Quality Assessment

This thesis presents the outcomes of research carried out by the PhD candidate Ping Zhao during 2012 to 2015 in Gjøvik University College. The underlying research was a part of the HyPerCept project, in the program of Strategic Projects for University Colleges, which was funded by The Research Council of Norway. The research was engaged under the supervision of Professor Jon Yngve Hardeberg and co-supervision of Associate Professor Marius Pedersen, from The Norwegian Colour and Visual Computing Laboratory, in the Faculty of Computer Science and Media Technology of Gjøvik University College; as well as the co-supervision of Associate Professor Jean-Baptiste Thomas, from The Laboratoire Electronique, Informatique et Image, in the Faculty of Computer Science of Universit´e de Bourgogne. The main goal of this research was to develop a fast and an inexpensive camera based display image quality assessment framework. Due to the limited time frame, we decided to focus only on projection displays with static images displayed on them. However, the proposed methods were not limited to projection displays, and they were expected to work with other types of displays, such as desktop monitors, laptop screens, smart phone screens, etc., with limited modifications. The primary contributions from this research can be summarized as follows: 1. We proposed a camera based display image quality assessment framework, which was originally designed for projection displays but it can be used for other types of displays with limited modifications. 2. We proposed a method to calibrate the camera in order to eliminate unwanted vignetting artifact, which is mainly introduced by the camera lens. 3. We proposed a method to optimize the camera’s exposure with respect to the measured luminance of incident light, so that after the calibration all camera sensors share a common linear response region. 4. We proposed a marker-less and view-independent method to register one captured image with its original at a sub-pixel level, so that we can incorporate existing full reference image quality metrics without modifying them. 5. We identified spatial uniformity, contrast and sharpness as the most important image quality attributes for projection displays, and we used the proposed framework to evaluate the prediction performance of the state-of-the-art image quality metrics regarding these attributes. The proposed image quality assessment framework is the core contribution of this research. Comparing to conventional image quality assessment approaches, which were largely based on the measurements of colorimeter or spectroradiometer, using camera as the acquisition device has the advantages of quickly recording all displayed pixels in one shot, relatively inexpensive to purchase the instrument. Therefore, the consumption of time and resources for image quality assessment can be largely reduced. We proposed a method to calibrate the camera in order to eliminate unwanted vignetting artifact primarily introduced by the camera lens. We used a hazy sky as a closely uniform light source, and the vignetting mask was generated with respect to the median sensor responses over i only a few rotated shots of the same spot on the sky. We also proposed a method to quickly determine whether all camera sensors were sharing a common linear response region. In order to incorporate existing full reference image quality metrics without modifying them, an accurate registration of pairs of pixels between one captured image and its original is required. We proposed a marker-less and view-independent image registration method to solve this problem. The experimental results proved that the proposed method worked well in the viewing conditions with a low ambient light. We further identified spatial uniformity, contrast and sharpness as the most important image quality attributes for projection displays. Subsequently, we used the developed framework to objectively evaluate the prediction performance of the state-of-art image quality metrics regarding these attributes in a robust manner. In this process, the metrics were benchmarked with respect to the correlations between the prediction results and the perceptual ratings collected from subjective experiments. The analysis of the experimental results indicated that our proposed methods were effective and efficient. Subjective experiment is an essential component for image quality assessment; however it can be time and resource consuming, especially in the cases that additional image distortion levels are required to extend the existing subjective experimental results. For this reason, we investigated the possibility of extending subjective experiments with baseline adjustment method, and we found that the method could work well if appropriate strategies were applied. The underlying strategies referred to the best distortion levels to be included in the baseline, as well as the number of them

Towards greater clarity for the analysis of imaging studies: Development & validation of an alternative to the area under the receiver-operator characteristic curve.

This thesis arose from a 2006 study performed by the author and his collaborators that attempted to gain regulatory approval for computer-assisted detection (CAD) software. The USA Food & Drug Administration (FDA) obliged us to use the change in the area under the receiver-operator characteristic curve (ROC AUC) as our primary outcome. Despite its wide dissemination in radiology research, we found implementation of ROC AUC very problematic. This thesis explores the hurdles we encountered and argues for an alternative approach. Chapter 1 describes the rationale for and against ROC AUC as a measure of diagnostic performance. An alternative analysis based on net benefit is proposed on the basis that it is more transparent and simpler to interpret. Chapter 2 uses the net benefit method to analyse a multi-reader multi-case (MRMC) study of CAD for CT colonography. The analysis requires an estimate of relative misclassification costs for false-negative versus false-positive diagnoses; “W”. This study used a conservative value for W, arrived at via consensus. In Chapter 3 an evidence-based value for W in the context of screening for colorectal cancer and polyps by CT colonography is arrived at via a discrete choice experiment (DCE) of patients and healthcare workers. Chapter 4 uses the value for W obtained in Chapter 3 in a net benefit analysis to compare observer performance in two MRMC studies of CAD for CT colonography. Chapter 5 obtains W by DCE for a different clinical context – detection of extracolonic pathology by CT colonography. Chapter 6 describes a systematic review that aims to determine whether reporting of MRMC ROC AUC methods in the radiological literature is comprehensive. Chapter 7 then provides guidelines for the comprehensive reporting of MRMC ROC AUC studies. The thesis finishes with a summary of the work performed and suggestions for further research

Texture and Colour in Image Analysis

Research in colour and texture has experienced major changes in the last few years. This book presents some recent advances in the field, specifically in the theory and applications of colour texture analysis. This volume also features benchmarks, comparative evaluations and reviews

Digital Light

Light symbolises the highest good, it enables all visual art, and today it lies at the heart of billion-dollar industries. The control of light forms the foundation of contemporary vision. Digital Light brings together artists, curators, technologists and media archaeologists to study the historical evolution of digital light-based technologies. Digital Light provides a critical account of the capacities and limitations of contemporary digital light-based technologies and techniques by tracing their genealogies and comparing them with their predecessor media. As digital light remediates multiple historical forms (photography, print, film, video, projection, paint), the collection draws from all of these histories, connecting them to the digital present and placing them in dialogue with one another. Light is at once universal and deeply historical. The invention of mechanical media (including photography and cinematography) allied with changing print technologies (half-tone, lithography) helped structure the emerging electronic media of television and video, which in turn shaped the bitmap processing and raster display of digital visual media. Digital light is, as Stephen Jones points out in his contribution, an oxymoron: light is photons, particulate and discrete, and therefore always digital. But photons are also waveforms, subject to manipulation in myriad ways. From Fourier transforms to chip design, colour management to the translation of vector graphics into arithmetic displays, light is constantly disciplined to human purposes. In the form of fibre optics, light is now the infrastructure of all our media; in urban plazas and handheld devices, screens have become ubiquitous, and also standardised. This collection addresses how this occurred, what it means, and how artists, curators and engineers confront and challenge the constraints of increasingly normalised digital visual media. While various art pieces and other content are considered throughout the collection, the focus is specifically on what such pieces suggest about the intersection of technique and technology. Including accounts by prominent artists and professionals, the collection emphasises the centrality of use and experimentation in the shaping of technological platforms. Indeed, a recurring theme is how techniques of previous media become technologies, inscribed in both digital software and hardware. Contributions include considerations of image-oriented software and file formats; screen technologies; projection and urban screen surfaces; histories of computer graphics, 2D and 3D image editing software, photography and cinematic art; and transformations of light-based art resulting from the distributed architectures of the internet and the logic of the database. Digital Light brings together high profile figures in diverse but increasingly convergent fields, from academy award-winner and co-founder of Pixar, Alvy Ray Smith to feminist philosopher Cathryn Vasseleu