Subjective Annotation for a Frame Interpolation Benchmark using Artefact Amplification

Current benchmarks for optical flow algorithms evaluate the estimation either directly by comparing the predicted flow fields with the ground truth or indirectly by using the predicted flow fields for frame interpolation and then comparing the interpolated frames with the actual frames. In the latter case, objective quality measures such as the mean squared error are typically employed. However, it is well known that for image quality assessment, the actual quality experienced by the user cannot be fully deduced from such simple measures. Hence, we conducted a subjective quality assessment crowdscouring study for the interpolated frames provided by one of the optical flow benchmarks, the Middlebury benchmark. We collected forced-choice paired comparisons between interpolated images and corresponding ground truth. To increase the sensitivity of observers when judging minute difference in paired comparisons we introduced a new method to the field of full-reference quality assessment, called artefact amplification. From the crowdsourcing data, we reconstructed absolute quality scale values according to Thurstone's model. As a result, we obtained a re-ranking of the 155 participating algorithms w.r.t. the visual quality of the interpolated frames. This re-ranking not only shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks, the results also provide the ground truth for designing novel image quality assessment (IQA) methods dedicated to perceptual quality of interpolated images. As a first step, we proposed such a new full-reference method, called WAE-IQA. By weighing the local differences between an interpolated image and its ground truth WAE-IQA performed slightly better than the currently best FR-IQA approach from the literature.Comment: arXiv admin note: text overlap with arXiv:1901.0536

Exploiting Temporal Image Information in Minimally Invasive Surgery

Minimally invasive procedures rely on medical imaging instead of the surgeons direct vision. While preoperative images can be used for surgical planning and navigation, once the surgeon arrives at the target site real-time intraoperative imaging is needed. However, acquiring and interpreting these images can be challenging and much of the rich temporal information present in these images is not visible. The goal of this thesis is to improve image guidance for minimally invasive surgery in two main areas. First, by showing how high-quality ultrasound video can be obtained by integrating an ultrasound transducer directly into delivery devices for beating heart valve surgery. Secondly, by extracting hidden temporal information through video processing methods to help the surgeon localize important anatomical structures. Prototypes of delivery tools, with integrated ultrasound imaging, were developed for both transcatheter aortic valve implantation and mitral valve repair. These tools provided an on-site view that shows the tool-tissue interactions during valve repair. Additionally, augmented reality environments were used to add more anatomical context that aids in navigation and in interpreting the on-site video. Other procedures can be improved by extracting hidden temporal information from the intraoperative video. In ultrasound guided epidural injections, dural pulsation provides a cue in finding a clear trajectory to the epidural space. By processing the video using extended Kalman filtering, subtle pulsations were automatically detected and visualized in real-time. A statistical framework for analyzing periodicity was developed based on dynamic linear modelling. In addition to detecting dural pulsation in lumbar spine ultrasound, this approach was used to image tissue perfusion in natural video and generate ventilation maps from free-breathing magnetic resonance imaging. A second statistical method, based on spectral analysis of pixel intensity values, allowed blood flow to be detected directly from high-frequency B-mode ultrasound video. Finally, pulsatile cues in endoscopic video were enhanced through Eulerian video magnification to help localize critical vasculature. This approach shows particular promise in identifying the basilar artery in endoscopic third ventriculostomy and the prostatic artery in nerve-sparing prostatectomy. A real-time implementation was developed which processed full-resolution stereoscopic video on the da Vinci Surgical System

Reconstruction of solidification history from cast microstructure in remelted nickel alloy 718

Digital image analysis techniques were developed to autonomously characterize dendritic solidification microstructures and estimate melt pool profiles and solidification rates in remelted nickel alloy 718 ingots. Automated macrophotography was used to image dendritic microstructures in etched ingot cross-sections and create large image montages. Two analysis techniques, particle identification and two-point correlation function analysis, were developed to measure primary dendrite arm orientation and secondary dendrite arm spacing from these digital image montages. Particle identification techniques identified individual primary dendrite arms from the montage images. Primary dendrite arm orientations were measured from the geometry and location of the identified particles. A peak-counting technique was then implemented to measure secondary dendrite arm spacing after primary dendrite arms were identified. Two-point correlation functions were used to measure average primary dendrite arm orientations and secondary dendrite arm spacings from controlled image areas. Fourier analysis was then used to measure the primary dendrite arm orientation from the two-point correlation function. A peak-counting technique was used to measure secondary dendrite arm spacing after primary dendrite arm orientation was measured. The results produced using both analysis techniques were used to estimate melt pool profiles and solidification rates in a remelted alloy 718 ingot. Melt pool profile and solidification rate histories were calculated from primary dendrite arm orientations and secondary dendrite arm spacings, respectively. The techniques developed in this dissertation provide new technology and data needed by industry to validate computational process models of remelting processes such as electro-slag remelting (ESR) and vacuum-arc remelting (VAR).Mechanical Engineerin

Identification and mitigation of narrow spectral artifacts that degrade searches for persistent gravitational waves in the first two observing runs of Advanced LIGO

Searches are under way in Advanced LIGO and Virgo data for persistent gravitational waves from continuous sources, e.g. rapidly rotating galactic neutron stars, and stochastic sources, e.g. relic gravitational waves from the Big Bang or superposition of distant astrophysical events such as mergers of black holes or neutron stars. These searches can be degraded by the presence of narrow spectral artifacts (lines) due to instrumental or environmental disturbances. We describe a variety of methods used for finding, identifying and mitigating these artifacts, illustrated with particular examples. Results are provided in the form of lists of line artifacts that can safely be treated as non-astrophysical. Such lists are used to improve the efficiencies and sensitivities of continuous and stochastic gravitational wave searches by allowing vetoes of false outliers and permitting data cleaning

Identification and mitigation of narrow spectral artifacts that degrade searches for persistent gravitational waves in the first two observing runs of Advanced LIGO

Searches are under way in Advanced LIGO and Virgo data for persistent gravitational waves from continuous sources, e.g. rapidly rotating galactic neutron stars, and stochastic sources, e.g. relic gravitational waves from the Big Bang or superposition of distant astrophysical events such as mergers of black holes or neutron stars. These searches can be degraded by the presence of narrow spectral artifacts (lines) due to instrumental or environmental disturbances. We describe a variety of methods used for finding, identifying and mitigating these artifacts, illustrated with particular examples. Results are provided in the form of lists of line artifacts that can safely be treated as non-astrophysical. Such lists are used to improve the efficiencies and sensitivities of continuous and stochastic gravitational wave searches by allowing vetoes of false outliers and permitting data cleaning.Comment: 21 page

Identification and mitigation of narrow spectral artifacts that degrade searches for persistent gravitational waves in the first two observing runs of Advanced LIGO

Searches are under way in Advanced LIGO and Virgo data for persistent gravitational waves from continuous sources, e.g. rapidly rotating galactic neutron stars, and stochastic sources, e.g. relic gravitational waves from the Big Bang or superposition of distant astrophysical events such as mergers of black holes or neutron stars. These searches can be degraded by the presence of narrow spectral artifacts (lines) due to instrumental or environmental disturbances. We describe a variety of methods used for finding, identifying and mitigating these artifacts, illustrated with particular examples. Results are provided in the form of lists of line artifacts that can safely be treated as non-astrophysical. Such lists are used to improve the efficiencies and sensitivities of continuous and stochastic gravitational wave searches by allowing vetoes of false outliers and permitting data cleaning

Identification and mitigation of narrow spectral artifacts that degrade searches for persistent gravitational waves in the first two observing runs of Advanced LIGO

Searches are under way in Advanced LIGO and Virgo data for persistent gravitational waves from continuous sources, e.g. rapidly rotating galactic neutron stars, and stochastic sources, e.g. relic gravitational waves from the Big Bang or superposition of distant astrophysical events such as mergers of black holes or neutron stars. These searches can be degraded by the presence of narrow spectral artifacts (lines) due to instrumental or environmental disturbances. We describe a variety of methods used for finding, identifying and mitigating these artifacts, illustrated with particular examples. Results are provided in the form of lists of line artifacts that can safely be treated as non-astrophysical. Such lists are used to improve the efficiencies and sensitivities of continuous and stochastic gravitational wave searches by allowing vetoes of false outliers and permitting data cleaning