Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

The Cosmic Microwave Background & Inflation, Then & Now

Boomerang, Maxima, DASI, CBI and VSA significantly increase the case for accelerated expansion in the early universe (the inflationary paradigm) and at the current epoch (dark energy dominance), especially when combined with data on high redshift supernovae (SN1) and large scale structure (LSS). There are ``7 pillars of Inflation'' that can be shown with the CMB probe, and at least 5, and possibly 6, of these have already been demonstrated in the CMB data: (1) a large scale gravitational potential; (2) acoustic peaks/dips; (3) damping due to shear viscosity; (4) a Gaussian (maximally random) distribution; (5) secondary anisotropies; (6) polarization. A 7th pillar, anisotropies induced by gravity wave quantum noise, could be too small. A minimal inflation parameter set, \omega_b,\omega_{cdm}, \Omega_{tot}, \Omega_Q,w_Q,n_s,\tau_C, \sigma_8}, is used to illustrate the power of the current data. We find the CMB+LSS+SN1 data give \Omega_{tot} =1.00^{+.07}_{-.03}, consistent with (non-baroque) inflation theory. Restricting to \Omega_{tot}=1, we find a nearly scale invariant spectrum, n_s =0.97^{+.08}_{-.05}. The CDM density, \Omega_{cdm}{\rm h}^2 =.12^{+.01}_{-.01}, and baryon density, \Omega_b {\rm h}^2 = >.022^{+.003}_{-.002}, are in the expected range. (The Big Bang nucleosynthesis estimate is 0.019\pm 0.002.) Substantial dark (unclustered) energy is inferred, \Omega_Q \approx 0.68 \pm 0.05, and CMB+LSS \Omega_Q values are compatible with the independent SN1 estimates. The dark energy equation of state, crudely parameterized by a quintessence-field pressure-to-density ratio w_Q, is not well determined by CMB+LSS (w_Q < -0.4 at 95% CL), but when combined with SN1 the resulting w_Q < -0.7 limit is quite consistent with the w_Q=-1 cosmological constant case.Comment: 20 pages, 8 figures, in Theoretical Physics, MRST 2002: A Tribute to George Libbrandt (AIP), eds. V. Elias, R. Epp, R. Myer

Resolving depth measurement ambiguity with commercially available range imaging cameras

Time-of-flight range imaging is typically performed with the amplitude modulated continuous wave method. This involves illuminating a scene with amplitude modulated light. Reflected light from the scene is received by the sensor with the range to the scene encoded as a phase delay of the modulation envelope. Due to the cyclic nature of phase, an ambiguity in the measured range occurs every half wavelength in distance, thereby limiting the maximum useable range of the camera. This paper proposes a procedure to resolve depth ambiguity using software post processing. First, the range data is processed to segment the scene into separate objects. The average intensity of each object can then be used to determine which pixels are beyond the non-ambiguous range. The results demonstrate that depth ambiguity can be resolved for various scenes using only the available depth and intensity information. This proposed method reduces the sensitivity to objects with very high and very low reflectance, normally a key problem with basic threshold approaches. This approach is very flexible as it can be used with any range imaging camera. Furthermore, capture time is not extended, keeping the artifacts caused by moving objects at a minimum. This makes it suitable for applications such as robot vision where the camera may be moving during captures. The key limitation of the method is its inability to distinguish between two overlapping objects that are separated by a distance of exactly one non-ambiguous range. Overall the reliability of this method is higher than the basic threshold approach, but not as high as the multiple frequency method of resolving ambiguity

NOVEL DENSE STEREO ALGORITHMS FOR HIGH-QUALITY DEPTH ESTIMATION FROM IMAGES

This dissertation addresses the problem of inferring scene depth information from a collection of calibrated images taken from different viewpoints via stereo matching. Although it has been heavily investigated for decades, depth from stereo remains a long-standing challenge and popular research topic for several reasons. First of all, in order to be of practical use for many real-time applications such as autonomous driving, accurate depth estimation in real-time is of great importance and one of the core challenges in stereo. Second, for applications such as 3D reconstruction and view synthesis, high-quality depth estimation is crucial to achieve photo realistic results. However, due to the matching ambiguities, accurate dense depth estimates are difficult to achieve. Last but not least, most stereo algorithms rely on identification of corresponding points among images and only work effectively when scenes are Lambertian. For non-Lambertian surfaces, the brightness constancy assumption is no longer valid. This dissertation contributes three novel stereo algorithms that are motivated by the specific requirements and limitations imposed by different applications. In addressing high speed depth estimation from images, we present a stereo algorithm that achieves high quality results while maintaining real-time performance. We introduce an adaptive aggregation step in a dynamic-programming framework. Matching costs are aggregated in the vertical direction using a computationally expensive weighting scheme based on color and distance proximity. We utilize the vector processing capability and parallelism in commodity graphics hardware to speed up this process over two orders of magnitude. In addressing high accuracy depth estimation, we present a stereo model that makes use of constraints from points with known depths - the Ground Control Points (GCPs) as referred to in stereo literature. Our formulation explicitly models the influences of GCPs in a Markov Random Field. A novel regularization prior is naturally integrated into a global inference framework in a principled way using the Bayes rule. Our probabilistic framework allows GCPs to be obtained from various modalities and provides a natural way to integrate information from various sensors. In addressing non-Lambertian reflectance, we introduce a new invariant for stereo correspondence which allows completely arbitrary scene reflectance (bidirectional reflectance distribution functions - BRDFs). This invariant can be used to formulate a rank constraint on stereo matching when the scene is observed by several lighting configurations in which only the lighting intensity varies

Photochemistry of HOx in the upper troposphere at northern midlatitudes

The factors controlling the concentrations of HOx radicals (= OH + peroxy) in the upper troposphere (8-12 km) are examined using concurrent aircraft observations of OH, HO2, H2O2, CH3OOH, and CH2O made during the Subsonic Assessment Ozone and Nitrogen Oxide Experiment (SONEX) at northern midlatitudes in the fall. These observations, complemented by concurrent measurements of O3, H2O, NO, peroxyacetyl nitrate (PAN), HNO3, CH4, CO, acetone, hydrocarbons, actinic fluxes, and aerosols, allow a highly constrained mass balance analysis of HOx and of the larger chemical family HOy (= HOx + 2 H2O2 + 2 CH3OOH + HNO2 + HNO4). Observations of OH and HO2 are successfully simulated to within 40% by a diel steady state model constrained with observed H2O2 and CH3OOH. The model captures 85% of the observed HOx variance, which is driven mainly by the concentrations of NOx (= NO + NO2) and by the strength of the HOx primary sources. Exceptions to the good agreement between modeled and observed HOx are at sunrise and sunset, where the model is too low by factors of 2-5, and inside cirrus clouds, where the model is too high by factors of 1.2-2. Heterogeneous conversion of NO2 to HONO on aerosols (γNO2=10-3) during the night followed by photolysis of HONO could explain part of the discrepancy at sunrise. Heterogeneous loss of HO2 on ice crystals (γice_HO2=0.025) could explain the discrepancy in cirrus. Primary sources of HOx from O(1D)+H2O and acetone photolysis were of comparable magnitude during SONEX. The dominant sinks of HOy were OH+HO2 (NOx&lt;50 parts per trillion by volume (pptv)) and OH+HNO4 (NOx&gt;50 pptv). Observed H2O2 concentrations are reproduced by model calculations to within 50% if one allows in the model for heterogeneous conversion of HO2 to H2O2 on aerosols (γHO2=0.2). Observed CH3OOH concentrations are underestimated by a factor of 2 on average. Observed CH2O concentrations were usually below the 50 pptv detection limit, consistent with model results; however, frequent occurrences of high values in the observations (up to 350 pptv) are not captured by the model. These high values are correlated with high CH3OH and with cirrus clouds. Heterogeneous oxidation of CH3OH to CH2O on aerosols or ice crystals might provide an explanation (γice_CH3OH∼0.01 would be needed). Copyright 2000 by the American Geophysical Union

Advances in Above- and In-Water Radiometry, Volume 1: Enhanced Legacy and State-of-the-Art Instrument Suites

This publication documents the scientific advances associated with new instrument systems and accessories built to improve above- and in-water observations of the apparent optical properties (AOPs) of aquatic ecosystems. The perspective is to obtain high quality data in offshore, nearshore, and inland waters with equal efficacy. The principal objective is to be prepared for the launch of the next-generation ocean color satellites with the most capable commercial off-the-shelf (COTS) instrumentation in the shortest time possible. The technologies described herein are designed to either improve legacy radiometric systems or to provide entirely new hybrid sampling capabilities, so as to satisfy the requirements established for diverse remote sensing requirements. Both above- and in-water instrument suites are documented with software options for autonomous control of data collection activities. The latter includes an airborne instrument system plus unmanned surface vessel (USV) and buoy concepts

3-D Scene Reconstruction from Aerial Imagery

3-D scene reconstructions derived from Structure from Motion (SfM) and Multi-View Stereo (MVS) techniques were analyzed to determine the optimal reconnaissance flight characteristics suitable for target reconstruction. In support of this goal, a preliminary study of a simple 3-D geometric object facilitated the analysis of convergence angles and number of camera frames within a controlled environment. Reconstruction accuracy measurements revealed at least 3 camera frames and a 6 convergence angle were required to achieve results reminiscent of the original structure. The central investigative effort sought the applicability of certain airborne reconnaissance flight profiles to reconstructing ground targets. The data sets included images collected within a synthetic 3-D urban environment along circular, linear and s-curve aerial flight profiles equipped with agile and non-agile sensors. S-curve and dynamically controlled linear flight paths provided superior results, whereas with sufficient data conditioning and combination of orthogonal flight paths, all flight paths produced quality reconstructions under a wide variety of operational considerations

Wind Retrieval Algorithms for the IWRAP and HIWRAP Airborne Doppler Radars with Applications to Hurricanes

Algorithms for the retrieval of atmospheric winds in precipitating systems from downward-pointing, conically-scanning airborne Doppler radars are presented. The focus in the paper is on two radars: the Imaging Wind and Rain Airborne Profiler(IWRAP) and the High-altitude IWRAP (HIWRAP). The IWRAP is a dual-frequency (Cand Ku band), multi-beam (incidence angles of 30 50) system that flies on the NOAAWP-3D aircraft at altitudes of 2-4 km. The HIWRAP is a dual-frequency (Ku and Kaband), dual-beam (incidence angles of 30 and 40) system that flies on the NASA Global Hawk aircraft at altitudes of 18-20 km. Retrievals of the three Cartesian wind components over the entire radar sampling volume are described, which can be determined using either a traditional least squares or variational solution procedure. The random errors in the retrievals are evaluated using both an error propagation analysis and a numerical simulation of a hurricane. These analyses show that the vertical and along-track wind errors have strong across-track dependence with values of 0.25 m s-1 at nadir to 2.0 m s-1 and 1.0 m s-1 at the swath edges, respectively. The across-track wind errors also have across-track structure and are on average, 3.0 3.5 m s-1 or 10 of the hurricane wind speed. For typical rotated figure four flight patterns through hurricanes, the zonal and meridional wind speed errors are 2 3 m s-1.Examples of measured data retrievals from IWRAP during an eyewall replacement cycle in Hurricane Isabel (2003) and from HIWRAP during the development of Tropical Storm Matthew (2010) are shown