Fast occlusion sweeping

Abstract. While realistic illumination significantly improves the visual quality and perception of rendered images, it is often very expensive to compute. In this paper, we propose a new algorithm for embedding a global ambient occlusion computation within the fast sweeping algorithm while determining isosurfaces. With this method we can approximate ambient occlusion for rendering volumetric data with minimal additional cost over fast sweeping. We compare visualizations rendered with our algorithm to visualizations computed with only local shading, and with a ambient occlusion calculation using Monte Carlo sampling method. We also show how this method can be used for approximating low frequency shadows and subsurface scattering. Realistic illumination techniques used in digitally synthesized images are known to greatly enhance the perception of shape. This is as true for renderings of volume data as it is for geometric models. For example, Qiu et al. [1] used full global illumination techniques to improve visualizations of volumetric data, and Stewart [2] shows how computation of local ambient occlusion enhances the perception of grooves in a brain CT scanned dataset. Tarini et al. In this paper, we provide a new solution for ambient occlusion computation that is significantly faster than existing techniques. The method integrates well with a volumetric ray marching algorithm implemented on the GPU. While not a full global illumination solution, ambient occlusion provides a more realistic illumination model than does local illumination, and permits the use of realistic light sources, like skylights. For accelerating our ray marching algorithm, we build a volumetric signed distance field using the fast sweeping method, and we embed our ambient occlusion approximatio

Fast and Accurate Depth Estimation from Sparse Light Fields

We present a fast and accurate method for dense depth reconstruction from sparsely sampled light fields obtained using a synchronized camera array. In our method, the source images are over-segmented into non-overlapping compact superpixels that are used as basic data units for depth estimation and refinement. Superpixel representation provides a desirable reduction in the computational cost while preserving the image geometry with respect to the object contours. Each superpixel is modeled as a plane in the image space, allowing depth values to vary smoothly within the superpixel area. Initial depth maps, which are obtained by plane sweeping, are iteratively refined by propagating good correspondences within an image. To ensure the fast convergence of the iterative optimization process, we employ a highly parallel propagation scheme that operates on all the superpixels of all the images at once, making full use of the parallel graphics hardware. A few optimization iterations of the energy function incorporating superpixel-wise smoothness and geometric consistency constraints allows to recover depth with high accuracy in textured and textureless regions as well as areas with occlusions, producing dense globally consistent depth maps. We demonstrate that while the depth reconstruction takes about a second per full high-definition view, the accuracy of the obtained depth maps is comparable with the state-of-the-art results.Comment: 15 pages, 15 figure

Autonomous Tissue Scanning under Free-Form Motion for Intraoperative Tissue Characterisation

In Minimally Invasive Surgery (MIS), tissue scanning with imaging probes is required for subsurface visualisation to characterise the state of the tissue. However, scanning of large tissue surfaces in the presence of deformation is a challenging task for the surgeon. Recently, robot-assisted local tissue scanning has been investigated for motion stabilisation of imaging probes to facilitate the capturing of good quality images and reduce the surgeon's cognitive load. Nonetheless, these approaches require the tissue surface to be static or deform with periodic motion. To eliminate these assumptions, we propose a visual servoing framework for autonomous tissue scanning, able to deal with free-form tissue deformation. The 3D structure of the surgical scene is recovered and a feature-based method is proposed to estimate the motion of the tissue in real-time. A desired scanning trajectory is manually defined on a reference frame and continuously updated using projective geometry to follow the tissue motion and control the movement of the robotic arm. The advantage of the proposed method is that it does not require the learning of the tissue motion prior to scanning and can deal with free-form deformation. We deployed this framework on the da Vinci surgical robot using the da Vinci Research Kit (dVRK) for Ultrasound tissue scanning. Since the framework does not rely on information from the Ultrasound data, it can be easily extended to other probe-based imaging modalities.Comment: 7 pages, 5 figures, ICRA 202

An ultra-fast digital diffuse optical spectroscopic imaging system for neoadjuvant chemotherapy monitoring

Up to 20% of breast cancer patients who undergo presurgical (neoadjuvant) chemotherapy have no response to treatment. Standard-of-care imaging modalities, including MRI, CT, mammography, and ultrasound, measure anatomical features and tumor size that reveal response only after months of treatment. Recently, non-invasive, near-infrared optical markers have shown promise in indicating the efficacy of treatment at the outset of the chemotherapy treatment. For example, frequency domain Diffuse Optical Spectroscopic Imaging (DOSI) can be used to characterize the optical scattering and absorption properties of thick tissue, including breast tumors. These parameters can then be used to calculate tissue concentrations of chromophores, including oxyhemoglobin, deoxyhemoglobin, water, and lipids. Tumors differ in hemoglobin concentration, as compared with healthy background tissue, and changes in hemoglobin concentration during neoadjuvant chemotherapy have been shown to correlate with efficacy of treatment. Using DOSI early in treatment to measure chromophore concentrations may be a powerful tool for guiding neoadjuvant chemotherapy treatment. Previous frequency-domain DOSI systems have been limited by large device footprints, complex electronics, high costs, and slow acquisition speeds, all of which complicate access to patients in the clinical setting. In this work a new digital DOSI (dDOSI) system has been developed, which is relatively inexpensive and compact, allowing for use at the bedside, while providing unprecedented measurement speeds. The system builds on, and significantly advances, previous dDOSI setups developed by our group and, for the first time, utilizes hardware-integrated custom board-level direct digital synthesizers (DDS) and analog to digital converters (ADC) to generate and directly measure signals utilizing undersampling techniques. The dDOSI system takes high-speed optical measurements by utilizing wavelength multiplexing while sweeping through hundreds of modulation frequencies in tens of milliseconds. The new dDOSI system is fast, inexpensive, and compact without compromising accuracy and precision

Automated model acquisition from range images with view planning

We present an incremental system that builds accurate CAD models of objects from multiple range images. Using a hybrid of surface mesh and volumetric representations, the system creates a "water-tight" 3D model at each step of the modeling process, allowing reasonable models to be built from a small number of views. We also present a method that can be used to plan the next view and reduce the number of scans needed to recover the object. Results are presented for the creation of 3D models of a computer game controller, a hip joint prosthesis, and a mechanical strut