Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Advances in Stereo Vision

Stereopsis is a vision process whose geometrical foundation has been known for a long time, ever since the experiments by Wheatstone, in the 19th century. Nevertheless, its inner workings in biological organisms, as well as its emulation by computer systems, have proven elusive, and stereo vision remains a very active and challenging area of research nowadays. In this volume we have attempted to present a limited but relevant sample of the work being carried out in stereo vision, covering significant aspects both from the applied and from the theoretical standpoints

Avian surface reconstruction in free-flight with application to flight stability analysis of a barn owl and peregrine falcon

Birds primarily create and control the forces necessary for flight through changing the shape and orientation of their wings and tail. Their wing geometry is characterised by complex variation in parameters such as camber, twist, sweep and dihedral. To characterise this complexity, a multi-stereo photogrammetry setup was developed for accurately measuring surface geometry in high-resolution during free-flight. The natural patterning of the birds was used as the basis for phase correlation-based image matching, allowing indoor or outdoor use while being non-intrusive for the birds. The accuracy of the method was quantified and shown to be sufficient for characterising the geometric parameters of interest, but with a reduction in accuracy close to the wing edge and in some localized regions. To demonstrate the method's utility, surface reconstructions are presented for a barn owl (Tyto alba) and peregrine falcon (Falco peregrinus) during three instants of gliding flight per bird. The barn owl flew with a consistent geometry, with positive wing camber and longitudinal anhedral. Based on flight dynamics theory this suggests it was longitudinally statically unstable during these flights. The peregrine flew with a consistent glide angle, but at a range of airspeeds with varying geometry. Unlike the barn owl, its glide configuration did not provide a clear indication of longitudinal static stability/instability. Aspects of the geometries adopted by both birds appeared to be related to control corrections and this method would be well suited for future investigations in this area, as well as for other quantitative studies into avian flight dynamics.Flight O1 - original uncompressed tif images for flight O1 of the barn owlO1_images.zipFlight O2 - original uncompressed tif images for flight O2 of the barn owlO2_images.zipFlight O3 - original uncompressed tif images for flight O3 of the barn owlO3_images.zipFlight P1 - original uncompressed tif images for flight P1 of the peregrineP1_images.zipFlight P2 - original uncompressed tif images for flight P2 of the peregrineP2_images.zipFlight P3 - original uncompressed tif images for flight P3 of the peregrineP3_images.zipREADM

High resolution radargrammetry with COSMO-SkyMed, TerraSAR-X and RADARSAT-2 imagery: development and implementation of an image orientation model for Digital Surface Model generation

Digital Surface and Terrain Models (DSM/DTM) have large relevance in several territorial applications, such as topographic mapping, monitoring engineering, geology, security, land planning and management of Earth's resources. The satellite remote sensing data offer the opportunity to have continuous observation of Earth's surface for territorial application, with short acquisition and revisit times. Meeting these requirements, the SAR (Synthetic Aperture Radar) high resolution satellite imagery could offer night-and-day and all-weather functionality (clouds, haze and rain penetration). Two different methods may be used in order to generate DSMs from SAR data: the interferometric and the radargrammetric approaches. The radargrammetry uses only the intensity information of the SAR images and reconstructs the 3D information starting from a couple of images similarly to photogrammetry. Radargrammetric DSM extraction procedure consists of two basic steps: the stereo pair orientation and the image matching for the automatic detection of homologous points. The goal of this work is the definition and the implementation of a geometric model in order to orientate SAR imagery in zero Doppler geometry. The radargrammetric model implemented in SISAR (Software per Immagini Satellitari ad Alta Risoluzione - developed at the Geodesy and Geomatic Division - University of Rome "La Sapienza") is based on the equation of radar target acquisition and zero Doppler focalization Moreover a tool for the SAR Rational Polynomial Coefficients (RPCs) generation has been implemented in SISAR software, similarly to the one already developed for the optical sensors. The possibility to generate SAR RPCs starting from a radargrammetric model sounds of particular interest since, at present, the most part of SAR imagery is not supplied with RPCs, although the RPFs model is available in several commercial software. Only RADARSAT-2 data are supplied with vendors RPCs. To test the effectiveness of the implemented RPCs generation tool and the SISAR radargrammetric orientation model the reference results were computed: the stereo pairs were orientated with the two model. The tests were carried out on several test site using COSMO-SkyMed, TerraSAR-X and RADARSAT-2 data. Moreover, to evaluate the advantages and the different accuracy between the orientation models computed without GCPs and the orientation model with GCPs a Monte Carlo test was computed. At last, to define the real effectiveness of radargrammetric technique for DSM extraction and to compare the radrgrammetric tool implemented in a commercial software PCI-Geomatica v. 2012 and SISAR software, the images acquired on Beauport test site were used for DSM extraction. It is important underline that several test were computed. Part of this tests were carried out under the supervision of Prof. Thierry Toutin at CCRS (Canada Centre of Remote Sensing) where the PCI-Geomatica orientation model was developed, in order to check the better parameters solution to extract radargrammetric DSMs. In conclusion, the results obtained are representative of the geometric potentialities of SAR stereo pairs as regards 3D surface reconstruction

Pose Graph Optimization for Unsupervised Monocular Visual Odometry

Unsupervised Learning based monocular visual odometry (VO) has lately drawn significant attention for its potential in label-free leaning ability and robustness to camera parameters and environmental variations. However, partially due to the lack of drift correction technique, these methods are still by far less accurate than geometric approaches for large-scale odometry estimation. In this paper, we propose to leverage graph optimization and loop closure detection to overcome limitations of unsupervised learning based monocular visual odometry. To this end, we propose a hybrid VO system which combines an unsupervised monocular VO called NeuralBundler with a pose graph optimization back-end. NeuralBundler is a neural network architecture that uses temporal and spatial photometric loss as main supervision and generates a windowed pose graph consists of multi-view 6DoF constraints. We propose a novel pose cycle consistency loss to relieve the tensions in the windowed pose graph, leading to improved performance and robustness. In the back-end, a global pose graph is built from local and loop 6DoF constraints estimated by NeuralBundler and is optimized over SE(3). Empirical evaluation on the KITTI odometry dataset demonstrates that 1) NeuralBundler achieves state-of-the-art performance on unsupervised monocular VO estimation, and 2) our whole approach can achieve efficient loop closing and show favorable overall translational accuracy compared to established monocular SLAM systems.Comment: Accepted to ICRA'201