Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Structured Light-Based 3D Reconstruction System for Plants.

Camera-based 3D reconstruction of physical objects is one of the most popular computer vision trends in recent years. Many systems have been built to model different real-world subjects, but there is lack of a completely robust system for plants. This paper presents a full 3D reconstruction system that incorporates both hardware structures (including the proposed structured light system to enhance textures on object surfaces) and software algorithms (including the proposed 3D point cloud registration and plant feature measurement). This paper demonstrates the ability to produce 3D models of whole plants created from multiple pairs of stereo images taken at different viewing angles, without the need to destructively cut away any parts of a plant. The ability to accurately predict phenotyping features, such as the number of leaves, plant height, leaf size and internode distances, is also demonstrated. Experimental results show that, for plants having a range of leaf sizes and a distance between leaves appropriate for the hardware design, the algorithms successfully predict phenotyping features in the target crops, with a recall of 0.97 and a precision of 0.89 for leaf detection and less than a 13-mm error for plant size, leaf size and internode distance

Using Acoustic Holography for Vibration Analysis

Disertační práce se zabývá bezkontaktní analýzou vibrací pomocí metod akustické holografie v blízkém poli. Akustická holografie v blízkém poli je experimentální metoda, která rekonstruuje akustické pole v těsné blízkosti povrchu vibrujícího předmětu na základě měření akustického tlaku nebo akustické rychlosti v určité vzdálenosti od zkoumaného předmětu. Konkrétní realizace této metody závisí na použitém výpočetním algoritmu. Vlastní práce je zaměřena zejména na rozbor algoritmů, které využívají k rekonstrukci zvukového pole v blízkosti vibrujícího objektu transformaci do domény vlnových čísel (prostorová transformace), kde probíhá vlastní výpočet. V úvodu práce je vysvětlena základní teorie metody akustické holografie v blízkém poli s popisem základních vlastností a dále rozborem konkrétních nejčastěji používaných algoritmům pro lokalizaci a charakterizaci zdroje zvuku a pro následnou vibrační analýzu. Stěžejní část práce se věnuje pokročilým metodám zpracování, které se snaží určitým způsobem optimalizovat přesnost predice zvukového pole v blízkosti vibrujícího předmětu v reálných podmínkách. Jde zejména o problematiku použitého měřicího systému s akustickými snímači, které nejsou ideální, a dále o možnost měření v prostorách s difúzním charakterem zvukového pole. Pro tento případ byla na základě literárního průzkumu optimalizována a ověřena metoda využívající dvouvrstvé mikrofonní pole, které umožňuje oddělení zvukových polí přicházejících z různých stran a tedy úspěšné měření v uzavřených prostorách např. kabin automobilů a letadel. Součástí práce byla také optimalizace, rozšíření a následné ověření algoritmů publikovaných v posledních letech pro měření v reálných podmínkách za použití běžně dostupných akustických snímačů.The main aim of the thesis is application of near-field acoustic holography for non-contact vibration analysis. Near-field acoustic holography is an experimental technique for reconstruction of sound field close to the surface of the vibrating object based on measurement of sound pressure or acoustic particle velocity in certain distance from the examined object. Practical realization of this method depends on used calculation procedure. The thesis is focused on analysis of acoustic holography algorithms with transformation into wavenumber domain (spatial transformation) where the reconstruction of the sound field near vibrating object is calculated. The introductory part of the thesis describes the theory of near-field acoustic holography with general characteristics and with analysis of most common algorithms used for localization and characterization of sound source and consequent vibration analysis. Principal part of the thesis deals with advanced processing methods where these methods try to optimize the accuracy of prediction of sound field near vibrating object in real environment. In this study, real measurement conditions represent the measurement system with non-ideal acoustic sensors and also areas with reverberant sound field. Based on literature study, there has been optimized and verified the new method which uses double layer microphone array to separate incoming and outgoing sound field, thus allows successful measurement in confined space e.g. cabins of cars and airplanes. Part of the thesis has been also focused on optimization, extension and successive experimental validation of selected classical algorithms published in last decade for possible measurement in real conditions and with common acoustic sensors.

Progress in industrial photogrammetry by means of markerless solutions

174 p.La siguiente tesis está enfocada al desarrollo y uso avanzado de metodologías fotogramétrica sin dianas en aplicaciones industriales. La fotogrametría es una técnica de medición óptica 3D que engloba múltiples configuraciones y aproximaciones. En este estudio se han desarrollado procedimientos de medición, modelos y estrategias de procesamiento de imagen que van más allá que la fotogrametría convencional y buscan el emplear soluciones de otros campos de la visión artificial en aplicaciones industriales. Mientras que la fotogrametría industrial requiere emplear dianas artificiales para definir los puntos o elementos de interés, esta tesis contempla la reducción e incluso la eliminación de las dianas tanto pasivas como activas como alternativas prácticas. La mayoría de los sistemas de medida utilizan las dianas tanto para definir los puntos de control, relacionar las distintas perspectivas, obtener precisión, así como para automatizar las medidas. Aunque en muchas situaciones el empleo de dianas no sea restrictivo existen aplicaciones industriales donde su empleo condiciona y restringe considerablemente los procedimientos de medida empleados en la inspección. Un claro ejemplo es la verificación y control de calidad de piezas seriadas, o la medición y seguimiento de elementos prismáticos relacionados con un sistema de referencia determinado. Es en este punto donde la fotogrametría sin dianas puede combinarse o complementarse con soluciones tradicionales para tratar de mejorar las prestaciones actuales

Towards Intelligent Telerobotics: Visualization and Control of Remote Robot

Human-machine cooperative or co-robotics has been recognized as the next generation of robotics. In contrast to current systems that use limited-reasoning strategies or address problems in narrow contexts, new co-robot systems will be characterized by their flexibility, resourcefulness, varied modeling or reasoning approaches, and use of real-world data in real time, demonstrating a level of intelligence and adaptability seen in humans and animals. The research I focused is in the two sub-field of co-robotics: teleoperation and telepresence. We firstly explore the ways of teleoperation using mixed reality techniques. I proposed a new type of display: hybrid-reality display (HRD) system, which utilizes commodity projection device to project captured video frame onto 3D replica of the actual target surface. It provides a direct alignment between the frame of reference for the human subject and that of the displayed image. The advantage of this approach lies in the fact that no wearing device needed for the users, providing minimal intrusiveness and accommodating users eyes during focusing. The field-of-view is also significantly increased. From a user-centered design standpoint, the HRD is motivated by teleoperation accidents, incidents, and user research in military reconnaissance etc. Teleoperation in these environments is compromised by the Keyhole Effect, which results from the limited field of view of reference. The technique contribution of the proposed HRD system is the multi-system calibration which mainly involves motion sensor, projector, cameras and robotic arm. Due to the purpose of the system, the accuracy of calibration should also be restricted within millimeter level. The followed up research of HRD is focused on high accuracy 3D reconstruction of the replica via commodity devices for better alignment of video frame. Conventional 3D scanner lacks either depth resolution or be very expensive. We proposed a structured light scanning based 3D sensing system with accuracy within 1 millimeter while robust to global illumination and surface reflection. Extensive user study prove the performance of our proposed algorithm. In order to compensate the unsynchronization between the local station and remote station due to latency introduced during data sensing and communication, 1-step-ahead predictive control algorithm is presented. The latency between human control and robot movement can be formulated as a linear equation group with a smooth coefficient ranging from 0 to 1. This predictive control algorithm can be further formulated by optimizing a cost function. We then explore the aspect of telepresence. Many hardware designs have been developed to allow a camera to be placed optically directly behind the screen. The purpose of such setups is to enable two-way video teleconferencing that maintains eye-contact. However, the image from the see-through camera usually exhibits a number of imaging artifacts such as low signal to noise ratio, incorrect color balance, and lost of details. Thus we develop a novel image enhancement framework that utilizes an auxiliary color+depth camera that is mounted on the side of the screen. By fusing the information from both cameras, we are able to significantly improve the quality of the see-through image. Experimental results have demonstrated that our fusion method compares favorably against traditional image enhancement/warping methods that uses only a single image