Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Computerized Analysis of Magnetic Resonance Images to Study Cerebral Anatomy in Developing Neonates

The study of cerebral anatomy in developing neonates is of great importance for the understanding of brain development during the early period of life. This dissertation therefore focuses on three challenges in the modelling of cerebral anatomy in neonates during brain development. The methods that have been developed all use Magnetic Resonance Images (MRI) as source data. To facilitate study of vascular development in the neonatal period, a set of image analysis algorithms are developed to automatically extract and model cerebral vessel trees. The whole process consists of cerebral vessel tracking from automatically placed seed points, vessel tree generation, and vasculature registration and matching. These algorithms have been tested on clinical Time-of- Flight (TOF) MR angiographic datasets. To facilitate study of the neonatal cortex a complete cerebral cortex segmentation and reconstruction pipeline has been developed. Segmentation of the neonatal cortex is not effectively done by existing algorithms designed for the adult brain because the contrast between grey and white matter is reversed. This causes pixels containing tissue mixtures to be incorrectly labelled by conventional methods. The neonatal cortical segmentation method that has been developed is based on a novel expectation-maximization (EM) method with explicit correction for mislabelled partial volume voxels. Based on the resulting cortical segmentation, an implicit surface evolution technique is adopted for the reconstruction of the cortex in neonates. The performance of the method is investigated by performing a detailed landmark study. To facilitate study of cortical development, a cortical surface registration algorithm for aligning the cortical surface is developed. The method first inflates extracted cortical surfaces and then performs a non-rigid surface registration using free-form deformations (FFDs) to remove residual alignment. Validation experiments using data labelled by an expert observer demonstrate that the method can capture local changes and follow the growth of specific sulcus

Probabilistic three-dimensional object tracking based on adaptive depth segmentation

Object tracking is one of the fundamental topics of computer vision with diverse applications. The arising challenges in tracking, i.e., cluttered scenes, occlusion, complex motion, and illumination variations have motivated utilization of depth information from 3D sensors. However, current 3D trackers are not applicable to unconstrained environments without a priori knowledge. As an important object detection module in tracking, segmentation subdivides an image into its constituent regions. Nevertheless, the existing range segmentation methods in literature are difficult to implement in real-time due to their slow performance. In this thesis, a 3D object tracking method based on adaptive depth segmentation and particle filtering is presented. In this approach, the segmentation method as the bottom-up process is combined with the particle filter as the top-down process to achieve efficient tracking results under challenging circumstances. The experimental results demonstrate the efficiency, as well as robustness of the tracking algorithm utilizing real-world range information

Correction of Errors in Time of Flight Cameras

En esta tesis se aborda la corrección de errores en cámaras de profundidad basadas en tiempo de vuelo (Time of Flight - ToF). De entre las más recientes tecnologías, las cámaras ToF de modulación continua (Continuous Wave Modulation - CWM) son una alternativa prometedora para la creación de sensores compactos y rápidos. Sin embargo, existen gran variedad de errores que afectan notablemente la medida de profundidad, poniendo en compromiso posibles aplicaciones. La corrección de dichos errores propone un reto desafiante. Actualmente, se consideran dos fuentes principales de error: i) sistemático y ii) no sistemático. Mientras que el primero admite calibración, el último depende de la geometría y el movimiento relativo de la escena. Esta tesis propone métodos que abordan i) la distorsión sistemática de profundidad y dos de las fuentes de error no sistemático más relevantes: ii.a) la interferencia por multicamino (Multipath Interference - MpI) y ii.b) los artefactos de movimiento. La distorsión sistemática de profundidad en cámaras ToF surge principalmente debido al uso de señales sinusoidales no perfectas para modular. Como resultado, las medidas de profundidad aparecen distorsionadas, pudiendo ser reducidas con una etapa de calibración. Esta tesis propone un método de calibración basado en mostrar a la cámara un plano en diferentes posiciones y orientaciones. Este método no requiere de patrones de calibración y, por tanto, puede emplear los planos, que de manera natural, aparecen en la escena. El método propuesto encuentra una función que obtiene la corrección de profundidad correspondiente a cada píxel. Esta tesis mejora los métodos existentes en cuanto a precisión, eficiencia e idoneidad. La interferencia por multicamino surge debido a la superposición de la señal reflejada por diferentes caminos con la reflexión directa, produciendo distorsiones que se hacen más notables en superficies convexas. La MpI es la causa de importantes errores en la estimación de profundidad en cámaras CWM ToF. Esta tesis propone un método que elimina la MpI a partir de un solo mapa de profundidad. El enfoque propuesto no requiere más información acerca de la escena que las medidas ToF. El método se fundamenta en un modelo radio-métrico de las medidas que se emplea para estimar de manera muy precisa el mapa de profundidad sin distorsión. Una de las tecnologías líderes para la obtención de profundidad en imagen ToF está basada en Photonic Mixer Device (PMD), la cual obtiene la profundidad mediante el muestreado secuencial de la correlación entre la señal de modulación y la señal proveniente de la escena en diferentes desplazamientos de fase. Con movimiento, los píxeles PMD capturan profundidades diferentes en cada etapa de muestreo, produciendo artefactos de movimiento. El método propuesto en esta tesis para la corrección de dichos artefactos destaca por su velocidad y sencillez, pudiendo ser incluido fácilmente en el hardware de la cámara. La profundidad de cada píxel se recupera gracias a la consistencia entre las muestras de correlación en el píxel PMD y de la vecindad local. Este método obtiene correcciones precisas, reduciendo los artefactos de movimiento enormemente. Además, como resultado de este método, puede obtenerse el flujo óptico en los contornos en movimiento a partir de una sola captura. A pesar de ser una alternativa muy prometedora para la obtención de profundidad, las cámaras ToF todavía tienen que resolver problemas desafiantes en relación a la corrección de errores sistemáticos y no sistemáticos. Esta tesis propone métodos eficaces para enfrentarse con estos errores

Correction of Errors in Time of Flight Cameras

En esta tesis se aborda la corrección de errores en cámaras de profundidad basadas en tiempo de vuelo (Time of Flight - ToF). De entre las más recientes tecnologías, las cámaras ToF de modulación continua (Continuous Wave Modulation - CWM) son una alternativa prometedora para la creación de sensores compactos y rápidos. Sin embargo, existen gran variedad de errores que afectan notablemente la medida de profundidad, poniendo en compromiso posibles aplicaciones. La corrección de dichos errores propone un reto desafiante. Actualmente, se consideran dos fuentes principales de error: i) sistemático y ii) no sistemático. Mientras que el primero admite calibración, el último depende de la geometría y el movimiento relativo de la escena. Esta tesis propone métodos que abordan i) la distorsión sistemática de profundidad y dos de las fuentes de error no sistemático más relevantes: ii.a) la interferencia por multicamino (Multipath Interference - MpI) y ii.b) los artefactos de movimiento. La distorsión sistemática de profundidad en cámaras ToF surge principalmente debido al uso de señales sinusoidales no perfectas para modular. Como resultado, las medidas de profundidad aparecen distorsionadas, pudiendo ser reducidas con una etapa de calibración. Esta tesis propone un método de calibración basado en mostrar a la cámara un plano en diferentes posiciones y orientaciones. Este método no requiere de patrones de calibración y, por tanto, puede emplear los planos, que de manera natural, aparecen en la escena. El método propuesto encuentra una función que obtiene la corrección de profundidad correspondiente a cada píxel. Esta tesis mejora los métodos existentes en cuanto a precisión, eficiencia e idoneidad. La interferencia por multicamino surge debido a la superposición de la señal reflejada por diferentes caminos con la reflexión directa, produciendo distorsiones que se hacen más notables en superficies convexas. La MpI es la causa de importantes errores en la estimación de profundidad en cámaras CWM ToF. Esta tesis propone un método que elimina la MpI a partir de un solo mapa de profundidad. El enfoque propuesto no requiere más información acerca de la escena que las medidas ToF. El método se fundamenta en un modelo radio-métrico de las medidas que se emplea para estimar de manera muy precisa el mapa de profundidad sin distorsión. Una de las tecnologías líderes para la obtención de profundidad en imagen ToF está basada en Photonic Mixer Device (PMD), la cual obtiene la profundidad mediante el muestreado secuencial de la correlación entre la señal de modulación y la señal proveniente de la escena en diferentes desplazamientos de fase. Con movimiento, los píxeles PMD capturan profundidades diferentes en cada etapa de muestreo, produciendo artefactos de movimiento. El método propuesto en esta tesis para la corrección de dichos artefactos destaca por su velocidad y sencillez, pudiendo ser incluido fácilmente en el hardware de la cámara. La profundidad de cada píxel se recupera gracias a la consistencia entre las muestras de correlación en el píxel PMD y de la vecindad local. Este método obtiene correcciones precisas, reduciendo los artefactos de movimiento enormemente. Además, como resultado de este método, puede obtenerse el flujo óptico en los contornos en movimiento a partir de una sola captura. A pesar de ser una alternativa muy prometedora para la obtención de profundidad, las cámaras ToF todavía tienen que resolver problemas desafiantes en relación a la corrección de errores sistemáticos y no sistemáticos. Esta tesis propone métodos eficaces para enfrentarse con estos errores

Probabilistic RGB-D Odometry based on Points, Lines and Planes Under Depth Uncertainty

This work proposes a robust visual odometry method for structured environments that combines point features with line and plane segments, extracted through an RGB-D camera. Noisy depth maps are processed by a probabilistic depth fusion framework based on Mixtures of Gaussians to denoise and derive the depth uncertainty, which is then propagated throughout the visual odometry pipeline. Probabilistic 3D plane and line fitting solutions are used to model the uncertainties of the feature parameters and pose is estimated by combining the three types of primitives based on their uncertainties. Performance evaluation on RGB-D sequences collected in this work and two public RGB-D datasets: TUM and ICL-NUIM show the benefit of using the proposed depth fusion framework and combining the three feature-types, particularly in scenes with low-textured surfaces, dynamic objects and missing depth measurements.Comment: Major update: more results, depth filter released as opensource, 34 page

E. coli immunosensori arendus ja rakendamine

Väitekirja elektrooniline versioon ei sisalda publikatsiooneIga aastaga muutuvad keskkonna kvaliteet ja puhtus maailmas aina olulisemaks. Üks tähtsamaid küsimusi on inimeste ligipääs puhtale veele. Kui tavaliselt mõeldakse selle all eelkõige joogivett, siis sama oluline on ka suplusvee puhtus ja ohutus. Üheks vee kvaliteedi parameetriks on tema mikrobioloogiline ohutus, mille hindamiseks kasutatakse teatud bakteriliike, ehk indikaatorliike, mille olemasolu ja arvukuse järgi hinnatakse vee kvaliteeti. Üheks levinumaks indikaatorliigiks vees on Escherichia coli ehk soolekepike. Mõned E. coli tüved võivad olla ka patogeensed. Tavaliselt hinnatakse E. coli arvukust mikrobioloogilistel meetoditel, kultiveerides proove spetsiaalsetel söötmetel, kuid see on aeganõudev. Molekulaarsed meetodid (PCR) on küll kiiremad, kuid nõuavad keeruka aparatuuri kasutamist ning on tundlikud võimaliku saastuse ja proovide maatriksist tuleneva inhibitsiooni suhtes. Doktoritöö eesmärgiks oli välja töötada immunobiosensorsüsteem E. coli tuvastamiseks ning testida selle biosensori rakendamise võimalusi looduslike veeproovide ja kliiniliste uriiniproovide analüüsil. Kasutatud biosensori bioloogiline äratundmiskomponendina kasutati fluorestsentsmärgisega konjugeeritud anti - E. coli ankeha. Analüüsi kõrge tundlikkus saavutati tänu proovis leiduva E. coli sidumisele ühekordse kasutusega mikrokolonnile ning seondunud bakterite spetsiifilisele detekteerimisele. Erinevatest allikatest pärinevates proovides saadud analüüsitulemusi võrreldi alternatiivsete E. coli määramismeetodite, mikrobioloogilise külvi ja kvantitatiivse PCR abil saadud tulemustega. Nimetatud metoodikad võimaldavad küll kõik hinnata E. coli arvukust, kuid mõõdavad erinevaid rakku iseloomustavaid suurusi. Mikrobioloogiliste külvide meetod võtab arvesse elusaid kultiveeritavid rakke; kvantitatiivne PCR hindab E. coli genoomse DNA kogust (elusad + mitte-kultiveeritavad ja surnud rakud), ning biosensor mõõdab E. coli mebraanivalkude kontsentratsiooni proovis. Mõõtes näiteks ühte ja sama veeproovi kirjeldatud meetoditega selgus, et oodatult kõige madalama tulemuse andis mikrobioloogiline meetod (40 korda madalam, kui biosensor), ning ka qPCR meetod andis keskmiselt 4 korda madalama tulemuse kui biosensor. Töö selgitati välja põhjused, mis selliseid erinevusi põhjustasid. Esiteks, biosensoris põhjustasid mõõdetava signaali ka rakkude mehhaanilisel ja keemilisel töötlemisel saadud rakumembraanide fragmendid. Teise olulise tulemusena selgus, et biosensoris kasutatava antikeha äratundmisreaktsioon oli komplekses mikrobioloogilises keskkonnas eeldatust vähem selektiivne. Lisaks E. coli´le on looduslikes keskkondades palju sarnaseid bakteriliike (kolivormseid), millest mõnedel on potentsiaalselt afiinsus immunosensoris kasutatud E. coli antikeha suhtes. Kuna selliste bakterite üldhulk looduslikes vetes võib olla kõrge, siis tuleb biosensori mõõtetulemuste interpreteerimisel arvestada ka nende poolt genereeritava signaaliga. Arvestades erinevate rakufragmentide ning kolivormsete rakkude poolt põhjustatud signaali osakaalu, siis elusate kultiveeritavate E. coli rakkude poolt tingitud signaali osakaal on immunosensori kogusignaalist 10%. E. coli immunosensorit kasutati ka uropatogeense E. coli tuvastamiseks ja kvantiteerimiseks kliinilistes uriiniproovides, kus biosensoriga saadud analüüsitulemused langesid kokku mikrobioloogiliste ja molekulaarsete (qPCR) meetoditega saadud tulemustega. Väljatöötatud biosensorsüsteem võimaldas määrata E. coli sisalduse vee- või uriiniproovides vahemikus 7-107 rakku milliliitris 20 minuti jooksul, mis loob eelduse E. coli automaatseks kohapealseks määramiseks, vältides vajadust proovide transpordiks laborisse ning analüüsile eelnevaks töötluseks.The quality of water is among the major global problems usually associated with drinking water. However, problems with the physical, chemical, and biological pollution of bathing water are increasing. The biological pollution is commonly assessed using microbiology methods by identifying and quantifying microbial indicator organisms. The most common indicator species for water analysis is Escherichia coli – gram-negative, rod-shaped bacteria generally found in the guts of warm-blooded animals. Most E. coli strains are harmless, but there is also a group of E. coli strains, which are human pathogens Uropathogenic E. coli (UPEC) is the main human urinary tract pathogen. The most common method for E. coli enumeration is still microbiological cultivation. This method is reliable and simple, but the analysis time is long, the sensitivity is quite poor and the cultivation requires special lab conditions. In addition, E. coli can be detected with qPCR. A good alternative for E. coli indication and enumeration are biosensor-based systems, which can provide short analysis time, high specificity, and sensitivity. Biosensors also offer options for automation and on-site analysis required to meet modern requirements for data collection. The objective of this thesis was the design and production of an E. coli-specific immunosensor, its testing for potential applications in environmental monitoring and clinical laboratory analysis, and validation of the biosensor results. The proposed E. coli immunosensor integrates the use of polyclonal E. coli antibodies for bio-recognition and single-use microcolumn analysis system for the rapid detection of E. coli from bathing water and urine samples. The immunosensor the detection limit was below 10 cells/ml, and the working range was between 10…108 cells/ml. In urine, there was no inference other bacterial species present in urine to the biosensor signal, as there is a small probability of the presence of dead and/or fragmented E. coli cells in urine. The E. coli biosensor results were in the same range as those obtained with qPCR and cultivation methods. The analysis of the biosensor signal in bathing water samples revealed that the signal was strongly affected by dead cells, cell fragments, and different coliforms, which are abundant in natural waters. The proportion of cultivable E. coli cells in the immunosensor entire signal was only about 10%. The signal of non-cultivable E. coli cells (measured by qPCR) formed 30% of the immunosensor signal and the majority of the measured signal, 60%, was most likely generated by different forms of coliform bacteria and E. coli cell fragments. Using renewable, single-use E. coli immunosensor is an excellent alternative to time-consuming microbiological and molecular methods for analyzing complex natural samples. These immunosensors can significantly shorten the time required to determine and quantify E. coli. It could be used for automated analyses, as quick identification of E. coli allows to take timely measures to minimize potential health risks.https://www.ester.ee/record=b550784

Neuromorphic object localization using resistive memories and ultrasonic transducers

Real-world sensory-processing applications require compact, low-latency, and low-power computing systems. Enabled by their in-memory event-driven computing abilities, hybrid memristive-Complementary Metal-Oxide Semiconductor neuromorphic architectures provide an ideal hardware substrate for such tasks. To demonstrate the full potential of such systems, we propose and experimentally demonstrate an end-to-end sensory processing solution for a real-world object localization application. Drawing inspiration from the barn owl’s neuroanatomy, we developed a bio-inspired, event-driven object localization system that couples state-of-the-art piezoelectric micromachined ultrasound transducer sensors to a neuromorphic resistive memories-based computational map. We present measurement results from the fabricated system comprising resistive memories-based coincidence detectors, delay line circuits, and a full-custom ultrasound sensor. We use these experimental results to calibrate our system-level simulations. These simulations are then used to estimate the angular resolution and energy efficiency of the object localization model. The results reveal the potential of our approach, evaluated in orders of magnitude greater energy efficiency than a microcontroller performing the same task

Particle Filters for Colour-Based Face Tracking Under Varying Illumination

Automatic human face tracking is the basis of robotic and active vision systems used for facial feature analysis, automatic surveillance, video conferencing, intelligent transportation, human-computer interaction and many other applications. Superior human face tracking will allow future safety surveillance systems which monitor drowsy drivers, or patients and elderly people at the risk of seizure or sudden falls and will perform with lower risk of failure in unexpected situations. This area has actively been researched in the current literature in an attempt to make automatic face trackers more stable in challenging real-world environments. To detect faces in video sequences, features like colour, texture, intensity, shape or motion is used. Among these feature colour has been the most popular, because of its insensitivity to orientation and size changes and fast process-ability. The challenge of colour-based face trackers, however, has been dealing with the instability of trackers in case of colour changes due to the drastic variation in environmental illumination. Probabilistic tracking and the employment of particle filters as powerful Bayesian stochastic estimators, on the other hand, is increasing in the visual tracking field thanks to their ability to handle multi-modal distributions in cluttered scenes. Traditional particle filters utilize transition prior as importance sampling function, but this can result in poor posterior sampling. The objective of this research is to investigate and propose stable face tracker capable of dealing with challenges like rapid and random motion of head, scale changes when people are moving closer or further from the camera, motion of multiple people with close skin tones in the vicinity of the model person, presence of clutter and occlusion of face. The main focus has been on investigating an efficient method to address the sensitivity of the colour-based trackers in case of gradual or drastic illumination variations. The particle filter is used to overcome the instability of face trackers due to nonlinear and random head motions. To increase the traditional particle filter\u27s sampling efficiency an improved version of the particle filter is introduced that considers the latest measurements. This improved particle filter employs a new colour-based bottom-up approach that leads particles to generate an effective proposal distribution. The colour-based bottom-up approach is a classification technique for fast skin colour segmentation. This method is independent to distribution shape and does not require excessive memory storage or exhaustive prior training. Finally, to address the adaptability of the colour-based face tracker to illumination changes, an original likelihood model is proposed based of spatial rank information that considers both the illumination invariant colour ordering of a face\u27s pixels in an image or video frame and the spatial interaction between them. The original contribution of this work lies in the unique mixture of existing and proposed components to improve colour-base recognition and tracking of faces in complex scenes, especially where drastic illumination changes occur. Experimental results of the final version of the proposed face tracker, which combines the methods developed, are provided in the last chapter of this manuscript