Vision-based retargeting for endoscopic navigation

Endoscopy is a standard procedure for visualising the human gastrointestinal tract. With the advances in biophotonics, imaging techniques such as narrow band imaging, confocal laser endomicroscopy, and optical coherence tomography can be combined with normal endoscopy for assisting the early diagnosis of diseases, such as cancer. In the past decade, optical biopsy has emerged to be an effective tool for tissue analysis, allowing in vivo and in situ assessment of pathological sites with real-time feature-enhanced microscopic images. However, the non-invasive nature of optical biopsy leads to an intra-examination retargeting problem, which is associated with the difficulty of re-localising a biopsied site consistently throughout the whole examination. In addition to intra-examination retargeting, retargeting of a pathological site is even more challenging across examinations, due to tissue deformation and changing tissue morphologies and appearances. The purpose of this thesis is to address both the intra- and inter-examination retargeting problems associated with optical biopsy. We propose a novel vision-based framework for intra-examination retargeting. The proposed framework is based on combining visual tracking and detection with online learning of the appearance of the biopsied site. Furthermore, a novel cascaded detection approach based on random forests and structured support vector machines is developed to achieve efficient retargeting. To cater for reliable inter-examination retargeting, the solution provided in this thesis is achieved by solving an image retrieval problem, for which an online scene association approach is proposed to summarise an endoscopic video collected in the first examination into distinctive scenes. A hashing-based approach is then used to learn the intrinsic representations of these scenes, such that retargeting can be achieved in subsequent examinations by retrieving the relevant images using the learnt representations. For performance evaluation of the proposed frameworks, extensive phantom, ex vivo and in vivo experiments have been conducted, with results demonstrating the robustness and potential clinical values of the methods proposed.Open Acces

Online tracking and retargeting with applications to optical biopsy in gastrointestinal endoscopic examinations

With recent advances in biophotonics, techniques such as narrow band imaging, confocal laser endomicroscopy, fluorescence spectroscopy, and optical coherence tomography, can be combined with normal white-light endoscopes to provide in vivo microscopic tissue characterisation, potentially avoiding the need for offline histological analysis. Despite the advantages of these techniques to provide online optical biopsy in situ, it is challenging for gastroenterologists to retarget the optical biopsy sites during endoscopic examinations. This is because optical biopsy does not leave any mark on the tissue. Furthermore, typical endoscopic cameras only have a limited field-of-view and the biopsy sites often enter or exit the camera view as the endoscope moves. In this paper, a framework for online tracking and retargeting is proposed based on the concept of tracking-by-detection. An online detection cascade is proposed where a random binary descriptor using Haar-like features is included as a random forest classifier. For robust retargeting, we have also proposed a RANSAC-based location verification component that incorporates shape context. The proposed detection cascade can be readily integrated with other temporal trackers. Detailed performance evaluation on in vivo gastrointestinal video sequences demonstrates the performance advantage of the proposed method over the current state-of-the-art

Planification de l’ablation radiofréquence des arythmies cardiaques en combinant modélisation et apprentissage automatique

Cardiac arrhythmias are heart rhythm disruptions which can lead to sudden cardiac death. They require a deeper understanding for appropriate treatment planning. In this thesis, we integrate personalized structural and functional data into a 3D tetrahedral mesh of the biventricular myocardium. Next, the Mitchell-Schaeffer (MS) simplified biophysical model is used to study the spatial heterogeneity of electrophysiological (EP) tissue properties and their role in arrhythmogenesis. Radiofrequency ablation (RFA) with the elimination of local abnormal ventricular activities (LAVA) has recently arisen as a potentially curative treatment for ventricular tachycardia but the EP studies required to locate LAVA are lengthy and invasive. LAVA are commonly found within the heterogeneous scar, which can be imaged non-invasively with 3D delayed enhanced magnetic resonance imaging (DE-MRI). We evaluate the use of advanced image features in a random forest machine learning framework to identify areas of LAVA-inducing tissue. Furthermore, we detail the dataset’s inherent error sources and their formal integration in the training process. Finally, we construct MRI-based structural patient-specific heart models and couple them with the MS model. We model a recording catheter using a dipole approach and generate distinct normal and LAVA-like electrograms at locations where they have been found in clinics. This enriches our predictions of the locations of LAVA-inducing tissue obtained through image-based learning. Confidence maps can be generated and analyzed prior to RFA to guide the intervention. These contributions have led to promising results and proofs of concepts.Les arythmies sont des perturbations du rythme cardiaque qui peuvent entrainer la mort subite et requièrent une meilleure compréhension pour planifier leur traitement. Dans cette thèse, nous intégrons des données structurelles et fonctionnelles à un maillage 3D tétraédrique biventriculaire. Le modèle biophysique simplifié de Mitchell-Schaeffer (MS) est utilisé pour étudier l’hétérogénéité des propriétés électrophysiologiques (EP) du tissu et leur rôle sur l’arythmogénèse. L’ablation par radiofréquence (ARF) en éliminant les activités ventriculaires anormales locales (LAVA) est un traitement potentiellement curatif pour la tachycardie ventriculaire, mais les études EP requises pour localiser les LAVA sont longues et invasives. Les LAVA se trouvent autour de cicatrices hétérogènes qui peuvent être imagées de façon non-invasive par IRM à rehaussement tardif. Nous utilisons des caractéristiques d’image dans un contexte d’apprentissage automatique avec des forêts aléatoires pour identifier des aires de tissu qui induisent des LAVA. Nous détaillons les sources d’erreur inhérentes aux données et leur intégration dans le processus d’apprentissage. Finalement, nous couplons le modèle MS avec des géométries du coeur spécifiques aux patients et nous modélisons le cathéter avec une approche par un dipôle pour générer des électrogrammes normaux et des LAVA aux endroits où ils ont été localisés en clinique. Cela améliore la prédiction de localisation du tissu induisant des LAVA obtenue par apprentissage sur l’image. Des cartes de confiance sont générées et peuvent être utilisées avant une ARF pour guider l’intervention. Les contributions de cette thèse ont conduit à des résultats et des preuves de concepts prometteurs

Molecular Mechanisms Underlying Osteocyte Apoptosis and the Associated Osteoclastogenesis in CX43-Deficiency and Aging

Indiana University-Purdue University Indianapolis (IUPUI)Old age is associated with increased bone fragility and risk of fracture as a result of skeletal alterations, including low bone density and cortical thinning. Further, apoptotic osteocytes accumulate in old mice and humans. We have previously shown that mice lacking osteocytic connexin (Cx) 43 (Cx43ΔOt) exhibit a phenotype similar to that of the aging skeleton, with elevated osteocyte apoptosis and an associated increase in osteoclastogenesis. These findings suggest that osteocyte apoptosis results in the release of factors that recruit osteoclasts to bone surfaces close to areas that contain apoptotic osteocytes. However, the specific chemotactic signals, the events mediating their release, and the mechanisms of their action remain unknown. Consistent with this notion, we also found that HMGB1 released by Cx43-deficient (Cx43def) MLO-Y4 osteocytic cells enhances osteoclastogenesis in part by increasing osteocytic RANKL, which promotes osteoclastogenesis, and, at the same time, directly stimulating osteoclastogenesis. Further, expression of the pro-survival microRNA (miR), miR21, is low in Cx43def cells and bones from old female mice, and low miR21 levels increase osteocyte apoptosis. However, surprisingly, mice lacking miR21 (miR21ΔOt) have decreased osteoclast number and activity even under conditions of elevated osteocyte apoptosis; suggesting that osteocytic miR21 may mediate osteoclast precursor recruitment/survival induced by apoptotic osteocytes. However, whether HMGB1/miR21 are released by osteocytes, and if the HMGB1 receptors, receptor for advanced glycation end products (RAGE) and/or tolllike receptor (TLR4) are involved in osteoclast recruitment in Cx43ΔOt and old mice is unknown. The overall objectives of this series of studies were to elucidate the mechanism

A Framework for the Semantics-aware Modelling of Objects

The evolution of 3D visual content calls for innovative methods for modelling shapes based on their intended usage, function and role in a complex scenario. Even if different attempts have been done in this direction, shape modelling still mainly focuses on geometry. However, 3D models have a structure, given by the arrangement of salient parts, and shape and structure are deeply related to semantics and functionality. Changing geometry without semantic clues may invalidate such functionalities or the meaning of objects or their parts. We approach the problem by considering semantics as the formalised knowledge related to a category of objects; the geometry can vary provided that the semantics is preserved. We represent the semantics and the variable geometry of a class of shapes through the parametric template: an annotated 3D model whose geometry can be deformed provided that some semantic constraints remain satisfied. In this work, we design and develop a framework for the semantics-aware modelling of shapes, offering the user a single application environment where the whole workflow of defining the parametric template and applying semantics-aware deformations can take place. In particular, the system provides tools for the selection and annotation of geometry based on a formalised contextual knowledge; shape analysis methods to derive new knowledge implicitly encoded in the geometry, and possibly enrich the given semantics; a set of constraints that the user can apply to salient parts and a deformation operation that takes into account the semantic constraints and provides an optimal solution. The framework is modular so that new tools can be continuously added. While producing some innovative results in specific areas, the goal of this work is the development of a comprehensive framework combining state of the art techniques and new algorithms, thus enabling the user to conceptualise her/his knowledge and model geometric shapes. The original contributions regard the formalisation of the concept of annotation, with attached properties, and of the relations between significant parts of objects; a new technique for guaranteeing the persistence of annotations after significant changes in shape's resolution; the exploitation of shape descriptors for the extraction of quantitative information and the assessment of shape variability within a class; and the extension of the popular cage-based deformation techniques to include constraints on the allowed displacement of vertices. In this thesis, we report the design and development of the framework as well as results in two application scenarios, namely product design and archaeological reconstruction

Psr1p interacts with SUN/sad1p and EB1/mal3p to establish the bipolar spindle

Regular Abstracts - Sunday Poster Presentations: no. 382During mitosis, interpolar microtubules from two spindle pole bodies (SPBs) interdigitate to create an antiparallel microtubule array for accommodating numerous regulatory proteins. Among these proteins, the kinesin-5 cut7p/Eg5 is the key player responsible for sliding apart antiparallel microtubules and thus helps in establishing the bipolar spindle. At the onset of mitosis, two SPBs are adjacent to one another with most microtubules running nearly parallel toward the nuclear envelope, creating an unfavorable microtubule configuration for the kinesin-5 kinesins. Therefore, how the cell organizes the antiparallel microtubule array in the first place at mitotic onset remains enigmatic. Here, we show that a novel protein psrp1p localizes to the SPB and plays a key role in organizing the antiparallel microtubule array. The absence of psr1+ leads to a transient monopolar spindle and massive chromosome loss. Further functional characterization demonstrates that psr1p is recruited to the SPB through interaction with the conserved SUN protein sad1p and that psr1p physically interacts with the conserved microtubule plus tip protein mal3p/EB1. These results suggest a model that psr1p serves as a linking protein between sad1p/SUN and mal3p/EB1 to allow microtubule plus ends to be coupled to the SPBs for organization of an antiparallel microtubule array. Thus, we conclude that psr1p is involved in organizing the antiparallel microtubule array in the first place at mitosis onset by interaction with SUN/sad1p and EB1/mal3p, thereby establishing the bipolar spindle.postprin

Removal of antagonistic spindle forces can rescue metaphase spindle length and reduce chromosome segregation defects

Regular Abstracts - Tuesday Poster Presentations: no. 1925Metaphase describes a phase of mitosis where chromosomes are attached and oriented on the bipolar spindle for subsequent segregation at anaphase. In diverse cell types, the metaphase spindle is maintained at a relatively constant length. Metaphase spindle length is proposed to be regulated by a balance of pushing and pulling forces generated by distinct sets of spindle microtubules and their interactions with motors and microtubule-associated proteins (MAPs). Spindle length appears important for chromosome segregation fidelity, as cells with shorter or longer than normal metaphase spindles, generated through deletion or inhibition of individual mitotic motors or MAPs, showed chromosome segregation defects. To test the force balance model of spindle length control and its effect on chromosome segregation, we applied fast microfluidic temperature-control with live-cell imaging to monitor the effect of switching off different combinations of antagonistic forces in the fission yeast metaphase spindle. We show that spindle midzone proteins kinesin-5 cut7p and microtubule bundler ase1p contribute to outward pushing forces, and spindle kinetochore proteins kinesin-8 klp5/6p and dam1p contribute to inward pulling forces. Removing these proteins individually led to aberrant metaphase spindle length and chromosome segregation defects. Removing these proteins in antagonistic combination rescued the defective spindle length and, in some combinations, also partially rescued chromosome segregation defects. Our results stress the importance of proper chromosome-to-microtubule attachment over spindle length regulation for proper chromosome segregation.postprin