Stateless actor-critic for instance segmentation with high-level priors

Instance segmentation is an important computer vision problem which remains challenging despite impressive recent advances due to deep learning-based methods. Given sufficient training data, fully supervised methods can yield excellent performance, but annotation of ground-truth data remains a major bottleneck, especially for biomedical applications where it has to be performed by domain experts. The amount of labels required can be drastically reduced by using rules derived from prior knowledge to guide the segmentation. However, these rules are in general not differentiable and thus cannot be used with existing methods. Here, we relax this requirement by using stateless actor critic reinforcement learning, which enables non-differentiable rewards. We formulate the instance segmentation problem as graph partitioning and the actor critic predicts the edge weights driven by the rewards, which are based on the conformity of segmented instances to high-level priors on object shape, position or size. The experiments on toy and real datasets demonstrate that we can achieve excellent performance without any direct supervision based only on a rich set of priors

Variational methods and its applications to computer vision

Many computer vision applications such as image segmentation can be formulated in a ''variational'' way as energy minimization problems. Unfortunately, the computational task of minimizing these energies is usually difficult as it generally involves non convex functions in a space with thousands of dimensions and often the associated combinatorial problems are NP-hard to solve. Furthermore, they are ill-posed inverse problems and therefore are extremely sensitive to perturbations (e.g. noise). For this reason in order to compute a physically reliable approximation from given noisy data, it is necessary to incorporate into the mathematical model appropriate regularizations that require complex computations. The main aim of this work is to describe variational segmentation methods that are particularly effective for curvilinear structures. Due to their complex geometry, classical regularization techniques cannot be adopted because they lead to the loss of most of low contrasted details. In contrast, the proposed method not only better preserves curvilinear structures, but also reconnects some parts that may have been disconnected by noise. Moreover, it can be easily extensible to graphs and successfully applied to different types of data such as medical imagery (i.e. vessels, hearth coronaries etc), material samples (i.e. concrete) and satellite signals (i.e. streets, rivers etc.). In particular, we will show results and performances about an implementation targeting new generation of High Performance Computing (HPC) architectures where different types of coprocessors cooperate. The involved dataset consists of approximately 200 images of cracks, captured in three different tunnels by a robotic machine designed for the European ROBO-SPECT project.Open Acces

Pinning down loosened prostheses : imaging and planning of percutaneous hip refixation

This thesis examines how computer software can be used to analyse medical images of an aseptically loosening hip prosthesis, and subsequently to plan and guide a minimally invasive cement injection procedure to stabilize the prosthesis. We addressed the detection and measurement of periprosthetic bone lesions from CT image volumes. Post-operative CTs of patients treated at our institution were analysed. We developed tissue classification algorithms that automatically label periprosthetic bone, cement and fibrous interface tissue. An existing particle-based multi-material meshing algorithm was adapted for improved Finite Element model creation. We then presented HipRFX, a proof-of-concept software tool for planning and guidance during percutaneous cement refixation procedures.Advanced School for Computing and Imaging (ASCI), Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO), Stichting Anna Fonds, Technologiestichting STWUBL - phd migration 201

Biological model representation and analysis

In this thesis, we discuss solutions of phenotype description based on the microscopy image analysis to deal with biological problems both in 2D and 3D space. Our description of patterns goes beyond conventional features and helps to visualize the unseen in feature dataset. These solutions share several common processes which are based on similar principles. Furthermore, we notice that advanced features and classier strategies can help us improve the performance of the solutions. The biological problems that we have studied include the endocytosis routing using high-throughput screening in 2D and time and 3D geometrical representation from biological structures.China Scholarship CouncilComputer Systems, Imagery and Medi

Study on the Method of Constructing a Statistical Shape Model and Its Application to the Segmentation of Internal Organs in Medical Images

In image processing, segmentation is one of the critical tasks for diagnostic analysis and image interpretation. In the following thesis, we describe the investigation of three problems related to the segmentation algorithms for medical images: Active shape model algorithm, 3-dimensional (3-D) statistical shape model building and organic segmentation experiments. For the development of Active shape models, the constraints of statistical model reduced this algorithm to be difficult for various biological shapes. To overcome the coupling of parameters in the original algorithm, in this thesis, the genetic algorithm is introduced to relax the shape limitation. How to construct a robust and effective 3-D point model is still a key step in statistical shape models. Generally the shape information is obtained from manually segmented voxel data. In this thesis, a two-step procedure for generating these models was designed. After transformed the voxel data to triangular polygonal data, in the first step, attitudes of these interesting objects are aligned according their surface features. We propose to reflect the surface orientations by means of their Gauss maps. As well the Gauss maps are mapped to a complex plane using stereographic projection approach. The experiment was run to align a set of left lung models. The second step is identifying the positions of landmarks on polygonal surfaces. This is solved by surface parameterization method. We proposed two simplex methods to correspond the landmarks. A semi-automatic method attempts to “copy” the phasic positions of pre-placed landmarks to all the surfaces, which have been mapped to the same parameterization domain. Another automatic corresponding method attempts to place the landmarks equidistantly. Finally, the goodness experiments were performed to measure the difference to manually corresponded results. And we also compared the affection to correspondence when using different surface mapping methods. The third part of this thesis is applying the segmentation algorithms to solve clinical problems. We did not stick to the model-based methods but choose the suitable one or their complex according to the objects. In the experiment of lung regions segmentation which includes pulmonary nodules, we propose a complementary region growing method to deal with the unpredictable variation of image densities of lesion regions. In the experiments of liver regions, instead of using region growing method in 3-D style, we turn into a slice-by-slice style in order to reduce the overflows. The image intensity of cardiac regions is distinguishable from lung regions in CT image. But as to the adjacent zone of heart and liver boundary are generally blurry. We utilized a shape model guided method to refine the segmentation results.3-D segmentation techniques have been applied widely not only in medical imaging fields, but also in machine vision, computer graphic. At the last part of this thesis, we resume some interesting topics such as 3-D visualization for medical interpretation, human face recognition and object grasping robot etc.九州工業大学博士学位論文 学位記番号:工博甲第353号　学位授与年月日:平成25年9月27日Chapter 1: Introduction|Chapter 2: Framework of Medical Image Segmentation|Chapter 3: 2-D Organic Regions Using Active Shape Model and Genetic Algorithm|Chapter 4: Alignment of 3-D Models|Chapter 5: Corespondence of 3-D Models|Chapter 6:Experiments of Organic Segmentation|Chapter 7: Visualization Technology and Its Applications|Chapter 8: Conclusions and Future Works九州工業大学平成25年

Enabling technology for non-rigid registration during image-guided neurosurgery

In the context of image processing, non-rigid registration is an operation that attempts to align two or more images using spatially varying transformations. Non-rigid registration finds application in medical image processing to account for the deformations in the soft tissues of the imaged organs. During image-guided neurosurgery, non-rigid registration has the potential to assist in locating critical brain structures and improve identification of the tumor boundary. Robust non-rigid registration methods combine estimation of tissue displacement based on image intensities with the spatial regularization using biomechanical models of brain deformation. In practice, the use of such registration methods during neurosurgery is complicated by a number of issues: construction of the biomechanical model used in the registration from the image data, high computational demands of the application, and difficulties in assessing the registration results. In this dissertation we develop methods and tools that address some of these challenges, and provide components essential for the intra-operative application of a previously validated physics-based non-rigid registration method.;First, we study the problem of image-to-mesh conversion, which is required for constructing biomechanical model of the brain used during registration. We develop and analyze a number of methods suitable for solving this problem, and evaluate them using application-specific quantitative metrics. Second, we develop a high-performance implementation of the non-rigid registration algorithm and study the use of geographically distributed Grid resources for speculative registration computations. Using the high-performance implementation running on the remote computing resources we are able to deliver the results of registration within the time constraints of the neurosurgery. Finally, we present a method that estimates local alignment error between the two images of the same subject. We assess the utility of this method using multiple sources of ground truth to evaluate its potential to support speculative computations of non-rigid registration

A Modular and Open-Source Framework for Virtual Reality Visualisation and Interaction in Bioimaging

Life science today involves computational analysis of a large amount and variety of data, such as volumetric data acquired by state-of-the-art microscopes, or mesh data from analysis of such data or simulations. The advent of new imaging technologies, such as lightsheet microscopy, has resulted in the users being confronted with an ever-growing amount of data, with even terabytes of imaging data created within a day. With the possibility of gentler and more high-performance imaging, the spatiotemporal complexity of the model systems or processes of interest is increasing as well. Visualisation is often the first step in making sense of this data, and a crucial part of building and debugging analysis pipelines. It is therefore important that visualisations can be quickly prototyped, as well as developed or embedded into full applications. In order to better judge spatiotemporal relationships, immersive hardware, such as Virtual or Augmented Reality (VR/AR) headsets and associated controllers are becoming invaluable tools. In this work we present scenery, a modular and extensible visualisation framework for the Java VM that can handle mesh and large volumetric data, containing multiple views, timepoints, and color channels. scenery is free and open-source software, works on all major platforms, and uses the Vulkan or OpenGL rendering APIs. We introduce scenery's main features, and discuss its use with VR/AR hardware and in distributed rendering. In addition to the visualisation framework, we present a series of case studies, where scenery can provide tangible benefit in developmental and systems biology: With Bionic Tracking, we demonstrate a new technique for tracking cells in 4D volumetric datasets via tracking eye gaze in a virtual reality headset, with the potential to speed up manual tracking tasks by an order of magnitude. We further introduce ideas to move towards virtual reality-based laser ablation and perform a user study in order to gain insight into performance, acceptance and issues when performing ablation tasks with virtual reality hardware in fast developing specimen. To tame the amount of data originating from state-of-the-art volumetric microscopes, we present ideas how to render the highly-efficient Adaptive Particle Representation, and finally, we present sciview, an ImageJ2/Fiji plugin making the features of scenery available to a wider audience.:Abstract Foreword and Acknowledgements Overview and Contributions Part 1 - Introduction 1 Fluorescence Microscopy 2 Introduction to Visual Processing 3 A Short Introduction to Cross Reality 4 Eye Tracking and Gaze-based Interaction Part 2 - VR and AR for System Biology 5 scenery — VR/AR for Systems Biology 6 Rendering 7 Input Handling and Integration of External Hardware 8 Distributed Rendering 9 Miscellaneous Subsystems 10 Future Development Directions Part III - Case Studies C A S E S T U D I E S 11 Bionic Tracking: Using Eye Tracking for Cell Tracking 12 Towards Interactive Virtual Reality Laser Ablation 13 Rendering the Adaptive Particle Representation 14 sciview — Integrating scenery into ImageJ2 & Fiji Part IV - Conclusion 15 Conclusions and Outlook Backmatter & Appendices A Questionnaire for VR Ablation User Study B Full Correlations in VR Ablation Questionnaire C Questionnaire for Bionic Tracking User Study List of Tables List of Figures Bibliography Selbstständigkeitserklärun