3D mesh processing using GAMer 2 to enable reaction-diffusion simulations in realistic cellular geometries

Recent advances in electron microscopy have enabled the imaging of single cells in 3D at nanometer length scale resolutions. An uncharted frontier for in silico biology is the ability to simulate cellular processes using these observed geometries. Enabling such simulations requires watertight meshing of electron micrograph images into 3D volume meshes, which can then form the basis of computer simulations of such processes using numerical techniques such as the Finite Element Method. In this paper, we describe the use of our recently rewritten mesh processing software, GAMer 2, to bridge the gap between poorly conditioned meshes generated from segmented micrographs and boundary marked tetrahedral meshes which are compatible with simulation. We demonstrate the application of a workflow using GAMer 2 to a series of electron micrographs of neuronal dendrite morphology explored at three different length scales and show that the resulting meshes are suitable for finite element simulations. This work is an important step towards making physical simulations of biological processes in realistic geometries routine. Innovations in algorithms to reconstruct and simulate cellular length scale phenomena based on emerging structural data will enable realistic physical models and advance discovery at the interface of geometry and cellular processes. We posit that a new frontier at the intersection of computational technologies and single cell biology is now open.Comment: 39 pages, 14 figures. High resolution figures and supplemental movies available upon reques

RGB-D datasets using microsoft kinect or similar sensors: a survey

RGB-D data has turned out to be a very useful representation of an indoor scene for solving fundamental computer vision problems. It takes the advantages of the color image that provides appearance information of an object and also the depth image that is immune to the variations in color, illumination, rotation angle and scale. With the invention of the low-cost Microsoft Kinect sensor, which was initially used for gaming and later became a popular device for computer vision, high quality RGB-D data can be acquired easily. In recent years, more and more RGB-D image/video datasets dedicated to various applications have become available, which are of great importance to benchmark the state-of-the-art. In this paper, we systematically survey popular RGB-D datasets for different applications including object recognition, scene classification, hand gesture recognition, 3D-simultaneous localization and mapping, and pose estimation. We provide the insights into the characteristics of each important dataset, and compare the popularity and the difficulty of those datasets. Overall, the main goal of this survey is to give a comprehensive description about the available RGB-D datasets and thus to guide researchers in the selection of suitable datasets for evaluating their algorithms

Cellular 3D-reconstruction and analysis in the human cerebral cortex using automatic serial sections

Techniques involving three-dimensional (3D) tissue structure reconstruction and analysis provide a better understanding of changes in molecules and function. We have developed AutoCUTS-LM, an automated system that allows the latest advances in 3D tissue reconstruction and cellular analysis developments using light microscopy on various tissues, including archived tissue. The workflow in this paper involved advanced tissue sampling methods of the human cerebral cortex, an automated serial section collection system, digital tissue library, cell detection using convolution neural network, 3D cell reconstruction, and advanced analysis. Our results demonstrated the detailed structure of pyramidal cells (number, volume, diameter, sphericity and orientation) and their 3D spatial organization are arranged in a columnar structure. The pipeline of these combined techniques provides a detailed analysis of tissues and cells in biology and pathology

Development and application of quantitative image analysis for preclinical MRI research

The aim of this thesis is to develop quantitative analysis methods to validate MRI and improve the detection of tumour infiltration. The major components include a description of the development the quantitative methods to better validate imaging biomarkers and detect of infiltration of tumour cells into normal tissue, which were then applied to a mouse model of glioblastoma invasion. To do this, a new histology model, called Stacked In-plane Histology (SIH), was developed to allow a quantitative analysis of MRI. Validating imaging biomarkers for glioblastoma infiltration Cancer can be defined as a disease in which a group of abnormal cells grow uncontrollably, often with fatal outcomes. According to (Cancer research UK, 2019), there are more than 363,000 new cancer cases in the UK every year, an increase from the 990 cases reported daily in 2014-2016, with only half of all patients recovering. Glioblastoma (GB) is the most frequent and malignant form of primary brain tumours with a very poor prognosis. Even with the development of modern diagnostic strategies and new therapies, the five-year survival rate is just 5%, with the median survival time only 14 months. Unfortunately, glioblastoma can affect patients at any age, including young children, but has a peak occurrence between the ages of 65 and 75 years. The standard treatment for GB consists of surgical resection, followed by radiotherapy and chemotherapy. However, the infiltration of GB cells into healthy adjacent brain tissue is a major obstacle for successful treatment, making complete removal of a tumour by surgery a difficult task, with the potential for tumour recurrence. Magnetic Resonance Imaging (MRI) is a non-invasive, multipurpose imaging tool used for the diagnosis and monitoring of cancerous tumours. It can provide morphological, physiological, and metabolic information about the tumour. Currently, MRI is the standard diagnostic tool for GB before the pathological examination of tissue from surgical resection or biopsy specimens. The standard MRI sequences used for diagnosis of GB include T2-Weighted (T2W), T1-Weighted (T1W), Fluid-Attenuated Inversion Recovery (FLAIR), and Contrast Enhance T1 gadolinium (CE-T1) scans. These conventional scans are used to localize the tumour, in addition to associated oedema and necrosis. Although these scans can identify the bulk of the tumour, it is known that they do not detect regions infiltrated by GB cells. The MRI signal depends upon many physical parameters including water content, local structure, tumbling rates, diffusion, and hypoxia (Dominietto, 2014). There has been considerable interest in identifying whether such biologically indirect image contrasts can be used as non-invasive imaging biomarkers, either for normal biological functions, pathogenic processes or pharmacological responses to therapeutic interventions (Atkinson et al., 2001). In fact, when new MRI methods are proposed as imaging biomarkers of particular diseases, it is crucial that they are validated against histopathology. In humans, such validation is limited to a biopsy, which is the gold standard of diagnosis for most types of cancer. Some types of biopsies can take an image-guided approach using MRI, Computed Tomography (CT) or Ultrasound (US). However, a biopsy may miss the most malignant region of the tumour and is difficult to repeat. Biomarker validation can be performed in preclinical disease models, where the animal can be terminated immediately after imaging for histological analysis. Here, in principle, co-registration of the biomarker images with the histopathology would allow for direct validation. However, in practice, most preclinical validation studies have been limited to using simple visual comparisons to assess the correlation between the imaging biomarker and underlying histopathology. First objective (Chapter 5): Histopathology is the gold standard for assessing non-invasive imaging biomarkers, with most validation approaches involving a qualitative visual inspection. To allow a more quantitative analysis, previous studies have attempted to co-register MRI with histology. However, these studies have focused on developing better algorithms to deal with the distortions common in histology sections. By contrast, we have taken an approach to improve the quality of the histological processing and analysis, for example, by taking into account the imaging slice orientation and thickness. Multiple histology sections were cut in the MR imaging plane to produce a Stacked In-plane Histology (SIH) map. This approach, which is applied to the next two objectives, creates a histopathology map which can be used as the gold standard to quantitatively validate imaging biomarkers. Second objective (Chapter 6): Glioblastoma is the most malignant form of primary brain tumour and recurrence following treatment is common. Non-invasive MR imaging is an important component of brain tumour diagnosis and treatment planning. Unfortunately, clinic MRI (T1W, T2W, CE-T1, and FLAIR) fails to detect regions of glioblastoma cell infiltration beyond the solid tumour region identified by contrast enhanced T1 scans. However, advanced MRI techniques such as Arterial Spin Labelling (ASL) could provide us with extra information (perfusion) which may allow better detection of infiltration. In order to assess whether local perfusion perturbation could provide a useful biomarker for glioblastoma cell infiltration, we quantitatively analysed the correlation between perfusion MRI (ASL) and stacked in-plane histology. This work used a mouse model of glioblastoma that mimics the infiltrative behaviour found in human patients. The results demonstrate the ability of perfusion imaging to probe regions of low tumour cell infiltration, while confirming the sensitivity limitations of clinic imaging modalities. Third objective (Chapter 7): It is widely hypothesised that Multiparametric MRI (mpMRI), can extract more information than is obtained from the constituent individual MR images, by reconstructing a single map that contains complementary information. Using the MRI and histology dataset from objective 2, we used a multi-regression algorithm to reconstruct a single map which was highly correlated (r>0.6) with histology. The results are promising, showing that mpMRI can better predict the whole tumour region, including the region of tumour cell infiltration

Automatic segmentation of ventricular volume by 3D ultrasonography in post haemorrhagic ventricular dilatation among preterm infants

To train, evaluate, and validate the application of a deep learning framework in three-dimensional ultrasound (3D US) for the automatic segmentation of ventricular volume in preterm infants with post haemorrhagic ventricular dilatation (PHVD). We trained a 2D convolutional neural network (CNN) for automatic segmentation ventricular volume from 3D US of preterm infants with PHVD. The method was validated with the Dice similarity coefficient (DSC) and the intra-class coefficient (ICC) compared to manual segmentation. The mean birth weight of the included patients was 1233.1 g (SD 309.4) and mean gestational age was 28.1 weeks (SD 1.6). A total of 152 serial 3D US from 10 preterm infants with PHVD were analysed. 230 ventricles were manually segmented. Of these, 108 were used for training a 2D CNN and 122 for validating the methodology for automatic segmentation. The global agreement for manual versus automated measures in the validation data (n=122) was excellent with an ICC of 0.944 (0.874-0.971). The Dice similarity coefficient was 0.8 (+/- 0.01). 3D US based ventricular volume estimation through an automatic segmentation software developed through deep learning improves the accuracy and reduces the processing time needed for manual segmentation using VOCAL. 3D US should be considered a promising tool to help deepen our current understanding of the complex evolution of PHVD

종양의 3차원적 위치 파악을 위한 메쉬 구조의 3D 모델링 기술 개발 및 임상적 유용성 평가

학위논문(박사) -- 서울대학교대학원 : 의과대학 의학과, 2022.2. 김희찬.배 경: 3D 프린팅은 인체 종양의 3차원적 위치를 파악하기 위한 방법으로써, 이미 의학 영역에 다양한 목적으로 보급되어 사용되고 있으나, 높은 비용과 긴 제작 시간은 3D 모델의 활용에 제약이 되고 있다. 목 적: 첫 번째 연구의 목표는 인체 장기와 그 장기가 포함하고 있는 종양의 관계를 묘사하기 위해 메쉬 구조의 3D 모델링이라고 하는 새로운 기법을 개발하는 것으로, 비용의 절감 및 출력 시간에서 장점을 보일 것으로 가정하였다. 두 번째 연구는 딥러닝을 이용한 개별화된 3D 갑상선 모델을 메쉬 구조로 제작하고 수술 전 동의를 받는 과정에 이용하여 본 기술의 임상적 유용성을 평가하고자 하였다. 방 법: 메쉬 구조의 3차원 모델링은 단층 영상에서 일정 간격으로 좌표를 추출하고, 이를 그물망(메쉬) 형태로 연결하는 구조의 레플리카를 생성한다. 인접한 해부학적 구조는 메쉬의 밀도를 변화시켜 출력하는 방식으로 구분할 수 있으며, 정상 조직과 종양은 대조되는 색상으로 표시한다. 이를 이용한 임상 연구를 위해, 수술 전 동의서 작성에 3D 모형을 이용하는 전향적 무작위 배정 대조군 비교 임상시험(KCT0005069)을 설계하였다. 갑상선 수술을 받는 환자 53명을 대상으로, 수술 동의서 작성 시 개별화된 3D 프린팅 모델을 사용하는 군과, 사용하지 않고 기존의 방식대로 동의서를 작성하는 두 그룹으로 나누었다. 이 과정에서 U-Net 기반의 딥러닝 아키텍처와 메쉬 구조의 3D 모델링 기법을 활용하여 개인화된 3D 모델을 제작하였다. 결 과: 메쉬 구조의 3D 모델링을 이용해 융합 적층 모델링(FDM) 방식의 3D 프린터를 통해 출력한 결과, 낮은 비용($0.05/cm3)과 제작 시간(1.73 min/cm3)을 보였다. 실제 3D 프린팅 된 모형은 수술 중 절제된 검체와 비교했을 때 장기-종양 해부학 및 인접 조직을 시각적으로 구분하는데 충분한 수준을 보였다. 이후 시행한 전향적 임상 연구에서, 53명의 환자들의 갑상선 모형의 평균 3D 프린팅 시간은 258.9분이었고 평균 제작비는 환자 1인당 USD 4.23였다. 모든 3차원 모형은 종양과 갑상선의 크기, 위치, 해부학적 관계를 효과적으로 반영할 수 있었다. 수술 동의서 작성시 개별화된 3D 프린팅 모델을 제공받은 그룹은 4가지 범주(일반 지식, 수술의 이점, 수술의 위험, 만족도) 모두에서 통계적으로 유의한 수준의 개선을 보였다 (모두 p <0.05). 모든 환자는 수술 후 개별화된 3D 모델을 제공받았으며, 질병, 수술 및 가능한 합병증 및 전반적인 만족도 향상에 도움이 되었음을 확인할 수 있었다. 결 론: 개별화된 3D 갑상선 모델은 수술 전 동의서 작성 과정에서 환자의 이해와 만족도를 향상시키는 효과적인 도구가 될 수 있었다. 새롭게 고안한 메쉬 구조의 3D 모델링 기법은 장기의 크기/윤곽 및 종양의 위치를 시각화 하는데 효과적이었으며, 이러한 방법론은 개별화된 치료를 위한 해부학적 모델링을 용이하게 하고, 수술 동의서 작성과 같은 설명 과정에 있어, 환자의 효과적인 의학적 지식 습득을 도울 수 있음을 확인하였다.Background: As a method of three-dimensional (3D) localization of tumor, 3D printing is introduced to medicine. However, the high costs and lengthy production times required have limited their application. Objectives: The goal of the first study was to develop a new and less costly 3D modeling method, “mesh-type 3D modeling”, to depict organ–tumor relations. The second study was designed to evaluate the clinical usefulness of a personalized mesh-type 3D-printed thyroid gland model for obtaining informed consent. Methods: For the mesh-type 3D modeling, coordinates were extracted at a specified distance interval from tomographic images, connecting them to create mesh-work replicas. Adjacent constructs were depicted by density variations, showing anatomical targets (i.e., tumors) in contrasting colors. A randomized, controlled prospective clinical trial (KCT0005069) was designed. A total of 53 patients undergoing thyroid surgery were randomly assigned to two groups: with or without a 3D-printed model of their thyroid lesion upon obtaining informed consent. A U-Net-based deep learning architecture and the mesh-type 3D modeling technique were used to fabricate the personalized 3D model. Results: To establish the mesh-type 3D modeling technique, an array of organ-solid tumor models was printed via a Fused Deposition Modeling 3D printer at a lower cost ($0.05 USD/cm3) and time expenditure (1.73 min/cm3). Printed models helped promote visual appreciation of organ-tumor anatomy and adjacent tissues. In the prospective clinical study, the mean 3D printing time was 258.9 min, and the mean price for production was USD 4.23 for each patient. The size, location, and anatomical relationship of the tumor with respect to the thyroid gland could be effectively presented. The group provided with personalized 3D-printed models significantly improved across all four categories (i.e., general knowledge, benefits of surgery, risks of surgery, and satisfaction; all p < .05). All patients received a personalized 3D model after surgery and found it helpful toward understanding the disease, operation, and possible complications, as well as enhancing their overall satisfaction. Conclusion: The personalized 3D-printed thyroid gland model may be an effective tool for improving a patient’s understanding and satisfaction during the informed consent process. Furthermore, the mesh-type 3D modeling reproduced glandular size/contour and tumor location, readily approximating the surgical specimen. This newly devised mesh-type 3D printing method may facilitate anatomical modeling for personalized care and improve patient awareness during informed surgical consent.Chapter 1. Introduction 1 Chapter 2. Materials and Methods 4 Chapter 3. Results 24 Chapter 4. Discussion 43 Chapter 5. Conclusions 49 Acknowledgements 50 Bibliography 51 Abstract in Korean 55박