This paper presents results in the image processing and visualization aspects of a virtual liver biopsy system (a system for simulating the medical procedure of liver biopsy). The creation of 3D models from 2D images of the organs involved is described, and segmentation requirements of this process are discussed. Endoscopic images of the liver that simulate the needle's point of view are created by means of combined volume and surface rendering. For this purpose ray casting is used with the ray start and end points being constrained within a surface rendered environment. Visualization of the needle insertion process from an exterior point of view is presented. A real-time sectional imaging component is also used in which the displayed 2D-image section of the 3D volume tracks the tip of the needleThis paper presents results in the image processing and visualization aspects of a virtual liver biopsy system (a system for simulating the medical procedure of liver biopsy). The creation of 3D models from 2D images of the organs involved is described, and segmentation requirements of this process are discussed. Endoscopic images of the liver that simulate the needle's point of view are created by means of combined volume and surface rendering. For this purpose ray casting is used with the ray start and end points being constrained within a surface rendered environment. Visualization of the needle insertion process from an exterior point of view is presented. A real-time sectional imaging component is also used in which the displayed 2D-image section of the 3D volume tracks the tip of the needle

Agrafiotis, D

Bull, DR

Canagarajah, CN

Halliwell, [No Value]

Halliwell, M

Jones, MG

Nikolov, SG

English

Explore Bristol Research

                          Agrafiotis, D., Jones, M. G., Nikolov, S. G., Halliwell, . N. V., Halliwell, M.,Bull, D. R., & Canagarajah, C. N. (2001). Virtual liver biopsy: imageprocessing and 3D visualization. In 2001 International Conference on  ImageProcessing, Thessaloniki, Greece. (Vol. 2, pp. 331 - 334). Institute ofElectrical and Electronics Engineers (IEEE). 10.1109/ICIP.2001.958495Link to published version (if available):10.1109/ICIP.2001.958495Link to publication record in Explore Bristol ResearchPDF-documentUniversity of Bristol - Explore Bristol ResearchGeneral rightsThis document is made available in accordance with publisher policies. Please cite only the publishedversion using the reference above. Full terms of use are available:http://www.bristol.ac.uk/pure/about/ebr-terms.htmlTake down policyExplore Bristol Research is a digital archive and the intention is that deposited content should not beremoved. However, if you believe that this version of the work breaches copyright law please contactopen-access@bristol.ac.uk and include the following information in your message:• Your contact details• Bibliographic details for the item, including a URL• An outline of the nature of the complaintOn receipt of your message the Open Access Team will immediately investigate your claim, make aninitial judgement of the validity of the claim and, where appropriate, withdraw the item in questionfrom public view.VIRTUAL LIVER BIOPSY: IMAGE PROCESSING AND 3D VISUALIZATION D. Agrafiotis', M.G. Jones', S. G.Nikolov', M. Halliwel12, D. Bull', N. Canagarajah' 'Image Communications Group, CCR, University of Bristol, Bristol BS8 lUB, U.K. 2Centre for Physics & Engineering Research in Medicine, University of Bristol & United Bristol Healthcare Trust, Bristol General Hospital, Bristol BS1 6SY, U.K. ABSTRACT This paper presents results in the image processing and visualization aspects of a virtual liver biopsy system (a system for simulating the medical procedure of liver biopsy). The creation of 3D models from 2D images of the organs involved is described, and segmentation requirements of this process are discussed. Endoscopic images of the liver that simulate the needle's point of view are created by means of combined volume and surface rendering. For this purpose ray casting is used with the ray start and end points being constrained within a surface rendered environment. Visualization of the needle insertion process from an exterior point of view is presented. A real- time sectional imaging component is also used in which the displayed 2D-image section of the 3D volume tracks the tip of the needle. 1. INTRODUCTION Computer-based surgical simulation finds application in education and training, surgical planning and intra- operative assistance. Surgical simulation can be used in medical education and training to reduce costs, provide experience of a geater variety of pathologies, and to make it possible to repeat or replay training procedures. In surgical planning simulation can enable rehearsal of difficult procedures or planning on patient specific anatomy. Visual realism and real-time interactions are essential in surgical simulation. Generally the requirements of a surgical simulator include the following components [ 11: acquisition, processing and segmentation of 3D medical images; creation and visualization of enhanced 3D object models with measured material properties; visual feedback through interactive volume visualization; image manipulation; [if used for training and rehearsal] haptic feedback through a surgical instrumnt controlled by an 0-7803-6725-1/01/$10.00 02001 IEEE 33 1 electromechanical force feedback device and model manipulation through the haptic interface. This paper presents some aspects of the image processing and visualization modules of a virtual liver biopsy system. Liver biopsy is a medical procedure performed to obtain a small piece of liver tissue for diagnosis. A special needle is used by the doctor (usually a radiologist) who conducts the biopsy. Computed tomography or ultrasound is used for guidance of the needle to the biopsy site. The risks associated with liver biopsy include prolonged internal bleeding and puncturing of other internal organs 121. In 0.4% of cases a patient With liver cancer can develop a fatal hemorrhage due to puncturing of the blood vessels supplying a t m r .  A simulation of liver biopsy could potentially reduce the operational risks by providing the opportunity for better training and planning of the real procedure. Three- dimensional visualization of the organs and structures involved in this operation is of great importance. 1.2. Visualization of Volumetric Images Volume visualization is the process of projecting a multidimensional (volumetric) image onto a two- dimensional image plane for the purpose of gaining an understanding of the structure contained within the volumetric image. It finds extensive use in the medical field where imaging modalities like MR and CT result in such volumetric images. Two broad categories of volume visualization algorithms are surface rendering and volume rendering, based on the type of primitives used (polygons, or voxels). 1.2.1. Surface rendering Surface based rendering involves the use of algorithms which fit  surface primitives. such as polygons, to constant- value contour surfaces in volumetric images - and then render the resulting geometric representation. One of the most well-know surface rendering algorithms is Marching Cubes (MC) 131. Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 5, 2009 at 08:58 from IEEE Xplore.  Restrictions apply.This algorithm proceeds by examining eight neighboring voxels on two adjacent slices (four on each slice) at each step. The values of Ihe resulting cube vertices determine whether and if so how, a surface passes through a voxel, via examination of a pre-calculated table of edge intersection cases. Dt:cimation can be employed to reduce the large number of polygons produced by the algorithm 1.2.2. Volume render;ng Volume rendering approaches omit any intermediate geometric representation and instead process all the voxels in the 3D dataset in order to produce a 2D image of the volume. Two basic. categories of volume rendering algorithms are object space (or forward mapping) and image space (or backward mapping). Object space techniques [4] calculate the projection and contribution of each voxel to the pixels in the image plane. Image space techniques [5] cast rays from the eye through each pixel on the screen into the volume and process the voxels along the rays. Acceleration methods are usually necessary to achieve some level of interactivity. Such methods are available through soft ware and hardware [ 61. 1.2.3. Volume naviga!ion Volume navigation is the interactive exploration of volumetric images by flying (i.e. translating and rotating) the viewpoint through the data, producing a rendered view at each frame [7]. In this way, it becomes easier to avoid occluding objects and focus on structures of interest. Discerning small complicated internal structures can, potentially, be performed more accurately. Perspective projection is necessary if things are to look natural since the viewpoint is very close to the data. Recently there has been an extensive hiterest in volume navigation for the simulation of endoscopic procedures, giving rise to a new technique named virtual endoscopy [SI. The hollow areas of anatomy through which a catheter would normally m v e  serve as the regions of low opacity through which the viewpoint is navigated. 2. IMAGE PRCCESSING & SEGMENTATION A simulation system for patient-specific data requires segmentation of CT or MR images using an interactive semi-automatic segmentation tool. The radiological examinations that arc usually performed for the liver, are spiral CT during arlerial portography (CTAP), contrast- enhanced CT, or MR scans with a high vessel-to-tissue contrast in the portal vein [9]. All result in images where the hepatic vasculatue and any existing tumors differ in g a y  values from the rest of the liver. The anatomical structures that need 10 be segmented are that of the liver parenchyma (liver contour), the vessels, the gallbladder, and any lesions that night be present. (a) (b) Figure 1: Liver segmentation: a) VHD slice; b) segmented and labelled liver. A good candidate method for tracing the liver contour is the live wire algorithm [lo]. With this method. object contours are found as minimal cost paths between interactively placed contour points, With respect to an underlying cost function. An optimized version of this algorithm for segmentation of 3D medical images is presented in [ l l ] .  For the extraction of the contrast- enhanced intrahepatic vessels, a threshold-based method can be used, like threshold-based region gowing. The choice of threshold can be done interactively or by means of histogram manipulation. For example, in [9] a histogram entropy method together with a knowledge based algorithm were used for choosing an optimal threshold for the segmentation of liver vessels. Similar methods can be used for extraction of the remaining structures. For this study, the full colour axial anatomical images of the male body from the Visible Human Dataset (VHD), National Library of Medicine, were used [12]. These are 24-hit colour images of slices of the body taken at lmm intervals containing 2048x1216 pixels and with an in-slice resolution of 0.33 mm. Out of the total 1878 slices, 183 include the liver. The WID also includes the same number of axial CT images of the whole body taken at lmm intervals. The colour slices were preferred due to their superior resolution, which allows easier recognition of the various anatomical elements. Using these slices the liver contour was hand traced and separated from the rest of the dataset (Figure la). It was then gray scaled and cropped to create an 8-bit volume of size 650x650~183. For computational speed the volume was further reduced to a size of 216x216~183 yielding lxlxl mm3 sized voxels. Thresholding was applied to extract the main anatomical structures resulting in three binary volumes representing the parenchyma, vessels (hepatic and portal veins), and gall bladder. Further processing was carried out on the vessels: holes -islands of voxels totally ,surrounded by vessels- were filled and connected component analysis was applied to select the largest components (Figure lb). 332 Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 5, 2009 at 08:58 from IEEE Xplore.  Restrictions apply.Figure 2: The visualization setup. Left panel (anti- clockwise starting from bottom left): needles point of view, opposite view; needle insertion overview; view perpendicular to needle. Right panel (top to bottom): axial; sagittal; coronal sectional images. 3. 3D VISUALIZATION We define the visualization requirements of a virtual liver biopsy system as follows: an endoscopic visualization component which presents the data from the “needle’s point of view”, and thus enables the user to see what is ahead of the needle all the time; an exterior visualization component which provides an overview of the needle insertion process; a real-time sectional imaging component, in which the displayed 2D image section of the 3D volume tracks the tip of the needle. Two additional cameras are used: one exactly opposite the needle tip, either on the parenchyma surface or on the target object, e.g. t umr ,  (depending on where the needle is aiming); one external camera perpendicular to the needle and focused on the tip. The visualization setup is shown in Figure 2. 3.1. Endoscopic visualization: Needle point of view The MC algorithm is used to extract a polygonal representation of the anatomical structures of interest (parenchyma, gall bladder, and vessels) from the three segmented binary volumes (Figure 2a,b). Natural colours are assigned to the reconstructed surfaces - for example: brown for the liver; green-yellow for the gall bladder; and red for the vessels. To avoid staircase artefacts due to the high frequencies present in the binary datasets, Gaussian smoothing is applied. For a smoother reconstruction of the vessels, distance maps are calculated which are then processed using the MC algorithm. The polygonal representation of the parenchyma surface is used in all the endoscopic visualizations. The viewpoint is following the path of the needle. The camera Figure 3: Surface rendering: a) liver and gall bladder; b) vessels. [horizontal] field-of-view is set to 60 degrees. The focal point of the camera is situated very close to the near clipping plane of the frustum. For volume rendering, the original gray scale dataset is used. Perspective projection is used in all cases. A pure surface-rendered approach can be used for fast interactive sessions (Figure 4a). Combining volume and surface rendering can potentially yield more accurate and detailed visualization while maintaining some level of interactivity (Figure 4b). In this case ray-casting is used with the rays being enclosed and bounded by the surtiace rendered liver. The outer surface polygons are rendered i i s t  (parenchyma surface) and the depth values of the pixels are used to bound the rays. Pixel colours obtained from the ray casting are blended with the colours produced from surface rendering [ 131. A shape-based kernel can also be used to further bound the rays [14]. Surface representations are used for those regions outside the kernel (which comprise the remaining vessels and parenchyma surface) and volume rendering is used inside. Clipping takes place to empty the kernel from any contained geometry (Figure 6). In order to visualize what the needle is aiming at, the intersection of a ray (from the tip of the needle along the direction of projection) with the geometric representation of the parenchyma and with the biopsy target is calculated at every frame. A camera looking towards the needle is placed at the point of intersection (see Figure2: The visualization setup), allowing for a quick estimate of whether the needle will meet its target or not. (a) (b) Figure 4: Endoscopic view: a) surface rendering; b) volume rendering (ray casting). 333 Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 5, 2009 at 08:58 from IEEE Xplore.  Restrictions apply.Figure 5: Needle insertion overview: a) surface rendering b) volume rcndering (gallbladder omitted) 3.2. Exterior visualization: needle insertion overview Visualization of the needle insertion into the liver requires the combination of ii moving needle [surface] model with a static surface or volume rendered image of the liver and possibly surrounding organs. In the case of pure surface rendering, it is a matter of assigning different transparency values to the visualtzed structures (Figure 5a). In the case of volume renderin;; (Figure 5b), in order to combine the needle surface with the voxel data of the organs, depth information from both surface and volume rendering has to be considered. Two cases of overlap can be recognized [15]: the needle is out of the organ, in which case the rendered image of the needle is included in the output image; the needle is in the organ, in which case a-blending takes place to combine the needle and organs according to accumulated opacity. 4. CONCLUSIONS The image processing and visualization aspects and requirements of a virtual liver biopsy system developed by the authors havc: been analyzed and discussed. Visualization results were presented using data from the Visible Human prclject, the segmentation of which was described. Both surface and volume rendering methods were employed in order to achieve a detailed visualization with adequate interactivity. In our visualization setup, multiple views are used to visualize the insertion and path of the biopsy needle, together with a sectional imaging component in which the displayed 2D image section of the 3D volume tracks tlie tip of the needle. This system could be potentially used in planning of real liver biopsy procedures. Acknowledgements The financial support of the State Scholarship Foundation, Greece, is gratefully acknowledged. Figure 6: Endoscopic view using combined volume and surface rendering with (a) and without (b) a kernel. REFERENCES 113 S. Gibson, J. Samosky, A. Mor, C. Fyock et.al, “Simulating Arthroscopic Knee Surgery using Volumetric Object Representations, Real-Time Volume Rendering and Haptic Feedback ”, Proc. of CVRMed-MRCAS, 1997, pp. 369-378 [2] Hepatitis C F o m :  http://www.hepatitis-c.ddbiopsy.htm [3] W.E.Lorensen, H.E.Cline, “Marching Cubes: A High Resolution 3D Surface Constxuction Algorithm”, Computer Graphics, Vol. 21, No. 4, July 1987, pp.163 -169 [4] Lee.Westover, “Footprint Evaluation for Volume Rendering”, Computer Graphics, Vol. 24, N0.4, pp. 367-376 [SI M. Levoy, “Display of Surfaces ftom Volume Data”, IEEE Comp.Graphics & Applications, Vo1.8, No.3, May 88,pp.29-37 [6] R. Yagel, “Towards Real-Time Volume Rendexing“, Proceedings of GRAPHICON96, Vol. 1, July 1996, pp. 230-241. [71 M.L.Brady, K.K.Jung, H.T.Nguyen, T.P.Nguyen, “Interactive Volume Navigation”, IEEE Trans. On Visualization And Computer Graphics, Vol. 4, No.3, 1998, pp. 243-256 [SI M. Wan, W. Li, K. Kreeger, I. Bitter, A. Kaufman, Z .  Liang, D. Chen, and M. Wax, 3D Virtual Colonoscopy with Real-time Volume Rendering, SPIE Medical Imaging 2000, pp., Feb. 2000 [9] G. Glombitza, W.Lamade, A.Demiris, M.Gopfert et.al, ”Virtual planning of liver resections: image processing, visualization and volumetric evaluation”, International Journal of Medical Informatics, No. 53, 1999, pp.225-237 [ 101 J.K.Udupa, A Samarasekera, W.A.Barrett, “Boundary detection via dynamic programming’’, Proc. SPIE Vol 1808, pp. [ l  11 A.Schenk, G.Prause, H.Peitgen, “Efficient Semiautomatic Segmentation of 3D Objects in Medical Images“, Proc. of Medical Image Computing and Computer-assisted Intervention (MICCAI), Springer, LNCS, Vol. 1935, pp. 186-195 [12] U.S. National Library of Medicine, “The Visible Human project” [13] W.Schroeder, K. Martin, B. Lorensen, “The Visualization Toolkit (2nd edition)”, Prentice Hall, 1998. [14] D.Agrafiotis, M.Jones, S.Nikolov, D.Bul1 et.al, “Endoscopic visualization and exploration of the liver using combined volume and surface rendering”, Proc. ICAV3D 2001 [15] Keun Ho Kim, Jong Beom Ra, Hyun Wook Park, “Fast visualization of needle motion in volume rendered image for spine biopsy simulator“, SPIE Medical Imaging 2000: Image Gisplay and Visualization, 2000, pp. 218-227 33-39, 1992 3 34 Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 5, 2009 at 08:58 from IEEE Xplore.  Restrictions apply.

Footprint Evaluation for Volume Rendering”,

Fyock et.al, “Simulating Arthroscopic Knee Surgery using Volumetric Object Representations, Real-Time Volume Rendering and Haptic Feedback ”,

Hyun Wook Park, “Fast visualization of needle motion in volume rendered image for spine biopsy simulator“, SPIE Medical Imaging 2000: Image Gisplay and Visualization,

Library of Medicine, “The Visible Human project”

Marching Cubes: A High Resolution 3D Surface Constxuction Algorithm”,

S.Nikolov, D.Bul1 et.al, “Endoscopic visualization and exploration of the liver using combined volume and surface rendering”,

The Visualization Toolkit (2nd edition)”,

Towards Real-Time Volume Rendexing“,

W.Lamade, A.Demiris, M.Gopfert et.al, ”Virtual planning of liver resections: image processing, visualization and volumetric evaluation”,

Virtual liver biopsy: image processing and 3D visualization

Agrafiotis, D., Jones, MG., Nikolov, SG., Halliwell, N. V., Halliwell, M.,Bull, DR., & Canagarajah, CN. (2001). Virtual liver biopsy: imageprocessing and 3D visualization. In 2001 International Conference onImage Processing, Thessaloniki, Greece (Vol. 2, pp. 331 - 334).Institute of Electrical and Electronics Engineers (IEEE).https://doi.org/10.1109/ICIP.2001.958495Peer reviewed versionLink to published version (if available):10.1109/ICIP.2001.958495Link to publication record on the Bristol Research PortalPDF-documentUniversity of Bristol – Bristol Research PortalGeneral rightsThis document is made available in accordance with publisher policies. Please cite only thepublished version using the reference above. Full terms of use are available:http://www.bristol.ac.uk/red/research-policy/pure/user-guides/brp-terms/VIRTUAL LIVER BIOPSY: IMAGE PROCESSING AND 3D VISUALIZATION D. Agrafiotis', M.G. Jones', S. G.Nikolov', M. Halliwel12, D. Bull', N. Canagarajah' 'Image Communications Group, CCR, University of Bristol, Bristol BS8 lUB, U.K. 2Centre for Physics & Engineering Research in Medicine, University of Bristol & United Bristol Healthcare Trust, Bristol General Hospital, Bristol BS1 6SY, U.K. ABSTRACT This paper presents results in the image processing and visualization aspects of a virtual liver biopsy system (a system for simulating the medical procedure of liver biopsy). The creation of 3D models from 2D images of the organs involved is described, and segmentation requirements of this process are discussed. Endoscopic images of the liver that simulate the needle's point of view are created by means of combined volume and surface rendering. For this purpose ray casting is used with the ray start and end points being constrained within a surface rendered environment. Visualization of the needle insertion process from an exterior point of view is presented. A real- time sectional imaging component is also used in which the displayed 2D-image section of the 3D volume tracks the tip of the needle. 1. INTRODUCTION Computer-based surgical simulation finds application in education and training, surgical planning and intra- operative assistance. Surgical simulation can be used in medical education and training to reduce costs, provide experience of a geater variety of pathologies, and to make it possible to repeat or replay training procedures. In surgical planning simulation can enable rehearsal of difficult procedures or planning on patient specific anatomy. Visual realism and real-time interactions are essential in surgical simulation. Generally the requirements of a surgical simulator include the following components [ 11: acquisition, processing and segmentation of 3D medical images; creation and visualization of enhanced 3D object models with measured material properties; visual feedback through interactive volume visualization; image manipulation; [if used for training and rehearsal] haptic feedback through a surgical instrumnt controlled by an 0-7803-6725-1/01/$10.00 02001 IEEE 33 1 electromechanical force feedback device and model manipulation through the haptic interface. This paper presents some aspects of the image processing and visualization modules of a virtual liver biopsy system. Liver biopsy is a medical procedure performed to obtain a small piece of liver tissue for diagnosis. A special needle is used by the doctor (usually a radiologist) who conducts the biopsy. Computed tomography or ultrasound is used for guidance of the needle to the biopsy site. The risks associated with liver biopsy include prolonged internal bleeding and puncturing of other internal organs 121. In 0.4% of cases a patient With liver cancer can develop a fatal hemorrhage due to puncturing of the blood vessels supplying a t m r .  A simulation of liver biopsy could potentially reduce the operational risks by providing the opportunity for better training and planning of the real procedure. Three- dimensional visualization of the organs and structures involved in this operation is of great importance. 1.2. Visualization of Volumetric Images Volume visualization is the process of projecting a multidimensional (volumetric) image onto a two- dimensional image plane for the purpose of gaining an understanding of the structure contained within the volumetric image. It finds extensive use in the medical field where imaging modalities like MR and CT result in such volumetric images. Two broad categories of volume visualization algorithms are surface rendering and volume rendering, based on the type of primitives used (polygons, or voxels). 1.2.1. Surface rendering Surface based rendering involves the use of algorithms which fit  surface primitives. such as polygons, to constant- value contour surfaces in volumetric images - and then render the resulting geometric representation. One of the most well-know surface rendering algorithms is Marching Cubes (MC) 131. Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 5, 2009 at 08:58 from IEEE Xplore.  Restrictions apply.This algorithm proceeds by examining eight neighboring voxels on two adjacent slices (four on each slice) at each step. The values of Ihe resulting cube vertices determine whether and if so how, a surface passes through a voxel, via examination of a pre-calculated table of edge intersection cases. Dt:cimation can be employed to reduce the large number of polygons produced by the algorithm 1.2.2. Volume render;ng Volume rendering approaches omit any intermediate geometric representation and instead process all the voxels in the 3D dataset in order to produce a 2D image of the volume. Two basic. categories of volume rendering algorithms are object space (or forward mapping) and image space (or backward mapping). Object space techniques [4] calculate the projection and contribution of each voxel to the pixels in the image plane. Image space techniques [5] cast rays from the eye through each pixel on the screen into the volume and process the voxels along the rays. Acceleration methods are usually necessary to achieve some level of interactivity. Such methods are available through soft ware and hardware [ 61. 1.2.3. Volume naviga!ion Volume navigation is the interactive exploration of volumetric images by flying (i.e. translating and rotating) the viewpoint through the data, producing a rendered view at each frame [7]. In this way, it becomes easier to avoid occluding objects and focus on structures of interest. Discerning small complicated internal structures can, potentially, be performed more accurately. Perspective projection is necessary if things are to look natural since the viewpoint is very close to the data. Recently there has been an extensive hiterest in volume navigation for the simulation of endoscopic procedures, giving rise to a new technique named virtual endoscopy [SI. The hollow areas of anatomy through which a catheter would normally m v e  serve as the regions of low opacity through which the viewpoint is navigated. 2. IMAGE PRCCESSING & SEGMENTATION A simulation system for patient-specific data requires segmentation of CT or MR images using an interactive semi-automatic segmentation tool. The radiological examinations that arc usually performed for the liver, are spiral CT during arlerial portography (CTAP), contrast- enhanced CT, or MR scans with a high vessel-to-tissue contrast in the portal vein [9]. All result in images where the hepatic vasculatue and any existing tumors differ in g a y  values from the rest of the liver. The anatomical structures that need 10 be segmented are that of the liver parenchyma (liver contour), the vessels, the gallbladder, and any lesions that night be present. (a) (b) Figure 1: Liver segmentation: a) VHD slice; b) segmented and labelled liver. A good candidate method for tracing the liver contour is the live wire algorithm [lo]. With this method. object contours are found as minimal cost paths between interactively placed contour points, With respect to an underlying cost function. An optimized version of this algorithm for segmentation of 3D medical images is presented in [ l l ] .  For the extraction of the contrast- enhanced intrahepatic vessels, a threshold-based method can be used, like threshold-based region gowing. The choice of threshold can be done interactively or by means of histogram manipulation. For example, in [9] a histogram entropy method together with a knowledge based algorithm were used for choosing an optimal threshold for the segmentation of liver vessels. Similar methods can be used for extraction of the remaining structures. For this study, the full colour axial anatomical images of the male body from the Visible Human Dataset (VHD), National Library of Medicine, were used [12]. These are 24-hit colour images of slices of the body taken at lmm intervals containing 2048x1216 pixels and with an in-slice resolution of 0.33 mm. Out of the total 1878 slices, 183 include the liver. The WID also includes the same number of axial CT images of the whole body taken at lmm intervals. The colour slices were preferred due to their superior resolution, which allows easier recognition of the various anatomical elements. Using these slices the liver contour was hand traced and separated from the rest of the dataset (Figure la). It was then gray scaled and cropped to create an 8-bit volume of size 650x650~183. For computational speed the volume was further reduced to a size of 216x216~183 yielding lxlxl mm3 sized voxels. Thresholding was applied to extract the main anatomical structures resulting in three binary volumes representing the parenchyma, vessels (hepatic and portal veins), and gall bladder. Further processing was carried out on the vessels: holes -islands of voxels totally ,surrounded by vessels- were filled and connected component analysis was applied to select the largest components (Figure lb). 332 Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 5, 2009 at 08:58 from IEEE Xplore.  Restrictions apply.Figure 2: The visualization setup. Left panel (anti- clockwise starting from bottom left): needles point of view, opposite view; needle insertion overview; view perpendicular to needle. Right panel (top to bottom): axial; sagittal; coronal sectional images. 3. 3D VISUALIZATION We define the visualization requirements of a virtual liver biopsy system as follows: an endoscopic visualization component which presents the data from the “needle’s point of view”, and thus enables the user to see what is ahead of the needle all the time; an exterior visualization component which provides an overview of the needle insertion process; a real-time sectional imaging component, in which the displayed 2D image section of the 3D volume tracks the tip of the needle. Two additional cameras are used: one exactly opposite the needle tip, either on the parenchyma surface or on the target object, e.g. t umr ,  (depending on where the needle is aiming); one external camera perpendicular to the needle and focused on the tip. The visualization setup is shown in Figure 2. 3.1. Endoscopic visualization: Needle point of view The MC algorithm is used to extract a polygonal representation of the anatomical structures of interest (parenchyma, gall bladder, and vessels) from the three segmented binary volumes (Figure 2a,b). Natural colours are assigned to the reconstructed surfaces - for example: brown for the liver; green-yellow for the gall bladder; and red for the vessels. To avoid staircase artefacts due to the high frequencies present in the binary datasets, Gaussian smoothing is applied. For a smoother reconstruction of the vessels, distance maps are calculated which are then processed using the MC algorithm. The polygonal representation of the parenchyma surface is used in all the endoscopic visualizations. The viewpoint is following the path of the needle. The camera Figure 3: Surface rendering: a) liver and gall bladder; b) vessels. [horizontal] field-of-view is set to 60 degrees. The focal point of the camera is situated very close to the near clipping plane of the frustum. For volume rendering, the original gray scale dataset is used. Perspective projection is used in all cases. A pure surface-rendered approach can be used for fast interactive sessions (Figure 4a). Combining volume and surface rendering can potentially yield more accurate and detailed visualization while maintaining some level of interactivity (Figure 4b). In this case ray-casting is used with the rays being enclosed and bounded by the surtiace rendered liver. The outer surface polygons are rendered i i s t  (parenchyma surface) and the depth values of the pixels are used to bound the rays. Pixel colours obtained from the ray casting are blended with the colours produced from surface rendering [ 131. A shape-based kernel can also be used to further bound the rays [14]. Surface representations are used for those regions outside the kernel (which comprise the remaining vessels and parenchyma surface) and volume rendering is used inside. Clipping takes place to empty the kernel from any contained geometry (Figure 6). In order to visualize what the needle is aiming at, the intersection of a ray (from the tip of the needle along the direction of projection) with the geometric representation of the parenchyma and with the biopsy target is calculated at every frame. A camera looking towards the needle is placed at the point of intersection (see Figure2: The visualization setup), allowing for a quick estimate of whether the needle will meet its target or not. (a) (b) Figure 4: Endoscopic view: a) surface rendering; b) volume rendering (ray casting). 333 Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 5, 2009 at 08:58 from IEEE Xplore.  Restrictions apply.Figure 5: Needle insertion overview: a) surface rendering b) volume rcndering (gallbladder omitted) 3.2. Exterior visualization: needle insertion overview Visualization of the needle insertion into the liver requires the combination of ii moving needle [surface] model with a static surface or volume rendered image of the liver and possibly surrounding organs. In the case of pure surface rendering, it is a matter of assigning different transparency values to the visualtzed structures (Figure 5a). In the case of volume renderin;; (Figure 5b), in order to combine the needle surface with the voxel data of the organs, depth information from both surface and volume rendering has to be considered. Two cases of overlap can be recognized [15]: the needle is out of the organ, in which case the rendered image of the needle is included in the output image; the needle is in the organ, in which case a-blending takes place to combine the needle and organs according to accumulated opacity. 4. CONCLUSIONS The image processing and visualization aspects and requirements of a virtual liver biopsy system developed by the authors havc: been analyzed and discussed. Visualization results were presented using data from the Visible Human prclject, the segmentation of which was described. Both surface and volume rendering methods were employed in order to achieve a detailed visualization with adequate interactivity. In our visualization setup, multiple views are used to visualize the insertion and path of the biopsy needle, together with a sectional imaging component in which the displayed 2D image section of the 3D volume tracks tlie tip of the needle. This system could be potentially used in planning of real liver biopsy procedures. Acknowledgements The financial support of the State Scholarship Foundation, Greece, is gratefully acknowledged. Figure 6: Endoscopic view using combined volume and surface rendering with (a) and without (b) a kernel. REFERENCES 113 S. Gibson, J. Samosky, A. Mor, C. Fyock et.al, “Simulating Arthroscopic Knee Surgery using Volumetric Object Representations, Real-Time Volume Rendering and Haptic Feedback ”, Proc. of CVRMed-MRCAS, 1997, pp. 369-378 [2] Hepatitis C F o m :  http://www.hepatitis-c.ddbiopsy.htm [3] W.E.Lorensen, H.E.Cline, “Marching Cubes: A High Resolution 3D Surface Constxuction Algorithm”, Computer Graphics, Vol. 21, No. 4, July 1987, pp.163 -169 [4] Lee.Westover, “Footprint Evaluation for Volume Rendering”, Computer Graphics, Vol. 24, N0.4, pp. 367-376 [SI M. Levoy, “Display of Surfaces ftom Volume Data”, IEEE Comp.Graphics & Applications, Vo1.8, No.3, May 88,pp.29-37 [6] R. Yagel, “Towards Real-Time Volume Rendexing“, Proceedings of GRAPHICON96, Vol. 1, July 1996, pp. 230-241. [71 M.L.Brady, K.K.Jung, H.T.Nguyen, T.P.Nguyen, “Interactive Volume Navigation”, IEEE Trans. On Visualization And Computer Graphics, Vol. 4, No.3, 1998, pp. 243-256 [SI M. Wan, W. Li, K. Kreeger, I. Bitter, A. Kaufman, Z .  Liang, D. Chen, and M. Wax, 3D Virtual Colonoscopy with Real-time Volume Rendering, SPIE Medical Imaging 2000, pp., Feb. 2000 [9] G. Glombitza, W.Lamade, A.Demiris, M.Gopfert et.al, ”Virtual planning of liver resections: image processing, visualization and volumetric evaluation”, International Journal of Medical Informatics, No. 53, 1999, pp.225-237 [ 101 J.K.Udupa, A Samarasekera, W.A.Barrett, “Boundary detection via dynamic programming’’, Proc. SPIE Vol 1808, pp. [ l  11 A.Schenk, G.Prause, H.Peitgen, “Efficient Semiautomatic Segmentation of 3D Objects in Medical Images“, Proc. of Medical Image Computing and Computer-assisted Intervention (MICCAI), Springer, LNCS, Vol. 1935, pp. 186-195 [12] U.S. National Library of Medicine, “The Visible Human project” [13] W.Schroeder, K. Martin, B. Lorensen, “The Visualization Toolkit (2nd edition)”, Prentice Hall, 1998. [14] D.Agrafiotis, M.Jones, S.Nikolov, D.Bul1 et.al, “Endoscopic visualization and exploration of the liver using combined volume and surface rendering”, Proc. ICAV3D 2001 [15] Keun Ho Kim, Jong Beom Ra, Hyun Wook Park, “Fast visualization of needle motion in volume rendered image for spine biopsy simulator“, SPIE Medical Imaging 2000: Image Gisplay and Visualization, 2000, pp. 218-227 33-39, 1992 3 34 Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 5, 2009 at 08:58 from IEEE Xplore.  Restrictions apply.

https://research-information.bris.ac.uk/ws/files/3013305/agrafiotis_IEEE_ICIP_2001.pdf

Virtual liver biopsy: image processing and 3D visualization

Abstract

Similar works

Full text

Available Versions

Explore Bristol Research

Explore Bristol Research