A comparative study of breast surface reconstruction for aesthetic outcome assessment

Breast cancer is the most prevalent cancer type in women, and while its survival rate is generally high the aesthetic outcome is an increasingly important factor when evaluating different treatment alternatives. 3D scanning and reconstruction techniques offer a flexible tool for building detailed and accurate 3D breast models that can be used both pre-operatively for surgical planning and post-operatively for aesthetic evaluation. This paper aims at comparing the accuracy of low-cost 3D scanning technologies with the significantly more expensive state-of-the-art 3D commercial scanners in the context of breast 3D reconstruction. We present results from 28 synthetic and clinical RGBD sequences, including 12 unique patients and an anthropomorphic phantom demonstrating the applicability of low-cost RGBD sensors to real clinical cases. Body deformation and homogeneous skin texture pose challenges to the studied reconstruction systems. Although these should be addressed appropriately if higher model quality is warranted, we observe that low-cost sensors are able to obtain valuable reconstructions comparable to the state-of-the-art within an error margin of 3 mm.Comment: This paper has been accepted to MICCAI201

Quality of experience for 3-d immersive media streaming

Recent advances in media capture and processing technologies have enabled new forms of true 3-D media content that increase the degree of user immersion. The demand for more engaging forms of entertainment means that content distributors and broadcasters need to fine-tune their delivery mechanisms over the Internet as well as develop new models for quantifying and predicting user experience of these new forms of content. In the work described in this paper, we undertake one of the first studies into the quality of experience (QoE) of real-time 3-D media content streamed to virtual reality (VR) headsets for entertainment purposes, in the context of game spectating. Our focus is on tele-immersive media that embed real users within virtual environments of interactive games. A key feature of engaging and realistic experiences in full 3-D media environments, is allowing users unrestricted viewpoints. However, this comes at the cost of increased network bandwidth and the need of limiting network effects in order to transmit a realistic, real-time representation of the participants. The visual quality of 3-D media is affected by geometry and texture parameters while the temporal aspects of smooth movement and synchronization are affected by lag introduced by network transmission effects. In this paper, we investigate varying network conditions for a set of tele-immersive media sessions produced in a range of visual quality levels. Further, we investigate user navigation issues that inhibit free viewpoint VR spectating of live 3-D media. After reporting on a study with multiple users we analyze the results and assess the overall QoE with respect to a range of visual quality and latency parameters. We propose a neural network QoE prediction model for 3-D media, constructed from a combination of visual and network parameters

Development and implementation of a web-enabled 3D consultation tool for breast augmentation surgery based on 3D-image reconstruction of 2D pictures

Producing a rich, personalized Web-based consultation tool for plastic surgeons and patients is challenging

3D object reconstruction using computer vision : reconstruction and characterization applications for external human anatomical structures

Tese de doutoramento. Engenharia Informática. Faculdade de Engenharia. Universidade do Porto. 201

Three-dimensional breast assessment by multiple stereophotogrammetry after breast reconstruction with latissimus dorsi flap

Introduction: Numerous methods exist for the assessment of the female breast. Traditionally, a subjective approach was taken for surgical planning and evaluation of the postoperative outcome. Several objective methods have been developed to support this procedure, among which are laser scanning, MRI, mammography, ultrasound and photography. Recently, 3D imaging technology has been developed. Material & Method: 3D breast assessment by multiple stereophotogrammetry was examined. A custom-made imaging system with eight digital cameras arranged in four camera pods was utilised. This system was used for breast capture, resulting in eight images obtained by the cameras. The merging of these images and 3D image construction was carried out by C3D software and the volume assessment of the 3D images was made using breast analysis tool (BAT) software, developed by Glasgow University. A validation study was conducted. Nine plaster models were investigated and their volume determined by 3D stereophotogrammetry and water displacement method. Water displacement was considered to be the gold standard for comparison. The plaster models were specially made in order to represent a variety of shapes and sizes of the female breast. Each plaster model was examined 10 times by each method. Further, the volumes of the breasts of six female volunteer live models were investigated by the same two methods and the results compared. A special focus was placed on the reproducibility of the assessment. Each live model was captured with the 3D capture system three times at two different time points after retaking a special pose in a custom-made positioning frame. Altogether, each live model was captured six times, resulting in six 3D images, each of which was measured three times with BAT software. A patient study was conducted in 44 patients after unilateral immediate breast reconstruction with Latissimus dorsi flap and no contra-lateral surgery. Each patient underwent 3D imaging with the multiple stereophotogrammetry system. During capture, the special pose in the custom-made positioning frame was taken by the patient’s leaning forward almost horizontally with the upper body for the breasts to rise off the chest wall to enable full breast coverage by the cameras. 3D images were constructed with C3D software and volumes measured with BAT. For each patient, one 3D image was constructed and measured four times with BAT software. In addition to the volume determination, a shape analysis was conducted. For this purpose, 10 landmarks were determined according to recommendations in the literature. Two landmarks, sternal notch and xiphoid, were marked, forming an imaginary midline between each other and four landmarks on each breast, i.e. the medial and lateral ends of the infra-mammary fold, and the most prominent and most inferior breast points were utilised for symmetry assessment between the right and left breasts. Each landmark was recorded four times by the operator on the 3D image and three-dimensional coordinates obtained. By assessment of the left and right breasts a breast asymmetry score was calculated. Firstly, breast asymmetry was assessed objectively on the 3D images through the centroid size, which was determined as the square root of the sum of squared Euclidian distances from each landmark to the centroid. The centroid was the geometric mean of the landmarks. Secondly, asymmetry was assessed through breast volume by application of BAT software. Thirdly, asymmetry was examined through the landmarks themselves by investigation of the mismatch of the landmark configuration of one breast and its relabelled and matched reflection. The non-operated and reconstructed sides were compared and landmarks were recorded by the operator in three dimensions in four repeated tests. A decomposition of the total landmark asymmetry into its factors was conducted by fixation of the surface of the non-operated side and translation, rotation and scaling of the surface of the reconstructed side. For comparison, a subjective breast assessment was conducted by six expert observers who rated the results after breast reconstruction by subjective qualitative assessment of the symmetry in 2D images of the same 44 patients in six poses. For this purpose the Harris scale was utilised, providing a score of 1 to 4 for poor to excellent symmetry. Results: The results revealed that differences in the obtained volumes in the plaster models were not significant. In contrast, differences in the breast volumes measured in the live models were significant. The examination of the reproducibility revealed that overall reproducibility obtained by stereophotogrammetry was better than that obtained by water displacement. No correlation between breast size and reproducibility of the measurements was found. The results of the patient study demonstrated that the reproducibility of the landmarks was within 5 mm. There was a non-significant difference of the centroid sizes between both breasts. There was a significant difference of the volumes between the two breasts, with the non-operated side being larger than the reconstructed side. Volume was considered to be a more accurate measure for comparison of both breasts than centroid size as it was based on thousands of data points for the calculation as opposed to only four points of the centroid size. The statistical analysis of the landmark data provided a mathematical formula for determination of the breast asymmetry score. The average asymmetry score, derived by landmark assessment as the degree of mismatch between both sides, was 0.052 with scores ranging from 0.019 (lowest score) to 0.136 (highest score). The decomposition of the landmark-based asymmetry revealed that location was the most important factor contributing to breast asymmetry, ahead of intrinsic breast asymmetry, orientation and scale. When investigating the subjective assessment, the inter-observer agreement was good or substantial. There was moderate agreement on the controls and fair to substantial intra-observer agreement. When comparing the objective and subjective assessments, it was found that the relationship between the two scores was highly significant. Conclusion: We concluded that 3D breast assessment by multiple stereophotogrammetry was reliable for a comparative analysis and provided objective data to breast volume, shape and symmetry. A breast asymmetry score was developed, enabling an objective measurement of breast asymmetry after breast reconstruction. 3D breast assessment served as an objective method for comparison to subjective breast assessment

Engaging the Virtual Landscape: Toward an Experiential Approach to Exploring Place Through a Spatial Experience Engine

The utilization of Geographic Information Systems (GIS) and other geospatial technologies in historical inquiry and the humanities has led to a number of projects that are exploring digital representations of past landscapes and places as platforms for synthesizing and representing historical and geographic information. Recent advancements in geovisualization, immersive environments, and virtual reality offer the opportunity to generate digital representations of cultural and physical landscapes, and embed those virtual landscapes with information and knowledge from multiple GIS sources. The development of these technologies and their application to historical research has opened up new opportunities to synthesize historical records from disparate sources, represent these sources spatially in digital form, and to embed the qualitative data into those spatial representations that is often crucial to historical interpretation.;This dissertation explores the design and development of a serious game-based virtual engine, the Spatial Experience Engine (SEE), that provides an immersive and interactive platform for an experiential approach to exploring and understanding place. Through a case study focused on the late nineteenth-century urban landscape of Morgantown, West Virginia, the implementation of the SEE discussed in this dissertation demonstrates a compelling platform for building and exploring complex, virtual landscapes, enhanced with spatialized information and multimedia. The SEE not only provides an alternative approach for scholars exploring the spatial turn in history and a humanistic, experiential analysis of historical places, but its flexibility and extensibility also offer the potential for future implementations to explore a wide range of research questions related to the representation of geographic information within an immersive and interactive virtual landscape

Visual, spatial and temporal quality in video-based reconstruction of people : achieving, prototyping and evaluating

Capturing, recreating and representing a high fidelity virtual representation of the dynamic human form has long been a target for a diverse range of applications including tele-presence, games, film and TV special effects. The complexity of the challenge, to achieve a lifelike, faithful and believable representation, is such that a wide range of techniques and approaches have been developed. These are both due to research lead curiosity and requirements to address specific objective for particular problems.This work starts from a novel standpoint: that the processes of surfacing, tessellation and texturing, commonly used in 3D reconstruction, are computationally expensive and un-necessary. This work argues that by integrating the reconstruction and rendering processes into a single process that is aligned with the architecture of modern graphics hardware, a lightweight component solution can be achieved that is suitable for application on the end user systems within the many application domains. In order to achieve this aim the research undertaken seeks to both define an appropriate technique and develop detailed understanding of the reconstruction process pipeline and impacting factors. This is achieved through a complementary investigation of the tools and frameworks that are necessary to support iterative development of the approach with reliable, repeatable objective assessment. This reasons that by understanding the nature of the capture, reconstruction and presentation pipeline and by objective evaluation of the emerging reconstruction techniques this research will define an approach for 3D video based reconstruction that effectively utilises the processing potential of a single system to deliver acceptable levels of performance (speed) and fidelity (visual quality) for a componentised, multi-purpose 3D reconstruction and rendering solution. This thesis describes the research that has driven the evolution of technique and documents the iterations made. It presents a novel framework for experimentation and evaluation of the techniques and demonstrates how the use of these tools has enabled both rapid prototyping of approach and objective evaluation of improvement. The work concludes with a review of the approach taken and identifies approaches for evaluation of performance (speed) and fidelity (visual quality) that enable both repeatable experimentation within the research pipeline and reliable comparison of the end-to-end process against other techniques