Appearance Modelling and Reconstruction for Navigation in Minimally Invasive Surgery

Minimally invasive surgery is playing an increasingly important role for patient care. Whilst its direct patient benefit in terms of reduced trauma, improved recovery and shortened hospitalisation has been well established, there is a sustained need for improved training of the existing procedures and the development of new smart instruments to tackle the issue of visualisation, ergonomic control, haptic and tactile feedback. For endoscopic intervention, the small field of view in the presence of a complex anatomy can easily introduce disorientation to the operator as the tortuous access pathway is not always easy to predict and control with standard endoscopes. Effective training through simulation devices, based on either virtual reality or mixed-reality simulators, can help to improve the spatial awareness, consistency and safety of these procedures. This thesis examines the use of endoscopic videos for both simulation and navigation purposes. More specifically, it addresses the challenging problem of how to build high-fidelity subject-specific simulation environments for improved training and skills assessment. Issues related to mesh parameterisation and texture blending are investigated. With the maturity of computer vision in terms of both 3D shape reconstruction and localisation and mapping, vision-based techniques have enjoyed significant interest in recent years for surgical navigation. The thesis also tackles the problem of how to use vision-based techniques for providing a detailed 3D map and dynamically expanded field of view to improve spatial awareness and avoid operator disorientation. The key advantage of this approach is that it does not require additional hardware, and thus introduces minimal interference to the existing surgical workflow. The derived 3D map can be effectively integrated with pre-operative data, allowing both global and local 3D navigation by taking into account tissue structural and appearance changes. Both simulation and laboratory-based experiments are conducted throughout this research to assess the practical value of the method proposed

Research on real-time physics-based deformation for haptic-enabled medical simulation

This study developed a multiple effective visuo-haptic surgical engine to handle a variety of surgical manipulations in real-time. Soft tissue models are based on biomechanical experiment and continuum mechanics for greater accuracy. Such models will increase the realism of future training systems and the VR/AR/MR implementations for the operating room

Bayesian Estimation of Intrinsic Tissue Oxygenation and Perfusion from RGB Images

Multispectral imaging (MSI) can potentially assist the intra-operative assessment of tissue structure, function and viability, by providing information about oxygenation. In this paper, we present a novel technique for recovering intrinsic MSI measurements from endoscopic RGB images without custom hardware adaptations. The advantage of this approach is that it requires no modification to existing surgical and diagnostic endoscopic imaging systems. Our method uses a radiometric colour calibration of the endoscopic camera's sensor in conjunction with a Bayesian framework to recover a per-pixel measurement of the total blood volume (THb) and oxygen saturation (SO2) in the observed tissue. The sensor's pixel measurements are modelled as weighted sums over a mixture of Poisson distributions and we optimise the variables SO2 and THb to maximise the likelihood of the observations. To validate our technique, we use synthetic images generated from Monte Carlo (MC) physics simulation of light transport through soft tissue containing sub-surface blood vessels. We also validate our method on in vivo data by comparing it to a MSI dataset acquired with a hardware system that sequentially images multiple spectral bands without overlap. Our results are promising and show that we are able to provide surgeons with additional relevant information by processing endoscopic images with our modelling and inference framework

Appearance Modeling of Living Human Tissues

This is the peer reviewed version of the following article: Nunes, A.L.P., Maciel, A., Meyer, G.W., John, N.W., Baranoski, G.V.G., & Walter, M. (2019). Appearance Modeling of Living Human Tissues, Computer Graphics Forum, which has been published in final form at https://doi.org/10.1111/cgf.13604. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-ArchivingThe visual fidelity of realistic renderings in Computer Graphics depends fundamentally upon how we model the appearance of objects resulting from the interaction between light and matter reaching the eye. In this paper, we survey the research addressing appearance modeling of living human tissue. Among the many classes of natural materials already researched in Computer Graphics, living human tissues such as blood and skin have recently seen an increase in attention from graphics research. There is already an incipient but substantial body of literature on this topic, but we also lack a structured review as presented here. We introduce a classification for the approaches using the four types of human tissues as classifiers. We show a growing trend of solutions that use first principles from Physics and Biology as fundamental knowledge upon which the models are built. The organic quality of visual results provided by these Biophysical approaches is mainly determined by the optical properties of biophysical components interacting with light. Beyond just picture making, these models can be used in predictive simulations, with the potential for impact in many other areas

Development of novel laser processes for ophthalmology and dentistry

This thesis is focused on the development of novel laser processes for ophthalmology and dentistry. Firstly an in-depth investigation into the possibility of using 10.6 μm CO2 laser radiation for removal of hard dental tissue is presented. Different processing regimes are investigated to find optimal processing parameters that allow precise tissue removal without introducing any unwanted damage. Moreover the temperature increase during laser processing is measured to ensure process safety at the early stage research. When using a water spray the maximum recorded temperature increase in the pulpal chamber was ~3 oC (for a repetition rate of 250 Hz) which is below the critical level of 5.5 oC which would cause tissue necrosis. The application of picosecond plasma mediated ablation of porcine sclera to create cavities which could form portals of entry to enhance drug delivery to the posterior eye is also investigated. To ensure that the process would not present a risk of unwanted damage to the eye both the temperature rise in the tissue and power transmission through the tissue is investigated during laser processing. The feasibility of this tissue modification as a means to improve drug permeation through the sclera is investigated with a Franz cell setup. It was found that if a drug (fluorescein dextran was used to simulate drug diffusion) is placed in a 5 mm by 5 mm by 100 μm cavity its permeation through the tissue increases by 11 – 12 times in first 5 hours compared to unmodified tissue. To support experimental work and provide insight to the physical mechanism of the tissue removal process two theoretical computational models are developed. First an ALE moving boundary model is applied to investigate single pulse removal of hard dental tissue. For single line plasma mediated scanning ablation of porcine sclera a 2D finite element blow-off model is which confirms the presence of a heat accumulation phenomenon that can influence the depth of the cavity formed. Consequently in this thesis the viability of two laser processes for application to modification of hard dental and soft scleral tissue has been demonstrated which opens up the possibility for developing novel procedures for surgery and dieses treatment

Recent advances in biomedical photonic sensors: a focus on optical-fibre-based sensing

In this invited review, we provide an overview of the recent advances in biomedical pho tonic sensors within the last five years. This review is focused on works using optical-fibre technology, employing diverse optical fibres, sensing techniques, and configurations applied in several medical fields. We identified technical innovations and advancements with increased implementations of optical-fibre sensors, multiparameter sensors, and control systems in real applications. Examples of outstanding optical-fibre sensor performances for physical and biochemical parameters are covered, including diverse sensing strategies and fibre-optical probes for integration into medical instruments such as catheters, needles, or endoscopes.This work was supported by Ministerio de Ciencia e Innovación and Agencia Estatal de Investigación (PID2019-107270RB-C21/AEI/10.13039/501100011033), and TeDFeS Project (RTC-2017- 6321-1) co-funded by European FEDER funds. M.O. and J.F.A. received funding from Ministerio de Ciencia, Innovación y Universidades of Spain under Juan de la Cierva-Formación and Juan de la Cierva-Incorporación grants, respectively. P.R-V. received funding from Ministerio de Educación, Cultura y Deporte of Spain under PhD grant FPU2018/02797

Computational Multispectral Endoscopy

Minimal Access Surgery (MAS) is increasingly regarded as the de-facto approach in interventional medicine for conducting many procedures this is due to the reduced patient trauma and consequently reduced recovery times, complications and costs. However, there are many challenges in MAS that come as a result of viewing the surgical site through an endoscope and interacting with tissue remotely via tools, such as lack of haptic feedback; limited field of view; and variation in imaging hardware. As such, it is important best utilise the imaging data available to provide a clinician with rich data corresponding to the surgical site. Measuring tissue haemoglobin concentrations can give vital information, such as perfusion assessment after transplantation; visualisation of the health of blood supply to organ; and to detect ischaemia. In the area of transplant and bypass procedures measurements of the tissue tissue perfusion/total haemoglobin (THb) and oxygen saturation (SO2) are used as indicators of organ viability, these measurements are often acquired at multiple discrete points across the tissue using with a specialist probe. To acquire measurements across the whole surface of an organ one can use a specialist camera to perform multispectral imaging (MSI), which optically acquires sequential spectrally band limited images of the same scene. This data can be processed to provide maps of the THb and SO2 variation across the tissue surface which could be useful for intra operative evaluation. When capturing MSI data, a trade off often has to be made between spectral sensitivity and capture speed. The work in thesis first explores post processing blurry MSI data from long exposure imaging devices. It is of interest to be able to use these MSI data because the large number of spectral bands that can be captured, the long capture times, however, limit the potential real time uses for clinicians. Recognising the importance to clinicians of real-time data, the main body of this thesis develops methods around estimating oxy- and deoxy-haemoglobin concentrations in tissue using only monocular and stereo RGB imaging data

Essential techniques for laparoscopic surgery simulation

Laparoscopic surgery is a complex minimum invasive operation that requires long learning curve for the new trainees to have adequate experience to become a qualified surgeon. With the development of virtual reality technology, virtual reality-based surgery simulation is playing an increasingly important role in the surgery training. The simulation of laparoscopic surgery is challenging because it involves large non-linear soft tissue deformation, frequent surgical tool interaction and complex anatomical environment. Current researches mostly focus on very specific topics (such as deformation and collision detection) rather than a consistent and efficient framework. The direct use of the existing methods cannot achieve high visual/haptic quality and a satisfactory refreshing rate at the same time, especially for complex surgery simulation. In this paper, we proposed a set of tailored key technologies for laparoscopic surgery simulation, ranging from the simulation of soft tissues with different properties, to the interactions between surgical tools and soft tissues to the rendering of complex anatomical environment. Compared with the current methods, our tailored algorithms aimed at improving the performance from accuracy, stability and efficiency perspectives. We also abstract and design a set of intuitive parameters that can provide developers with high flexibility to develop their own simulators

A comprehensive review on photoacoustic-based devices for biomedical applications

The photoacoustic effect is an emerging technology that has sparked significant interest in the research field since an acoustic wave can be produced simply by the incidence of light on a material or tissue. This phenomenon has been extensively investigated, not only to perform photoacoustic imaging but also to develop highly miniaturized ultrasound probes that can provide biologically meaningful information. Therefore, this review aims to outline the materials and their fabrication process that can be employed as photoacoustic targets, both biological and non-biological, and report the main components’ features to achieve a certain performance. When designing a device, it is of utmost importance to model it at an early stage for a deeper understanding and to ease the optimization process. As such, throughout this article, the different methods already implemented to model the photoacoustic effect are introduced, as well as the advantages and drawbacks inherent in each approach. However, some remaining challenges are still faced when developing such a system regarding its fabrication, modeling, and characterization, which are also discussed.This work was supported by Fundação para a Ciência e Tecnologia national funds, under the national support to R&D units grant, through the reference project UIDB/04436/2020 and UIDP/04436/2020