Realistic and interactive high-resolution 4D environments for real-time surgeon and patient interaction

Copyright © 2016 John Wiley & Sons, Ltd. Background: Remote consultations that are realistic enough to be useful medically offer considerable clinical, logistical and cost benefits. Despite advances in virtual reality and vision hardware and software, these benefits are currently often unrealised. Method: The proposed approach combines high spatial and temporal resolution 3D and 2D machine vision with virtual reality techniques, in order to develop new environments and instruments that will enable realistic remote consultations and the generation of new types of useful clinical data. Results: New types of clinical data have been generated for skin analysis and respiration measurement; and the combination of 3D with 2D data was found to offer potential for the generation of realistic virtual consultations. Conclusion: An innovative combination of high resolution machine vision data and virtual reality online methods, promises to provide advanced functionality and significant medical benefits, particularly in regions where populations are dispersed or access to clinicians is limited. Copyright © 2016 John Wiley & Sons, Ltd

Markov-Gibbs Random Field Approach for Modeling of Skin Surface Textures

Medical imaging has been contributing to dermatology by providing computer-based assistance by 2D digital imaging of skin and processing of images. Skin imaging can be more effective by inclusion of 3D skin features. Furthermore, clinical examination of skin consists of both visual and tactile inspection. The tactile sensation is related to 3D surface profiles and mechanical parameters. The 3D imaging of skin can also be integrated with haptic technology for computer-based tactile inspection. The research objective of this work is to model 3D surface textures of skin. A 3D image acquisition set up capturing skin surface textures at high resolution (~0.1 mm) has been used. An algorithm to extract 2D grayscale texture (height map) from 3D texture has been presented. The extracted 2D textures are then modeled using Markov-Gibbs random field (MGRF) modeling technique. The modeling results for MGRF model depend on input texture characteristics. The homogeneous, spatially invariant texture patterns are modeled successfully. From the observation of skin samples, we classify three key features of3D skin profiles i.e. curvature of underlying limb, wrinkles/line like features and fine textures. The skin samples are distributed in three input sets to see the MGRF model's response to each of these 3D features. First set consists of all three features. Second set is obtained after elimination of curvature and contains both wrinkle/line like features and fine textures. Third set is also obtained after elimination of curvature but consists of fine textures only. MGRF modeling for set I did not result in any visual similarity. Hence the curvature of underlying limbs cannot be modeled successfully and makes an inhomogeneous feature. For set 2 the wrinkle/line like features can be modeled with low/medium visual similarity depending on the spatial invariance. The results for set 3 show that fine textures of skin are almost always modeled successfully with medium/high visual similarity and make a homogeneous feature. We conclude that the MGRF model is able to model fine textures of skin successfully which are on scale of~ 0.1 mm. The surface profiles on this resolution can provide haptic sensation of roughness and friction. Therefore fine textures can be an important clue to different skin conditions perceived through tactile inspection via a haptic device

Appearance Modeling of Living Human Tissues

This is the peer reviewed version of the following article: Nunes, A.L.P., Maciel, A., Meyer, G.W., John, N.W., Baranoski, G.V.G., & Walter, M. (2019). Appearance Modeling of Living Human Tissues, Computer Graphics Forum, which has been published in final form at https://doi.org/10.1111/cgf.13604. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-ArchivingThe visual fidelity of realistic renderings in Computer Graphics depends fundamentally upon how we model the appearance of objects resulting from the interaction between light and matter reaching the eye. In this paper, we survey the research addressing appearance modeling of living human tissue. Among the many classes of natural materials already researched in Computer Graphics, living human tissues such as blood and skin have recently seen an increase in attention from graphics research. There is already an incipient but substantial body of literature on this topic, but we also lack a structured review as presented here. We introduce a classification for the approaches using the four types of human tissues as classifiers. We show a growing trend of solutions that use first principles from Physics and Biology as fundamental knowledge upon which the models are built. The organic quality of visual results provided by these Biophysical approaches is mainly determined by the optical properties of biophysical components interacting with light. Beyond just picture making, these models can be used in predictive simulations, with the potential for impact in many other areas

Ex-vivo and In-vivo Characterization of Human Accommodation

A completely satisfying approach to restoring accommodation still needs to be developed. Besides, there are considerable discrepancies between objective and subjective trials to evaluate the therapeutic success. A substantial biomechanical understanding of all structures and processes involved in accommodation as well as presbyopia are needed to develop promising new strategies. This contribution focuses on developing advanced imaging techniques to create a basic understanding of accommodation and presbyopia and to evaluate existing concepts for restoring accommodation. Besides, the emphasis is also on replacing stiff presbyopic lenses by a material that imitates the young crystalline lens

Constitutive modelling of skin ageing

The objective of this chapter is to review the main biomechanical and structural aspects associated with both intrinsic and extrinsic skin ageing, and to present potential research avenues to account for these effects in mathematical and computational models of the skin. This will be illustrated through recent work of the authors which provides a basis to those interested in developing mechanistic constitutive models capturing the mechanobiology of skin across the life course

Advances in Ophthalmology

This book focuses on the different aspects of ophthalmology - the medical science of diagnosis and treatment of eye disorders. Ophthalmology is divided into various clinical subspecialties, such as cornea, cataract, glaucoma, uveitis, retina, neuro-ophthalmology, pediatric ophthalmology, oncology, pathology, and oculoplastics. This book incorporates new developments as well as future perspectives in ophthalmology and is a balanced product between covering a wide range of diseases and expedited publication. It is intended to be the appetizer for other books to follow. Ophthalmologists, researchers, specialists, trainees, and general practitioners with an interest in ophthalmology will find this book interesting and useful

Visual Tactile Integration in Rats and Underlying Neuronal Mechanisms

Our experience of the world depends on integration of cues from multiple senses to form unified percepts. How the brain merges information across sensory modalities has been the object of debate. To measure how rats bring together information across sensory modalities, we devised an orientation categorization task that combines vision and touch. Rats encounter an object\u2013comprised of alternating black and white raised bars\u2013that looks and feels like a grating and can be explored by vision (V), touch (T), or both (VT). The grating is rotated to assume one orientation on each trial, spanning a range of 180 degrees. Rats learn to lick one spout for orientations of 0\ub145 degrees (\u201chorizontal\u201d) and the opposite spout for orientations of 90\ub145\ub0 (\u201cvertical\u201d). Though training was in VT condition, rats could recognize the object and apply the rules of the task on first exposure to V and to T conditions. This suggests that the multimodal percept corresponds to that of the single modalities. Quantifying their performance, we found that rats have good orientation acuity using their whiskers and snout (T condition); however under our default conditions, typically performance is superior by vision (V condition). Illumination could be adjusted to render V and T performance equivalent. Independently of whether V and T performance is made equivalent, performance is always highest in the VT condition, indicating multisensory enhancement. Is the enhancement optimal with respect to the best linear combination? To answer this, we computed the performance expected by optimal integration in the framework of Bayesian decision theory and found that most rats combine visual and tactile information better than predicted by the standard ideal\u2013observer model. To confirm these results, we interpreted the data in two additional frameworks: Summation of mutual information for each sensory channel and probabilities of independent events. All three analyses agree that rats combine vision and touch better than could be accounted for by a linear interaction. Electrophysiological recordings in the posterior parietal cortex (PPC) of behaving rats revealed that neuronal activity is modulated by decision of the rats as well as by categorical or graded modality-shared representations of the stimulus orientation. Because the population of PPC neurons expresses activity ranging from strongly stimulus-related (e.g. graded in relation to stimulus orientation) to strongly choice-related (e.g. modulated by stimulus category but not by orientation within a category) we suggest that this region is involved in the percept-to-choice transformation

Optical Methods in Sensing and Imaging for Medical and Biological Applications

The recent advances in optical sources and detectors have opened up new opportunities for sensing and imaging techniques which can be successfully used in biomedical and healthcare applications. This book, entitled ‘Optical Methods in Sensing and Imaging for Medical and Biological Applications’, focuses on various aspects of the research and development related to these areas. The book will be a valuable source of information presenting the recent advances in optical methods and novel techniques, as well as their applications in the fields of biomedicine and healthcare, to anyone interested in this subject

Multimodal imaging in age-related macular degeneration

Age-related macular degeneration (AMD) is a leading cause of blindness and affects approximately one in seven Australians aged 50 years and above. Currently, this complex condition is not easily and uniformly assessed. The signs of AMD differ between eyes and also occur in other macular disorders. This hinders accurate diagnosis and classification, which is fundamental to optimal patient care. Ocular imaging and visual function assessment have the potential to minimise the devastating consequences of disease through early detection. However, multiple devices are now commercially available and the impact of these technologies in clinical practice may not be straightforward. For instance, their usefulness may depend on accessibility and the operator’s knowledge and clinical skills. The impact on patient management, as well as alternative models of eye-care delivery, requires clarification. This thesis aims to explore the current and potential utility of imaging technologies (optical coherence tomography, infrared imaging, monochromatic retinal photography and fundus autofluorescence) in the assessment and management of AMD and other diseases of retinal pigment epithelium dysfunction. The findings show that optometrists self-describe high levels of practice competency and make ready use of imaging in everyday practice. However, they also unwittingly demonstrated low awareness of the evidence base in AMD. Furthermore, when their interpretation of images was tested using a series of case vignettes, their diagnostic accuracy as a group improved by only five per cent (from 61 per cent to 66 per cent); their tendency to refer increased by four per cent. These factors might be improved through education. A series of open-access, chair-side reference charts were consequently devised to help optometrists use imaging technologies more effectively in clinical practice. The additive contribution of multimodal structural and functional testing was particularly emphasised. Finally, a novel model of intermediate-tier eye-care in Australia was shown to substantially reduce the number of false positive cases or cases without a specific diagnosis. Interestingly, this model was acclaimed by reviewers as “scoring highly for originality and of international relevance”. Most excitingly, the thesis concludes with future directions regarding collaborative care and multimodal imaging, where detection of disease might be facilitated via a computational approach

Towards tactile sensing active capsule endoscopy

Examination of the gastrointestinal(GI) tract has traditionally been performed using tethered endoscopy tools with limited reach and more recently with passive untethered capsule endoscopy with limited capability. Inspection of small intestines is only possible using the latter capsule endoscopy with on board camera system. Limited to visual means it cannot detect features beneath the lumen wall if they have not affected the lumen structure or colour. This work presents an improved capsule endoscopy system with locomotion for active exploration of the small intestines and tactile sensing to detect deformation of the capsule outer surface when it follows the intestinal wall. In laboratory conditions this system is capable of identifying sub-lumen features such as submucosal tumours.Through an extensive literary review the current state of GI tract inspection in particular using remote operated miniature robotics, was investigated, concluding no solution currently exists that utilises tactile sensing with a capsule endoscopy. In order to achieve such a platform, further investigation was made in to tactile sensing technologies, methods of locomotion through the gut, and methods to support an increased power requirement for additional electronics and actuation. A set of detailed criteria were compiled for a soft formed sensor and flexible bodied locomotion system. The sensing system is built on the biomimetic tactile sensing device, Tactip, \cite{Chorley2008, Chorley2010, Winstone2012, Winstone2013} which has been redesigned to fit the form of a capsule endoscopy. These modifications have required a $360^{o}$ cylindrical sensing surface with $360^{o}$ panoramic optical system. Multi-material 3D printing has been used to build an almost complete sensor assembly with a combination of hard and soft materials, presenting a soft compliant tactile sensing system that mimics the tactile sensing methods of the human finger. The cylindrical Tactip has been validated using artificial submucosal tumours in laboratory conditions. The first experiment has explored the new form factor and measured the device's ability to detect surface deformation when travelling through a pipe like structure with varying lump obstructions. Sensor data was analysed and used to reconstruct the test environment as a 3D rendered structure. A second tactile sensing experiment has explored the use of classifier algorithms to successfully discriminate between three tumour characteristics; shape, size and material hardness. Locomotion of the capsule endoscopy has explored further bio-inspiration from earthworm's peristaltic locomotion, which share operating environment similarities. A soft bodied peristaltic worm robot has been developed that uses a tuned planetary gearbox mechanism to displace tendons that contract each worm segment. Methods have been identified to optimise the gearbox parameter to a pipe like structure of a given diameter. The locomotion system has been tested within a laboratory constructed pipe environment, showing that using only one actuator, three independent worm segments can be controlled. This configuration achieves comparable locomotion capabilities to that of an identical robot with an actuator dedicated to each individual worm segment. This system can be miniaturised more easily due to reduced parts and number of actuators, and so is more suitable for capsule endoscopy. Finally, these two developments have been integrated to demonstrate successful simultaneous locomotion and sensing to detect an artificial submucosal tumour embedded within the test environment. The addition of both tactile sensing and locomotion have created a need for additional power beyond what is available from current battery technology. Early stage work has reviewed wireless power transfer (WPT) as a potential solution to this problem. Methods for optimisation and miniaturisation to implement WPT on a capsule endoscopy have been identified with a laboratory built system that validates the methods found. Future work would see this combined with a miniaturised development of the robot presented. This thesis has developed a novel method for sub-lumen examination. With further efforts to miniaturise the robot it could provide a comfortable and non-invasive procedure to GI tract inspection reducing the need for surgical procedures and accessibility for earlier stage of examination. Furthermore, these developments have applicability in other domains such as veterinary medicine, industrial pipe inspection and exploration of hazardous environments