Tissue surface shape and reflectance spectra provide rich intra-operative
information useful in surgical guidance. We propose a hybrid system which
displays an endoscopic image with a fast joint inspection of tissue surface
shape using structured light (SL) and hyperspectral imaging (HSI). For SL a
miniature fibre probe is used to project a coloured spot pattern onto the
tissue surface. In HSI mode standard endoscopic illumination is used, with the
fibre probe collecting reflected light and encoding the spatial information
into a linear format that can be imaged onto the slit of a spectrograph.
Correspondence between the arrangement of fibres at the distal and proximal
ends of the bundle was found using spectral encoding. Then during pattern
decoding, a fully convolutional network (FCN) was used for spot detection,
followed by a matching propagation algorithm for spot identification. This
method enabled fast reconstruction (12 frames per second) using a GPU. The
hyperspectral image was combined with the white light image and the
reconstructed surface, showing the spectral information of different areas.
Validation of this system using phantom and ex vivo experiments has been
demonstrated.Comment: This paper has been submitted to MICCAI2016 on 17 March, 2016, and
conditionally accepted on 2 June, 201
