Stereoscopic Medical Data Video Quality Issues

Stereoscopic medical videos are recorded, e.g., in stereo endoscopy or during video recording medical/dental operations. This paper examines quality issues in the recorded stereoscopic medical videos, as insufficient quality may induce visual fatigue to doctors. No attention has been paid to stereo quality and ensuing fatigue issues in the scientific literature so far. Two of the most commonly encountered quality issues in stereoscopic data, namely stereoscopic window violations and bent windows, were searched for in stereo endoscopic medical videos. Furthermore, an additional stereo quality issue encountered in dental operation videos, namely excessive disparity, was detected and fixed. The conducted experiments prove the existence of such quality issues in stereoscopic medical data and highlight the need for their detection and correction

Multimodal Stereoscopic Movie Summarization Conforming to Narrative Characteristics

Video summarization is a timely and rapidly developing research field with broad commercial interest, due to the increasing availability of massive video data. Relevant algorithms face the challenge of needing to achieve a careful balance between summary compactness, enjoyability, and content coverage. The specific case of stereoscopic 3D theatrical films has become more important over the past years, but not received corresponding research attention. In this paper, a multi-stage, multimodal summarization process for such stereoscopic movies is proposed, that is able to extract a short, representative video skim conforming to narrative characteristics from a 3D film. At the initial stage, a novel, low-level video frame description method is introduced (frame moments descriptor) that compactly captures informative image statistics from luminance, color, optical flow, and stereoscopic disparity video data, both in a global and in a local scale. Thus, scene texture, illumination, motion, and geometry properties may succinctly be contained within a single frame feature descriptor, which can subsequently be employed as a building block in any key-frame extraction scheme, e.g., for intra-shot frame clustering. The computed key-frames are then used to construct a movie summary in the form of a video skim, which is post-processed in a manner that also considers the audio modality. The next stage of the proposed summarization pipeline essentially performs shot pruning, controlled by a user-provided shot retention parameter, that removes segments from the skim based on the narrative prominence of movie characters in both the visual and the audio modalities. This novel process (multimodal shot pruning) is algebraically modeled as a multimodal matrix column subset selection problem, which is solved using an evolutionary computing approach. Subsequently, disorienting editing effects induced by summarization are dealt with, through manipulation of the video skim. At the last step, the skim is suitably post-processed in order to reduce stereoscopic video defects that may cause visual fatigue

Human-centered 2D/3D Video Content Analysis and Description

In this paper, we propose a way of using the AudioVisual Description Profile (AVDP) of the MPEG-7 standard for stereo video and multichannel audio content description. Our aim is to provide means of using AVDP in such a way, that 3D video and audio content can be correctly and consistently described. Since AVDP semantics do not include ways for dealing with 3D audiovisual content, a new semantic framework within AVDP is proposed and examples of using AVDP to describe the results of analysis algorithms on stereo video and multichannel audio content are presented

Vision-based pavement marking detection and condition assessment : a case study

Pavement markings constitute an effective way of conveying regulations and guidance to drivers. They constitute the most fundamental way to communicate with road users, thus, greatly contributing to ensuring safety and order on roads. However, due to the increasingly extensive traffic demand, pavement markings are subject to a series of deterioration issues (e.g., wear and tear). Markings in poor condition typically manifest as being blurred or even missing in certain places. The need for proper maintenance strategies on roadway markings, such as repainting, can only be determined based on a comprehensive understanding of their as-is worn condition. Given the fact that an efficient, automated and accurate approach to collect such condition information is lacking in practice, this study proposes a vision-based framework for pavement marking detection and condition assessment. A hybrid feature detector and a threshold-based method were used for line marking identification and classification. For each identified line marking, its worn/blurred severity level was then quantified in terms of worn percentage at a pixel level. The damage estimation results were compared to manual measurements for evaluation, indicating that the proposed method is capable of providing indicative knowledge about the as-is condition of pavement markings. This paper demonstrates the promising potential of computer vision in the infrastructure sector, in terms of implementing a wider range of managerial operations for roadway management

Creation of a Virtual Atlas of Neuroanatomy and Neurosurgical Techniques Using 3D Scanning Techniques

Neuroanatomy is one of the most challenging and fascinating topics within the human anatomy, due to the complexity and interconnection of the entire nervous system. The gold standard for learning neurosurgical anatomy is cadaveric dissections. Nevertheless, it has a high cost (needs of a laboratory, acquisition of cadavers, and fixation), is time-consuming, and is limited by sociocultural restrictions. Due to these disadvantages, other tools have been investigated to improve neuroanatomy learning. Three-dimensional modalities have gradually begun to supplement traditional 2-dimensional representations of dissections and illustrations. Volumetric models (VM) are the new frontier for neurosurgical education and training. Different workflows have been described to create these VMs -photogrammetry (PGM) and structured light scanning (SLS). In this study, we aimed to describe and use the currently available 3D scanning techniques to create a virtual atlas of neurosurgical anatomy. Dissections on post-mortem human heads and brains were performed at the skull base laboratories of Stanford University - NeuroTraIn Center and the University of California, San Francisco - SBCVL (skull base and cerebrovascular laboratory). Then VMs were created following either SLS or PGM workflow. Fiber tract reconstructions were also generated from DICOM using DSI-studio and incorporated into VMs from dissections. Moreover, common creative license materials models were used to simplify the understanding of the specific anatomical region. Both methods yielded VMs with suitable clarity and structural integrity for anatomical education, surgical illustration, and procedural simulation. We described the roadmap of SLS and PGM for creating volumetric models, including the required equipment and software. We have also provided step-by-step procedures on how users can post-processing and refine these images according to their specifications. The VMs generated were used for several publications, to describe the step-by-step of a specific neurosurgical approach and to enhance the understanding of an anatomical region and its function. These models were used in neuroanatomical education and research (workshops and publications). VMs offer a new, immersive, and innovative way to accurately visualize neuroanatomy. Given the straightforward workflow, the presently described techniques may serve as a reference point for an entirely new way of capturing and depicting neuroanatomy and offer new opportunities for the application of VMs in education, simulation, and surgical planning. The virtual atlas, divided into specific areas concerning different neurosurgical approaches (such as skull base, cortex and fiber tracts, and spine operative anatomy), will increase the viewer's understanding of neurosurgical anatomy. The described atlas is the first surgical collection of VMs from cadaveric dissections available in the medical field and could be a used as reference for future creation of analogous collection in the different medical subspeciality.La neuroanatomia è, grazie alle intricate connessioni che caratterizzano il sistema nervoso e alla sua affascinante complessità, una delle discipline più stimolanti della anatomia umana. Nonostante il gold standard per l’apprendimento dell’anatomia neurochirurgica sia ancora rappresentato dalle dissezioni cadaveriche, l’accessibilità a queste ultime rimane limitata, a causa della loro dispendiosità in termini di tempo e costi (necessità di un laboratorio, acquisizione di cadaveri e fissazione), e alle restrizioni socioculturali per la donazione di cadaveri. Al fine di far fronte a questi impedimenti, e con lo scopo di garantire su larga scala l’apprendimento tridimensionale della neuroanatomia, nel corso degli anni sono stati sviluppati nuovi strumenti e tecnologie. Le tradizionali rappresentazioni anatomiche bidimensionali sono state gradualmente sostituite dalle modalità 3-dimensionali (3D) – foto e video. Tra questi ultimi, i modelli volumetrici (VM) rappresentano la nuova frontiera per l'istruzione e la formazione neurochirurgica. Diversi metodi per creare questi VM sono stati descritti, tra cui la fotogrammetria (PGM) e la scansione a luce strutturata (SLS). Questo studio descrive l’utilizzo delle diverse tecniche di scansione 3D grazie alle quali è stato creato un atlante virtuale di anatomia neurochirurgica. Le dissezioni su teste e cervelli post-mortem sono state eseguite presso i laboratori di base cranica di Stanford University -NeuroTraIn Center e dell'Università della California, San Francisco - SBCVL. I VM dalle dissezioni sono stati creati seguendo i metodi di SLS e/o PGM. Modelli di fibra bianca sono stati generate utilizzando DICOM con il software DSI-studio e incorporati ai VM di dissezioni anatomiche. Inoltre, sono stati utilizzati VM tratti da common creative license material (materiale con licenze creative comuni) al fine di semplificare la comprensione di alcune regioni anatomiche. I VM generati con entrambi i metodi sono risultati adeguati, sia in termini di chiarezza che di integrità strutturale, per l’educazione anatomica, l’illustrazione medica e la simulazione chirurgica. Nel nostro lavoro sono stati esaustivamente descritti tutti gli step necessari, di entrambe le tecniche (SLS e PGM), per la creazione di VM, compresi le apparecchiature e i software utilizzati. Sono state inoltre descritte le tecniche di post-elaborazione e perfezionamento dei VM da poter utilizzare in base alle necessità richieste. I VM generati durante la realizzazione del nostro lavoro sono stati utilizzati per molteplici pubblicazioni, nella descrizione step-by-step di uno specifico approccio neurochirurgico o per migliorare la comprensione di una regione anatomica e della sua funzione. Questi modelli sono stati utilizzati a scopo didattico per la formazione neuroanatomica di studenti di medicina, specializzandi e giovani neurochirurghi. I VM offrono un modo nuovo, coinvolgente e innovativo con cui poter raggiungere un’accurata conoscenza tridimensionale della neuroanatomia. La metodologia delle due tecniche descritte può servire come punto di riferimento per un nuovo modo di acquisizione e rappresentazione della neuroanatomia, ed offrire nuove opportunità di utilizzo dei VM nella formazione didattica, nella simulazione e nella pianificazione chirurgica. L'atlante virtuale qui descritto, suddiviso in aree specifiche relative a diversi approcci neurochirurgici, aumenterà la comprensione dell'anatomia neurochirurgica da parte dello spettatore. Questa è la prima raccolta chirurgica di VM da dissezioni anatomiche disponibile in ambito medico e potrebbe essere utilizzato come riferimento per la futura creazione di analoga raccolta nelle diverse sotto specialità mediche

Advanced Endoscopic Navigation:Surgical Big Data,Methodology,and Applications

随着科学技术的飞速发展，健康与环境问题日益成为人类面临的最重大问题之一。信息科学、计算机技术、电子工程与生物医学工程等学科的综合应用交叉前沿课题，研究现代工程技术方法，探索肿瘤癌症等疾病早期诊断、治疗和康复手段。本论文综述了计算机辅助微创外科手术导航、多模态医疗大数据、方法论及其临床应用：从引入微创外科手术导航概念出发，介绍了医疗大数据的术前与术中多模态医学成像方法、阐述了先进微创外科手术导航的核心流程包括计算解剖模型、术中实时导航方案、三维可视化方法及交互式软件技术，归纳了各类微创外科手术方法的临床应用。同时，重点讨论了全球各种手术导航技术在临床应用中的优缺点，分析了目前手术导航领域内的最新技术方法。在此基础上，提出了微创外科手术方法正向数字化、个性化、精准化、诊疗一体化、机器人化以及高度智能化的发展趋势。【Abstract】Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.X.L. acknowledges funding from the Fundamental Research Funds for the Central Universities. T.M.P. acknowledges funding from the Canadian Foundation for Innovation, the Canadian Institutes for Health Research, the National Sciences and Engineering Research Council of Canada, and a grant from Intuitive Surgical Inc