Object Tracking and Mensuration in Surveillance Videos

This thesis focuses on tracking and mensuration in surveillance videos. The first part of the thesis discusses several object tracking approaches based on the different properties of tracking targets. For airborne videos, where the targets are usually small and with low resolutions, an approach of building motion models for foreground/background proposed in which the foreground target is simplified as a rigid object. For relatively high resolution targets, the non-rigid models are applied. An active contour-based algorithm has been introduced. The algorithm is based on decomposing the tracking into three parts: estimate the affine transform parameters between successive frames using particle filters; detect the contour deformation using a probabilistic deformation map, and regulate the deformation by projecting the updated model onto a trained shape subspace. The active appearance Markov chain (AAMC). It integrates a statistical model of shape, appearance and motion. In the AAMC model, a Markov chain represents the switching of motion phases (poses), and several pairwise active appearance model (P-AAM) components characterize the shape, appearance and motion information for different motion phases. The second part of the thesis covers video mensuration, in which we have proposed a heightmeasuring algorithm with less human supervision, more flexibility and improved robustness. From videos acquired by an uncalibrated stationary camera, we first recover the vanishing line and the vertical point of the scene. We then apply a single view mensuration algorithm to each of the frames to obtain height measurements. Finally, using the LMedS as the cost function and the Robbins-Monro stochastic approximation (RMSA) technique to obtain the optimal estimate

Image based approach for early assessment of heart failure.

In diagnosing heart diseases, the estimation of cardiac performance indices requires accurate segmentation of the left ventricle (LV) wall from cine cardiac magnetic resonance (CMR) images. MR imaging is noninvasive and generates clear images; however, it is impractical to manually process the huge number of images generated to calculate the performance indices. In this dissertation, we introduce a novel, fast, robust, bi-directional coupled parametric deformable models that are capable of segmenting the LV wall borders using first- and second-order visual appearance features. These features are embedded in a new stochastic external force that preserves the topology of the LV wall to track the evolution of the parametric deformable models control points. We tested the proposed segmentation approach on 15 data sets in 6 infarction patients using the Dice similarity coefficient (DSC) and the average distance (AD) between the ground truth and automated segmentation contours. Our approach achieves a mean DSC value of 0.926±0.022 and mean AD value of 2.16±0.60 mm compared to two other level set methods that achieve mean DSC values of 0.904±0.033 and 0.885±0.02; and mean AD values of 2.86±1.35 mm and 5.72±4.70 mm, respectively. Also, a novel framework for assessing both 3D functional strain and wall thickening from 4D cine cardiac magnetic resonance imaging (CCMR) is introduced. The introduced approach is primarily based on using geometrical features to track the LV wall during the cardiac cycle. The 4D tracking approach consists of the following two main steps: (i) Initially, the surface points on the LV wall are tracked by solving a 3D Laplace equation between two subsequent LV surfaces; and (ii) Secondly, the locations of the tracked LV surface points are iteratively adjusted through an energy minimization cost function using a generalized Gauss-Markov random field (GGMRF) image model in order to remove inconsistencies and preserve the anatomy of the heart wall during the tracking process. Then the circumferential strains are straight forward calculated from the location of the tracked LV surface points. In addition, myocardial wall thickening is estimated by co-allocation of the corresponding points, or matches between the endocardium and epicardium surfaces of the LV wall using the solution of the 3D laplace equation. Experimental results on in vivo data confirm the accuracy and robustness of our method. Moreover, the comparison results demonstrate that our approach outperforms 2D wall thickening estimation approaches

Fast catheter segmentation and tracking based on x-ray fluoroscopic and echocardiographic modalities for catheter-based cardiac minimally invasive interventions

X-ray fluoroscopy and echocardiography imaging (ultrasound, US) are two imaging modalities that are widely used in cardiac catheterization. For these modalities, a fast, accurate and stable algorithm for the detection and tracking of catheters is required to allow clinicians to observe the catheter location in real-time. Currently X-ray fluoroscopy is routinely used as the standard modality in catheter ablation interventions. However, it lacks the ability to visualize soft tissue and uses harmful radiation. US does not have these limitations but often contains acoustic artifacts and has a small field of view. These make the detection and tracking of the catheter in US very challenging. The first contribution in this thesis is a framework which combines Kalman filter and discrete optimization for multiple catheter segmentation and tracking in X-ray images. Kalman filter is used to identify the whole catheter from a single point detected on the catheter in the first frame of a sequence of x-ray images. An energy-based formulation is developed that can be used to track the catheters in the following frames. We also propose a discrete optimization for minimizing the energy function in each frame of the X-ray image sequence. Our approach is robust to tangential motion of the catheter and combines the tubular and salient feature measurements into a single robust and efficient framework. The second contribution is an algorithm for catheter extraction in 3D ultrasound images based on (a) the registration between the X-ray and ultrasound images and (b) the segmentation of the catheter in X-ray images. The search space for the catheter extraction in the ultrasound images is constrained to lie on or close to a curved surface in the ultrasound volume. The curved surface corresponds to the back-projection of the extracted catheter from the X-ray image to the ultrasound volume. Blob-like features are detected in the US images and organized in a graphical model. The extracted catheter is modelled as the optimal path in this graphical model. Both contributions allow the use of ultrasound imaging for the improved visualization of soft tissue. However, X-ray imaging is still required for each ultrasound frame and the amount of X-ray exposure has not been reduced. The final contribution in this thesis is a system that can track the catheter in ultrasound volumes automatically without the need for X-ray imaging during the tracking. Instead X-ray imaging is only required for the system initialization and for recovery from tracking failures. This allows a significant reduction in the amount of X-ray exposure for patient and clinicians.Open Acces

Ultrafast Ultrasound Imaging

Among medical imaging modalities, such as computed tomography (CT) and magnetic resonance imaging (MRI), ultrasound imaging stands out due to its temporal resolution. Owing to the nature of medical ultrasound imaging, it has been used for not only observation of the morphology of living organs but also functional imaging, such as blood flow imaging and evaluation of the cardiac function. Ultrafast ultrasound imaging, which has recently become widely available, significantly increases the opportunities for medical functional imaging. Ultrafast ultrasound imaging typically enables imaging frame-rates of up to ten thousand frames per second (fps). Due to the extremely high temporal resolution, this enables visualization of rapid dynamic responses of biological tissues, which cannot be observed and analyzed by conventional ultrasound imaging. This Special Issue includes various studies of improvements to the performance of ultrafast ultrasoun

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

Foetal echocardiographic segmentation

Congenital heart disease affects just under one percentage of all live births [1]. Those defects that manifest themselves as changes to the cardiac chamber volumes are the motivation for the research presented in this thesis. Blood volume measurements in vivo require delineation of the cardiac chambers and manual tracing of foetal cardiac chambers is very time consuming and operator dependent. This thesis presents a multi region based level set snake deformable model applied in both 2D and 3D which can automatically adapt to some extent towards ultrasound noise such as attenuation, speckle and partial occlusion artefacts. The algorithm presented is named Mumford Shah Sarti Collision Detection (MSSCD). The level set methods presented in this thesis have an optional shape prior term for constraining the segmentation by a template registered to the image in the presence of shadowing and heavy noise. When applied to real data in the absence of the template the MSSCD algorithm is initialised from seed primitives placed at the centre of each cardiac chamber. The voxel statistics inside the chamber is determined before evolution. The MSSCD stops at open boundaries between two chambers as the two approaching level set fronts meet. This has significance when determining volumes for all cardiac compartments since cardiac indices assume that each chamber is treated in isolation. Comparison of the segmentation results from the implemented snakes including a previous level set method in the foetal cardiac literature show that in both 2D and 3D on both real and synthetic data, the MSSCD formulation is better suited to these types of data. All the algorithms tested in this thesis are within 2mm error to manually traced segmentation of the foetal cardiac datasets. This corresponds to less than 10% of the length of a foetal heart. In addition to comparison with manual tracings all the amorphous deformable model segmentations in this thesis are validated using a physical phantom. The volume estimation of the phantom by the MSSCD segmentation is to within 13% of the physically determined volume

Combining Shape and Learning for Medical Image Analysis

Automatic methods with the ability to make accurate, fast and robust assessments of medical images are highly requested in medical research and clinical care. Excellent automatic algorithms are characterized by speed, allowing for scalability, and an accuracy comparable to an expert radiologist. They should produce morphologically and physiologically plausible results while generalizing well to unseen and rare anatomies. Still, there are few, if any, applications where today\u27s automatic methods succeed to meet these requirements.\ua0The focus of this thesis is two tasks essential for enabling automatic medical image assessment, medical image segmentation and medical image registration. Medical image registration, i.e. aligning two separate medical images, is used as an important sub-routine in many image analysis tools as well as in image fusion, disease progress tracking and population statistics. Medical image segmentation, i.e. delineating anatomically or physiologically meaningful boundaries, is used for both diagnostic and visualization purposes in a wide range of applications, e.g. in computer-aided diagnosis and surgery.The thesis comprises five papers addressing medical image registration and/or segmentation for a diverse set of applications and modalities, i.e. pericardium segmentation in cardiac CTA, brain region parcellation in MRI, multi-organ segmentation in CT, heart ventricle segmentation in cardiac ultrasound and tau PET registration. The five papers propose competitive registration and segmentation methods enabled by machine learning techniques, e.g. random decision forests and convolutional neural networks, as well as by shape modelling, e.g. multi-atlas segmentation and conditional random fields

Cardiac motion and deformation estimation in tagged magnetic resonance imaging

Dissertação de mestrado integrado em Engenharia Biomédica (área de especialização em Electrónica Médica)Cardiovascular diseases are the main cause of death in Europe, with an estimate of 4.3 million deaths each year. The assessment of the regional wall deformation is a relevant clinical indicator, and can be used to detect several cardiac lesions. Nowadays, this study can be performed using several image modalities. In the current thesis, we focus on tagged Magnetic Resonance imaging (t-MRI) technique. Such technique allows acquiring images with tags on the myocardium, which deform with the muscle. The present thesis intends to assess the left ventricle (LV) deformation using radial and circumferential strain. To compute such strain values, both endo- and epicardial contours of the LV are required. As such, a new framework to automatically assess the LV function is proposed. This framework presents: (i) an automatic segmentation technique, based on a tag suppression strategy followed by an active contour segmentation method, and (ii) a tracking approach to extract myocardial deformation, based on a non-rigid registration method. The automatic segmentation uses the B-spline Explicit Active Surface framework, which was previously applied in ultra-sound and cine-MRI images. In both cases, a real-time and accurate contour was achieved. Regarding the registration step, starting from a state-of-art approach, termed sequential 2D, we suggest a new method (termed sequential 2D+t), where the temporal information is included on the model. The tracking methods were first tested on synthetic data to study the registration parameters influence. Furthermore, the proposed and original methods were applied on porcine data with myocardial ischemia. Both methods were able to detect dysfunctional regions. A comparison between the strain curve in the sequential 2D and sequential 2D+t strategies was also shown. As conclusion, a smoothing effect in the strain curve was detected in the sequential 2D+t strategy. The validation of the segmentation approach uses a human dataset. A comparison between the manual contour and the proposed segmentation method results was performed. The results, suggest that proposed method has an acceptable performance, removing the tedious task related with manual segmentation and the intra-observer variability. Finally, a comparison between the proposed framework and the currently available commercial software was performed. The commercial software results were obtained from core-lab analysis. An acceptable result (r = 0.601) was achieved when comparing the strain peak values. Importantly, the proposed framework appears to present a more acceptable result.As doenças cardiovasculares são a principal causa de morte na Europa, com aproximadamente 4.7 milhões de mortes por ano. A avaliação da deformação do miocárdio a um nível local é um importante indicador clínico e pode ser usado para a deteção de lesões cardíacas. Este estudo é normalmente realizado usando várias modalidades de imagem médica. Nesta tese, a Resonância Magnética (RM) marcada foi a técnica selecionada. Estas imagens têm marcadores no músculo cardíaco, os quais se deformam com o miocárdio e podem ser usados para o estudo da deformação cardíaca. Nesta tese, pretende-se estudar a deformação radial e circunferencial do ventrículo esquerdo (VE). Assim, um contorno do endo- e epicárdio no VE é essencial. Desta forma, uma ferramenta para o estudo da deformação do VE foi desenvolvida. Esta possui: (i) um método de segmentação automático, usando uma estratégia de supressão dos marcadores, seguido de uma segmentação c um contorno ativo, e (ii) um método de tracking para determinação da deformação cardíaca, baseado em registo não rígido. A segmentação automática utiliza a ferramenta B-spline Explicit Active Surface, que foi previamente aplicada em imagens de ultrassons e cine-RM. Em ambos os casos, uma segmentação em tempo real e com elevada exatidão foi alcançada. Vários esquemas de registo foram apresentados. Neste ponto, começando com uma técnica do estado da arte (designada de sequencial 2D), uma nova metodologia foi proposta (sequencial 2D+t), onde a informação temporal é incorporada no modelo. De forma a analisar a influência dos parâmetros do registo, estes foram estudados num dataset sintético. De seguida, os diferentes esquemas de registo foram testados num dataset suíno com isquemia. Ambos os métodos foram capazes de detetar as regiões disfuncionais. De igual forma, utilizando as curvas de deformação obtidas para cada um dos métodos propostos, foi possível observar uma suavização na direção temporal para o método sequencial 2D+t. Relativamente à segmentação, esta foi validada com um dataset humano. Um contorno manual foi comparado com o obtido pelo método proposto. Os resultados sugerem que a nova estratégia é aceitável, sendo mais rápida do que a realização de um contorno manual e eliminando a variabilidade entre observadores. Por fim, realizou-se uma comparação entre a ferramenta proposta e um software comercial (com análise de core-lab). A comparação entre os valores de pico da deformação exibe uma correlação plausível (r=0.601). Contudo, é importante notar, que a nova ferramenta tende a apresentar um resultado mais aceitável