Spatiotemporal Barcodes for Image Sequence Analysis

Taking as input a time-varying sequence of two-dimensional (2D) binary images, we develop an algorithm for computing a spatiotemporal 0–barcode encoding lifetime of connected components on the image sequence over time. This information may not coincide with the one provided by the 0–barcode encoding the 0–persistent homology, since the latter does not respect the principle that it is not possible to move backwards in time. A cell complex K is computed from the given sequence, being the cells of K classified as spatial or temporal depending on whether they connect two consecutive frames or not. A spatiotemporal path is defined as a sequence of edges of K forming a path such that two edges of the path cannot connect the same two consecutive frames. In our algorithm, for each vertex v ∈ K, a spatiotemporal path from v to the “oldest” spatiotemporally-connected vertex is computed and the corresponding spatiotemporal 0–bar is added to the spatiotemporal 0–barcode.Junta de Andalucía FQM-369Ministerio de Economía y Competitividad MTM2012-3270

Image sequence analysis and merging

Peer ReviewedPostprint (published version

Online parameter estimation in dynamic Markov Random Fields for image sequence analysis

pre-printMarkov Random Fields (MRF) have proven to be extremely useful models for efficient and accurate image segmentation.Recent literature points to an increased effort towards incorporating useful priors (shape, geometry, context) in a MRF framework. However, topological priors, considered extremely crucial in biological and natural image sequences have been less explored. This work proposes a strategy wherein free parameters of the MRF are used to make it topology aware using a semantic graphical model working in conjunction with the MRF. Estimation of free parameters is constrained by prior knowledge of an object's topological dynamics encoded by the graphical model. Maximizing a regional conformance measure yields parameters for the frame under consideration. The application motivating this work is the tracing of neuronal structures across 3D serial section Transmission Electron Micrograph (ssTEM) stacks. Applicability of the proposed method is demonstrated by tracing 3D structures in ssTEM stacks

Monitoring MBE substrate deoxidation via RHEED image-sequence analysis by deep learning

Reflection high-energy electron diffraction (RHEED) is a powerful tool in molecular beam epitaxy (MBE), but RHEED images are often difficult to interpret, requiring experienced operators. We present an approach for automated surveillance of GaAs substrate deoxidation in MBE using deep learning based RHEED image-sequence classification. Our approach consists of an non-supervised auto-encoder (AE) for feature extraction, combined with a supervised convolutional classifier network. We demonstrate that our lightweight network model can accurately identify the exact deoxidation moment. Furthermore we show that the approach is very robust and allows accurate deoxidation detection during months without requiring re-training. The main advantage of the approach is that it can be applied to raw RHEED images without requiring further information such as the rotation angle, temperature, etc.Comment: 6 pages, 6 figure

Biomedical image sequence analysis with application to automatic quantitative assessment of facial paralysis

Facial paralysis is a condition causing decreased movement on one side of the face. A quantitative, objective, and reliable assessment system would be an invaluable tool for clinicians treating patients with this condition. This paper presents an approach based on the automatic analysis of patient video data. Facial feature localization and facial movement detection methods are discussed. An algorithm is presented to process the optical flow data to obtain the motion features in the relevant facial regions. Three classification methods are applied to provide quantitative evaluations of regional facial nerve function and the overall facial nerve function based on the House-Brackmann scale. Experiments show the radial basis function (RBF) neural network to have superior performance

Image sequence analysis for emerging interactive multimedia services - The European COST 211 framework

Cataloged from PDF version of article.Flexibility and efficiency of coding, content extraction, and content-based search are key research topics in the field of interactive multimedia. Ongoing ISO MPEG-4 and MPEG-7 activities are targeting standardization to facilitate such services. European COST Telecommunications activities provide a framework for research collaboration. COST 211bis and COST 211ter activities have been instrumental in the definition and development of the ITU-T H.261 and H.263 standards for videoconferencing over ISDN and videophony over regular phone lines, respectively. The group has also contributed significantly to the ISO MPEG-4 activities. At present a significant effort of the COST 211ter group activities is dedicated toward image and video sequence analysis and segmentation—an important technological aspect for the success of emerging object-based MPEG-4 and MPEG-7 multimedia applications. The current work of COST 211 is centered around the test model, called the Analysis Model (AM). The essential feature of the AM is its ability to fuse information from different sources to achieve a high-quality object segmentation. The current information sources are the intermediate results from frame-based (still) color segmentation, motion vector based segmentation, and changedetection-based segmentation. Motion vectors, which form the basis for the motion vector based intermediate segmentation, are estimated from consecutive frames. A recursive shortest spanning tree (RSST) algorithm is used to obtain intermediate color and motion vector based segmentation results. A rule-based region processor fuses the intermediate results; a postprocessor further refines the final segmentation output. The results of the current AM are satisfactory; it is expected that there will be further improvements of the AM within the COST 211 project

Biomedical image sequence analysis with application to automatic quantitative assessment of facial paralysis

Facial paralysis is a condition causing decreased movement on one side of the face. A quantitative, objective, and reliable assessment system would be an invaluable tool for clinicians treating patients with this condition. This paper presents an approach based on the automatic analysis of patient video data. Facial feature localization and facial movement detection methods are discussed. An algorithm is presented to process the optical flow data to obtain the motion features in the relevant facial regions. Three classification methods are applied to provide quantitative evaluations of regional facial nerve function and the overall facial nerve function based on the House-Brackmann scale. Experiments show the radial basis function (RBF) neural network to have superior performance

Analysis of optical flow models in the framework of calculus of variations

In image sequence analysis, variational optical flow computations require the solution of a parameter dependent optimization problem with a data term and a regularizer. In this paper we study existence and uniqueness of the optimizers. Our studies rely on quasiconvex functionals on the spaces W¹,P(Ω, IRd), with p > 1, BV(Ω, IRd), BD(&Omeag;). The methods that are covered by our results include several existing techniques. Experiments are presented that illustrate the behavior of these approaches