Spatiotemporal Barcodes for Image Sequence Analysis

Taking as input a time-varying sequence of two-dimensional (2D) binary images, we develop an algorithm for computing a spatiotemporal 0–barcode encoding lifetime of connected components on the image sequence over time. This information may not coincide with the one provided by the 0–barcode encoding the 0–persistent homology, since the latter does not respect the principle that it is not possible to move backwards in time. A cell complex K is computed from the given sequence, being the cells of K classified as spatial or temporal depending on whether they connect two consecutive frames or not. A spatiotemporal path is defined as a sequence of edges of K forming a path such that two edges of the path cannot connect the same two consecutive frames. In our algorithm, for each vertex v ∈ K, a spatiotemporal path from v to the “oldest” spatiotemporally-connected vertex is computed and the corresponding spatiotemporal 0–bar is added to the spatiotemporal 0–barcode.Junta de Andalucía FQM-369Ministerio de Economía y Competitividad MTM2012-3270

Variational recurrent sequence-to-sequence retrieval for stepwise illustration

We address and formalise the task of sequence-to-sequence (seq2seq) cross-modal retrieval. Given a sequence of text passages as query, the goal is to retrieve a sequence of images that best describes and aligns with the query. This new task extends the traditional cross-modal retrieval, where each image-text pair is treated independently ignoring broader context. We propose a novel variational recurrent seq2seq (VRSS) retrieval model for this seq2seq task. Unlike most cross-modal methods, we generate an image vector corresponding to the latent topic obtained from combining the text semantics and context. This synthetic image embedding point associated with every text embedding point can then be employed for either image generation or image retrieval as desired. We evaluate the model for the application of stepwise illustration of recipes, where a sequence of relevant images are retrieved to best match the steps described in the text. To this end, we build and release a new Stepwise Recipe dataset for research purposes, containing 10K recipes (sequences of image-text pairs) having a total of 67K image-text pairs. To our knowledge, it is the first publicly available dataset to offer rich semantic descriptions in a focused category such as food or recipes. Our model is shown to outperform several competitive and relevant baselines in the experiments. We also provide qualitative analysis of how semantically meaningful the results produced by our model are through human evaluation and comparison with relevant existing methods

Visualization and Correction of Automated Segmentation, Tracking and Lineaging from 5-D Stem Cell Image Sequences

Results: We present an application that enables the quantitative analysis of multichannel 5-D (x, y, z, t, channel) and large montage confocal fluorescence microscopy images. The image sequences show stem cells together with blood vessels, enabling quantification of the dynamic behaviors of stem cells in relation to their vascular niche, with applications in developmental and cancer biology. Our application automatically segments, tracks, and lineages the image sequence data and then allows the user to view and edit the results of automated algorithms in a stereoscopic 3-D window while simultaneously viewing the stem cell lineage tree in a 2-D window. Using the GPU to store and render the image sequence data enables a hybrid computational approach. An inference-based approach utilizing user-provided edits to automatically correct related mistakes executes interactively on the system CPU while the GPU handles 3-D visualization tasks. Conclusions: By exploiting commodity computer gaming hardware, we have developed an application that can be run in the laboratory to facilitate rapid iteration through biological experiments. There is a pressing need for visualization and analysis tools for 5-D live cell image data. We combine accurate unsupervised processes with an intuitive visualization of the results. Our validation interface allows for each data set to be corrected to 100% accuracy, ensuring that downstream data analysis is accurate and verifiable. Our tool is the first to combine all of these aspects, leveraging the synergies obtained by utilizing validation information from stereo visualization to improve the low level image processing tasks.Comment: BioVis 2014 conferenc

Motion connected operators for image sequences

This paper deals with motion-oriented connected operators. These operators eliminate from an original sequence the components that do not undergo a specific motion (defined as a filtering parameter). As any connected operator, they achieve a simplification of the original image while preserving the contour information of the components that have not be removed. Motion-oriented filtering may have a large number of applications including sequence analysis with motion multi-resolution decomposition or motion estimation.Peer ReviewedPostprint (published version

Image sequence analysis and merging

Peer ReviewedPostprint (published version