Image-based tracking technique assessment and application to a fluid-structure interaction experiment

This work analyses the accuracy and capabilities of two image-based tracking techniques related to digital image correlation and the Lucas–Kanade optical flow method, with the subsequent quantification of body motion in a fluid–structure interaction experiment. A computer-controlled shaker was used as a benchmark case to create a one-dimensional oscillatory target motion. Three target frequencies were recorded. The measurements obtained with a low-cost digital camera were compared to a high-precision motion tracking system. The comparison was performed under changes in image resolution, target motion and sampling frequency. The results show that, with a correct selection of the processing parameters, both tracking techniques were able to track the main motion and frequency of the target even after a reduction of four and five times the sampling frequency and image resolution, respectively. Within this good agreement, the Lucas–Kanade technique shows better accuracy under tested conditions, achieving up to 15.6% of lower tracking error. Nevertheless, the achievement of this higher accuracy is highly dependent on the position of the selected initial target point. These considerations are addressed to satisfactorily track the response of a wall-mounted cylinder subjected to a range of turbulent flows using a single camera as the measuring device

Deep Placental Vessel Segmentation for Fetoscopic Mosaicking

During fetoscopic laser photocoagulation, a treatment for twin-to-twin transfusion syndrome (TTTS), the clinician first identifies abnormal placental vascular connections and laser ablates them to regulate blood flow in both fetuses. The procedure is challenging due to the mobility of the environment, poor visibility in amniotic fluid, occasional bleeding, and limitations in the fetoscopic field-of-view and image quality. Ideally, anastomotic placental vessels would be automatically identified, segmented and registered to create expanded vessel maps to guide laser ablation, however, such methods have yet to be clinically adopted. We propose a solution utilising the U-Net architecture for performing placental vessel segmentation in fetoscopic videos. The obtained vessel probability maps provide sufficient cues for mosaicking alignment by registering consecutive vessel maps using the direct intensity-based technique. Experiments on 6 different in vivo fetoscopic videos demonstrate that the vessel intensity-based registration outperformed image intensity-based registration approaches showing better robustness in qualitative and quantitative comparison. We additionally reduce drift accumulation to negligible even for sequences with up to 400 frames and we incorporate a scheme for quantifying drift error in the absence of the ground-truth. Our paper provides a benchmark for fetoscopy placental vessel segmentation and registration by contributing the first in vivo vessel segmentation and fetoscopic videos dataset.Comment: Accepted at MICCAI 202

Video Fragmentation and Reverse Search on the Web

This chapter is focused on methods and tools for video fragmentation and reverse search on the web. These technologies can assist journalists when they are dealing with fake news—which nowadays are being rapidly spread via social media platforms—that rely on the reuse of a previously posted video from a past event with the intention to mislead the viewers about a contemporary event. The fragmentation of a video into visually and temporally coherent parts and the extraction of a representative keyframe for each defined fragment enables the provision of a complete and concise keyframe-based summary of the video. Contrary to straightforward approaches that sample video frames with a constant step, the generated summary through video fragmentation and keyframe extraction is considerably more effective for discovering the video content and performing a fragment-level search for the video on the web. This chapter starts by explaining the nature and characteristics of this type of reuse-based fake news in its introductory part, and continues with an overview of existing approaches for temporal fragmentation of single-shot videos into sub-shots (the most appropriate level of temporal granularity when dealing with user-generated videos) and tools for performing reverse search of a video on the web. Subsequently, it describes two state-of-the-art methods for video sub-shot fragmentation—one relying on the assessment of the visual coherence over sequences of frames, and another one that is based on the identification of camera activity during the video recording—and presents the InVID web application that enables the fine-grained (at the fragment-level) reverse search for near-duplicates of a given video on the web. In the sequel, the chapter reports the findings of a series of experimental evaluations regarding the efficiency of the above-mentioned technologies, which indicate their competence to generate a concise and complete keyframe-based summary of the video content, and the use of this fragment-level representation for fine-grained reverse video search on the web. Finally, it draws conclusions about the effectiveness of the presented technologies and outlines our future plans for further advancing them

Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration

Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor - STFD) that reflects the global motion information of crowds over time. A CNN has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: 1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds that is defined by the similarity of group trajectory structures in temporal space and the foreground blocks based on Gaussian Mixture Model (GMM) in spatial space; 2) dividing multiple clustered groups based on the spectral clustering method by considering image pixels from spatio-temporal segmentation regions as dynamic particles; 3) generating the STFD descriptor instances by calculating the attributes (i.e., collectiveness, stability, conflict and crowd density) of particles in the corresponding groups; 4) inputting generated STFD descriptor instances into the devised convolutional neural network (CNN) to detect suspicious crowd behaviors. The test and evaluation of the devised models and techniques have selected the PETS database as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for detecting crowd anomalies

Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration

Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor - STFD) that reflects the global motion information of crowds over time. A CNN has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: 1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds that is defined by the similarity of group trajectory structures in temporal space and the foreground blocks based on Gaussian Mixture Model (GMM) in spatial space; 2) dividing multiple clustered groups based on the spectral clustering method by considering image pixels from spatio-temporal segmentation regions as dynamic particles; 3) generating the STFD descriptor instances by calculating the attributes (i.e., collectiveness, stability, conflict and crowd density) of particles in the corresponding groups; 4) inputting generated STFD descriptor instances into the devised convolutional neural network (CNN) to detect suspicious crowd behaviors. The test and evaluation of the devised models and techniques have selected the PETS database as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for detecting crowd anomalies

Gaze Strategy in the Free Flying Zebra Finch (Taeniopygia guttata)

Fast moving animals depend on cues derived from the optic flow on their retina. Optic flow from translational locomotion includes information about the three-dimensional composition of the environment, while optic flow experienced during a rotational self motion does not. Thus, a saccadic gaze strategy that segregates rotations from translational movements during locomotion will facilitate extraction of spatial information from the visual input. We analysed whether birds use such a strategy by highspeed video recording zebra finches from two directions during an obstacle avoidance task. Each frame of the recording was examined to derive position and orientation of the beak in three-dimensional space. The data show that in all flights the head orientation was shifted in a saccadic fashion and was kept straight between saccades. Therefore, birds use a gaze strategy that actively stabilizes their gaze during translation to simplify optic flow based navigation. This is the first evidence of birds actively optimizing optic flow during flight

WormAssay: A Novel Computer Application for Whole-Plate Motion-based Screening of Macroscopic Parasites

Lymphatic filariasis is caused by filarial nematode parasites, including Brugia malayi. Adult worms live in the lymphatic system and cause a strong immune reaction that leads to the obstruction of lymph vessels and swelling of the extremities. Chronic disease leads to the painful and disfiguring condition known as elephantiasis. Current drug therapy is effective against the microfilariae (larval stage) of the parasite, but no drugs are effective against the adult worms. One of the major stumbling blocks toward developing effective macrofilaricides to kill the adult worms is the lack of a high throughput screening method for candidate drugs. Current methods utilize systems that measure one well at a time and are time consuming and often expensive. We have developed a low-cost and simple visual imaging system to automate and quantify screening entire plates based on parasite movement. This system can be applied to the study of many macroparasites as well as other macroscopic organisms

Modelling human musculoskeletal functional movements using ultrasound imaging

<p>Abstract</p> <p>Background</p> <p>A widespread and fundamental assumption in the health sciences is that muscle functions are related to a wide variety of conditions, for example pain, ischemic and neurological disorder, exercise and injury. It is therefore highly desirable to study musculoskeletal contributions in clinical applications such as the treatment of muscle injuries, post-surgery evaluations, monitoring of progressive degeneration in neuromuscular disorders, and so on.</p> <p>The spatial image resolution in ultrasound systems has improved tremendously in the last few years and nowadays provides detailed information about tissue characteristics. It is now possible to study skeletal muscles in real-time during activity.</p> <p>Methods</p> <p>The ultrasound images are transformed to be congruent and are effectively compressed and stacked in order to be analysed with multivariate techniques. The method is applied to a relevant clinical orthopaedic research field, namely to describe the dynamics in the Achilles tendon and the calf during real-time movements.</p> <p>Results</p> <p>This study introduces a novel method to medical applications that can be used to examine ultrasound image sequences and to detect, visualise and quantify skeletal muscle dynamics and functions.</p> <p>Conclusions</p> <p>This new objective method is a powerful tool to use when visualising tissue activity and dynamics of musculoskeletal ultrasound registrations.</p

Theory and Validation of Magnetic Resonance Fluid Motion Estimation Using Intensity Flow Data

15 p.Background Motion tracking based on spatial-temporal radio-frequency signals from the pixel representation of magnetic resonance (MR) imaging of a non-stationary fluid is able to provide two dimensional vector field maps. This supports the underlying fundamentals of magnetic resonance fluid motion estimation and generates a new methodology for flow measurement that is based on registration of nuclear signals from moving hydrogen nuclei in fluid. However, there is a need to validate the computational aspect of the approach by using velocity flow field data that we will assume as the true reference information or ground truth. Methodology/Principal Findings In this study, we create flow vectors based on an ideal analytical vortex, and generate artificial signal-motion image data to verify our computational approach. The analytical and computed flow fields are compared to provide an error estimate of our methodology. The comparison shows that the fluid motion estimation approach using simulated MR data is accurate and robust enough for flow field mapping. To verify our methodology, we have tested the computational configuration on magnetic resonance images of cardiac blood and proved that the theory of magnetic resonance fluid motion estimation can be applicable practically. Conclusions/Significance The results of this work will allow us to progress further in the investigation of fluid motion prediction based on imaging modalities that do not require velocity encoding. This article describes a novel theory of motion estimation based on magnetic resonating blood, which may be directly applied to cardiac flow imaging.Kelvin Kian Loong Wong, Richard Malcolm Kelso, Stephen Grant Worthley, Prashanthan Sanders, Jagannath Mazumdar, Derek Abbot