Deep neural networks for video classification in ecology

Analyzing large volumes of video data is a challenging and time-consuming task. Automating this process would very valuable, especially in ecological research where massive amounts of video can be used to unlock new avenues of ecological research into the behaviour of animals in their environments. Deep Neural Networks, particularly Deep Convolutional Neural Networks, are a powerful class of models for computer vision. When combined with Recurrent Neural Networks, Deep Convolutional models can be applied to video for frame level video classification. This research studies two datasets: penguins and seals. The purpose of the research is to compare the performance of image-only CNNs, which treat each frame of a video independently, against a combined CNN-RNN approach; and to assess whether incorporating the motion information in the temporal aspect of video improves the accuracy of classifications in these two datasets. Video and image-only models offer similar out-of-sample performance on the simpler seals dataset but the video model led to moderate performance improvements on the more complex penguin action recognition dataset

AXES at TRECVID 2012: KIS, INS, and MED

The AXES project participated in the interactive instance search task (INS), the known-item search task (KIS), and the multimedia event detection task (MED) for TRECVid 2012. As in our TRECVid 2011 system, we used nearly identical search systems and user interfaces for both INS and KIS. Our interactive INS and KIS systems focused this year on using classifiers trained at query time with positive examples collected from external search engines. Participants in our KIS experiments were media professionals from the BBC; our INS experiments were carried out by students and researchers at Dublin City University. We performed comparatively well in both experiments. Our best KIS run found 13 of the 25 topics, and our best INS runs outperformed all other submitted runs in terms of P@100. For MED, the system presented was based on a minimal number of low-level descriptors, which we chose to be as large as computationally feasible. These descriptors are aggregated to produce high-dimensional video-level signatures, which are used to train a set of linear classifiers. Our MED system achieved the second-best score of all submitted runs in the main track, and best score in the ad-hoc track, suggesting that a simple system based on state-of-the-art low-level descriptors can give relatively high performance. This paper describes in detail our KIS, INS, and MED systems and the results and findings of our experiments

Efficient Video Transport over Lossy Networks

Nowadays, packet video is an important application of the Internet. Unfortunately the capacity of the Internet is still very heterogeneous because it connects high bandwidth ATM networks as well as low bandwidth ISDN dial in lines. The MPEG-2 and MPEG-4 video compression standards provide efficient video encoding for high and low bandwidth media streams. In particular they include two paradigms which make those standards suitable for the transmission of video via heterogeneous networks. Both support layered video streams and MPEG-4 additionally allows the independent coding of video objects. In this paper we discuss those two paradigms, give an overview of the MPEG video compression standards and describe transport protocols for Real Time Media transport over lossy networks. Furthermore, we propose a real-time segmentation approach for extracting video objects in teleteaching scenarios

Suite2p: beyond 10,000 neurons with standard two-photon microscopy

Two-photon microscopy of calcium-dependent sensors has enabled unprecedented recordings from vast populations of neurons. While the sensors and microscopes have matured over several generations of development, computational methods to process the resulting movies remain inefficient and can give results that are hard to interpret. Here we introduce Suite2p: a fast, accurate and complete pipeline that registers raw movies, detects active cells, extracts their calcium traces and infers their spike times. Suite2p runs on standard workstations, operates faster than real time, and recovers ~2 times more cells than the previous state-of-the-art method. Its low computational load allows routine detection of ~10,000 cells simultaneously with standard two-photon resonant-scanning microscopes. Recordings at this scale promise to reveal the fine structure of activity in large populations of neurons or large populations of subcellular structures such as synaptic boutons

RPCA-KFE: Key Frame Extraction for Consumer Video based Robust Principal Component Analysis

Key frame extraction algorithms consider the problem of selecting a subset of the most informative frames from a video to summarize its content.Comment: This paper has been withdrawn by the author due to a crucial sign error in equation

Multi-shot Echo Planar Imaging for accelerated Cartesian MR Fingerprinting: An alternative to conventional spiral MR Fingerprinting.

PURPOSE: To develop an accelerated Cartesian MRF implementation using a multi-shot EPI sequence for rapid simultaneous quantification of T1 and T2 parameters. METHODS: The proposed Cartesian MRF method involved the acquisition of highly subsampled MR images using a 16-shot EPI readout. A linearly varying flip angle train was used for rapid, simultaneous T1 and T2 quantification. The results were compared to a conventional spiral MRF implementation. The acquisition time per slice was 8s and this method was validated on two different phantoms and three healthy volunteer brains in vivo. RESULTS: Joint T1 and T2 estimations using the 16-shot EPI readout are in good agreement with the spiral implementation using the same acquisition parameters (<4% deviation for T1 and <6% deviation for T2). The T1 and T2 values also agree with the conventional values previously reported in the literature. The visual qualities of fine brain structures in the multi-parametric maps generated by multi-shot EPI-MRF and Spiral-MRF implementations were comparable. CONCLUSION: The multi-shot EPI-MRF method generated accurate quantitative multi-parametric maps similar to conventional Spiral-MRF. This multi-shot approach achieved considerable k-space subsampling and comparatively short TRs in a similar manner to spirals and therefore provides an alternative for performing MRF using an accelerated Cartesian readout; thereby increasing the potential usability of MRF.The research leading to these results has received funding from the European Commission H2020 Framework Programme (H2020- MSCAITN- 2014), number 642685 MacSeNet, the Engineering and Physical Sciences Research Council (EPSRC) platform Compressed Quantitative MRI grant, number EP/M019802/1 and the Scottish Research Partnership in Engineering (SRPe) award, number SRPe PECRE1718/ 17

High‐speed 4‐dimensional scanning transmission electron microscopy using compressive sensing techniques

Here we show that compressive sensing allows 4‐dimensional (4‐D) STEM data to be obtained and accurately reconstructed with both high‐speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4‐D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal‐to‐noise ratio in the recovered phase using 0.3% of the total data. Lay abstract: Four‐dimensional scanning transmission electron microscopy (4‐D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2‐dimensional signal is acquired at each 2‐D probe location, equating to a 4‐D dataset. Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4‐D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256^2 probe locations can take on the order of 30s, approximately 100‐1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4‐D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination. Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10‐100 times. The acquired data is then processed to fill‐in or inpaint the missing data, taking advantage of the inherently low‐complex signals which can be linearly combined to recover the information. In this work, similar methods are demonstrated for the acquisition of 4‐D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4‐D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100‐300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera