Evaluating Two-Stream CNN for Video Classification

Videos contain very rich semantic information. Traditional hand-crafted features are known to be inadequate in analyzing complex video semantics. Inspired by the huge success of the deep learning methods in analyzing image, audio and text data, significant efforts are recently being devoted to the design of deep nets for video analytics. Among the many practical needs, classifying videos (or video clips) based on their major semantic categories (e.g., "skiing") is useful in many applications. In this paper, we conduct an in-depth study to investigate important implementation options that may affect the performance of deep nets on video classification. Our evaluations are conducted on top of a recent two-stream convolutional neural network (CNN) pipeline, which uses both static frames and motion optical flows, and has demonstrated competitive performance against the state-of-the-art methods. In order to gain insights and to arrive at a practical guideline, many important options are studied, including network architectures, model fusion, learning parameters and the final prediction methods. Based on the evaluations, very competitive results are attained on two popular video classification benchmarks. We hope that the discussions and conclusions from this work can help researchers in related fields to quickly set up a good basis for further investigations along this very promising direction.Comment: ACM ICMR'1

Efficient On-the-fly Category Retrieval using ConvNets and GPUs

We investigate the gains in precision and speed, that can be obtained by using Convolutional Networks (ConvNets) for on-the-fly retrieval - where classifiers are learnt at run time for a textual query from downloaded images, and used to rank large image or video datasets. We make three contributions: (i) we present an evaluation of state-of-the-art image representations for object category retrieval over standard benchmark datasets containing 1M+ images; (ii) we show that ConvNets can be used to obtain features which are incredibly performant, and yet much lower dimensional than previous state-of-the-art image representations, and that their dimensionality can be reduced further without loss in performance by compression using product quantization or binarization. Consequently, features with the state-of-the-art performance on large-scale datasets of millions of images can fit in the memory of even a commodity GPU card; (iii) we show that an SVM classifier can be learnt within a ConvNet framework on a GPU in parallel with downloading the new training images, allowing for a continuous refinement of the model as more images become available, and simultaneous training and ranking. The outcome is an on-the-fly system that significantly outperforms its predecessors in terms of: precision of retrieval, memory requirements, and speed, facilitating accurate on-the-fly learning and ranking in under a second on a single GPU.Comment: Published in proceedings of ACCV 201

Receptive Field Block Net for Accurate and Fast Object Detection

Current top-performing object detectors depend on deep CNN backbones, such as ResNet-101 and Inception, benefiting from their powerful feature representations but suffering from high computational costs. Conversely, some lightweight model based detectors fulfil real time processing, while their accuracies are often criticized. In this paper, we explore an alternative to build a fast and accurate detector by strengthening lightweight features using a hand-crafted mechanism. Inspired by the structure of Receptive Fields (RFs) in human visual systems, we propose a novel RF Block (RFB) module, which takes the relationship between the size and eccentricity of RFs into account, to enhance the feature discriminability and robustness. We further assemble RFB to the top of SSD, constructing the RFB Net detector. To evaluate its effectiveness, experiments are conducted on two major benchmarks and the results show that RFB Net is able to reach the performance of advanced very deep detectors while keeping the real-time speed. Code is available at https://github.com/ruinmessi/RFBNet.Comment: Accepted by ECCV 201

Towards Bottom-Up Analysis of Social Food

in ACM Digital Health Conference 201

The age of data-driven proteomics : how machine learning enables novel workflows

A lot of energy in the field of proteomics is dedicated to the application of challenging experimental workflows, which include metaproteomics, proteogenomics, data independent acquisition (DIA), non-specific proteolysis, immunopeptidomics, and open modification searches. These workflows are all challenging because of ambiguity in the identification stage; they either expand the search space and thus increase the ambiguity of identifications, or, in the case of DIA, they generate data that is inherently more ambiguous. In this context, machine learning-based predictive models are now generating considerable excitement in the field of proteomics because these predictive models hold great potential to drastically reduce the ambiguity in the identification process of the above-mentioned workflows. Indeed, the field has already produced classical machine learning and deep learning models to predict almost every aspect of a liquid chromatography-mass spectrometry (LC-MS) experiment. Yet despite all the excitement, thorough integration of predictive models in these challenging LC-MS workflows is still limited, and further improvements to the modeling and validation procedures can still be made. In this viewpoint we therefore point out highly promising recent machine learning developments in proteomics, alongside some of the remaining challenges

Melatonin in the dermal film limits the blood lymphocyte death in experimental thermal trauma

According to WHO data, about 11 million people need medical care after burns every year. In the overall structure of burns, the share of thermal trauma (TT) is 80%. Lymphocytopenia in TT is a risk factor for infectious complications and limited repair, and the development of new tools for TT therapy using dermal films is demanded in combustiology. The aim of the study was to evaluate changes in blood lymphocyte parameters, i.e., quantitative composition and their death during experimental thermal damage under the influence of the originally developed dermal film with melatonin (MT) in 49 inbred rats. The grade IIIA TT of 3.5% body surface was modeled by contact with boiling water for 12 s. Dermal films based on sodium carboxymethylcellulose supplemented with MT at a concentration of 0.005 g/g were applied daily for 5 days. The total numbers of lymphocytes, CD45RA+ and CD3+ cells, counts of lymphocytes with signs of partial necrosis, early and late apoptosis were assessed in blood. Relative decrease in the area and rate of the burn wound epithelization were also calculated. In animals with TT, the number of blood lymphocytes decreased on days 5, 10 and 20, including CD45RA+ and CD3+, along with increased amounts of lymphocytes with signs of necrosis, late and early apoptosis. By the term of 20 days, the burn wound area was reduced by 11.5%. Usage of dermal films with MT increased the amount of CD3+ cells in blood on days 5 and 10, CD45RA+ on days 5, 10 and 20, being associated with decreased number of lymphocytes showing signs of early apoptosis on days 5, 10 and 20, as well as features of necrosis and late apoptosis on days 5 following TT, accelerates the healing of a burn wound on days 5, 10 and 20 after TT. with a 20 cent reduction of its area by the day 20. Epithelization rate of the burn wound when applying MT-supplemented dermal film on days 5, 10 and 20 increases, along with higher amounts of CD3+ in the blood, and reduced counts of lymphocytes with signs of early apoptosis

Immunotropic effects of vitamin D3 in original rectal suppositories in experimental ulcerative colitis

Increased incidence of ulcerative colitis (UC) is a prerequisite for searching new therapeutic approaches, primarily with an opportunity of site-directed impact on the colon lesion. UC pathogenesis is associated with dysregulated immune response, and limited effectiveness of basic therapy for the disorder. Vitamin D3 exhibits antioxidant, anti-inflammatory, immunomodulatory and other properties, it has been shown to be effective in some autoimmune diseases, thus prompting us to study its effect on immune status in UC. We aimed for studying the effect of vitamin D3, as a component of original rectal suppositories, upon clinical course and indexes of immune status in experimental UC. UC in rats was modeled with 3% oxazolone solution. The vitamin D3-containing suppositories (1500 IU) weighing 300 mg were administered per rectum every 12 hours for 6 days. On days 2, 4 and 6 of UC, the clinical features were assessed as well as blood leukocyte counts, numbers of CD3+, CD45RA+; absorbing and NBT-reducing abilities of blood neutrophils were determined; IgM, IgG, IL-6 and IL-8 concentrations in serum were also studied.The DAI index increased in non-treated UC, along with raised neutrophil numbers in blood, their absorption and NBT-reducing activity was also increased, the total number of lymphocytes, including CD3+, CD45RA+ became higher, serum concentrations of IgM, IgG, IL-6, IL-8 increased. Local use of vitamin D3 in UC reduces DAI parameters, causes decrease in blood neutrophil counts, reducing and partially restoring absorptive and NBT-reducing abilities of neutrophils, decline of total lymphocyte counts in blood, partially restoring the CD3+ and CD45RA+ numbers, causing decline and partial restoration of serum IgM, IgG, IL-6, IL-8 concentrations. An association between clinical signs and indexes of immune status in UC was established under the conditions of vitamin D3 use. Conclusions: The protective effect of vitamin D3 in UC can be mediated by its antioxidant effect, changes in production of immunoregulatory cytokines, modulation of Th1-, Th2-, Th17-dependent reactions and Treg activity, being a pre-requisite for further studies to clarify the mechanism of vitamin D3 immunotropic action in UC,with an opportunity of using it in clinical practice

Throughput Prediction of Asynchronous SGD in TensorFlow

Modern machine learning frameworks can train neural networks using multiple nodes in parallel, each computing parameter updates with stochastic gradient descent (SGD) and sharing them asynchronously through a central parameter server. Due to communication overhead and bottlenecks, the total throughput of SGD updates in a cluster scales sublinearly, saturating as the number of nodes increases. In this paper, we present a solution to predicting training throughput from profiling traces collected from a single-node configuration. Our approach is able to model the interaction of multiple nodes and the scheduling of concurrent transmissions between the parameter server and each node. By accounting for the dependencies between received parts and pending computations, we predict overlaps between computation and communication and generate synthetic execution traces for configurations with multiple nodes. We validate our approach on TensorFlow training jobs for popular image classification neural networks, on AWS and on our in-house cluster, using nodes equipped with GPUs or only with CPUs. We also investigate the effects of data transmission policies used in TensorFlow and the accuracy of our approach when combined with optimizations of the transmission schedule

Weakly-Supervised Evidence Pinpointing and Description

We propose a learning method to identify which specific regions and features of images contribute to a certain classification. In the medical imaging context, they can be the evidence regions where the abnormalities are most likely to appear, and the discriminative features of these regions supporting the pathology classification. The learning is weakly-supervised requiring only the pathological labels and no other prior knowledge. The method can also be applied to learn the salient description of an anatomy discriminative from its background, in order to localise the anatomy before a classification step. We formulate evidence pinpointing as a sparse descriptor learning problem. Because of the large computational complexity, the objective function is composed in a stochastic way and is optimised by the Regularised Dual Averaging algorithm. We demonstrate that the learnt feature descriptors contain more specific and better discriminative information than hand-crafted descriptors contributing to superior performance for the tasks of anatomy localisation and pathology classification respectively. We apply our method on the problem of lumbar spinal stenosis for localising and classifying vertebrae in MRI images. Experimental results show that our method when trained with only target labels achieves better or competitive performance on both tasks compared with strongly-supervised methods requiring labels and multiple landmarks. A further improvement is achieved with training on additional weakly annotated data, which gives robust localisation with average error within 2 mm and classification accuracies close to human performance

Deep Learning for Single-Molecule Science

Exploring and making predictions based on single-molecule data can be challenging, not only due to the sheer size of the datasets, but also because a priori knowledge about the signal characteristics is typically limited and poor signal-to-noise ratio. For example, hypothesis-driven data exploration, informed by an expectation of the signal characteristics, can lead to interpretation bias or loss of information. Equally, even when the different data categories are known, e.g., the four bases in DNA sequencing, it is often difficult to know how to make best use of the available information content. The latest developments in Machine Learning (ML), so-called Deep Learning (DL) offers an interesting, new avenues to address such challenges. In some applications, such as speech and image recognition, DL has been able to outperform conventional Machine Learning strategies and even human performance. However, to date DL has not been applied much in single-molecule science, presumably in part because relatively little is known about the 'internal workings' of such DL tools within single-molecule science as a field. In this Tutorial, we make an attempt to illustrate in a step-by-step guide how one of those, a Convolutional Neural Network, may be used for base calling in DNA sequencing applications. We compare it with a Support Vector Machine as a more conventional ML method, and and discuss some of the strengths and weaknesses of the approach. In particular, a 'deep' neural network has many features of a 'black box', which has important implications on how we look at and interpret data