Going Deeper into Action Recognition: A Survey

Understanding human actions in visual data is tied to advances in complementary research areas including object recognition, human dynamics, domain adaptation and semantic segmentation. Over the last decade, human action analysis evolved from earlier schemes that are often limited to controlled environments to nowadays advanced solutions that can learn from millions of videos and apply to almost all daily activities. Given the broad range of applications from video surveillance to human-computer interaction, scientific milestones in action recognition are achieved more rapidly, eventually leading to the demise of what used to be good in a short time. This motivated us to provide a comprehensive review of the notable steps taken towards recognizing human actions. To this end, we start our discussion with the pioneering methods that use handcrafted representations, and then, navigate into the realm of deep learning based approaches. We aim to remain objective throughout this survey, touching upon encouraging improvements as well as inevitable fallbacks, in the hope of raising fresh questions and motivating new research directions for the reader

Human detection, tracking and segmentation from low-level to high-level vision

The goal of this research is to detect, segment and track a human body as well as estimate its limb configuration from cluttered background. These are fundamental research issues that have attracted intensive attention in the computer vision community because of their wide applications. Meanwhile they also remain to be ones of the most challenging research issues largely due to the ubiquitous visual ambiguities in images/videos. The other challenging factor is the ill-posed nature of the problems. Inspired by the recent findings in cognitive psychology, we adopt several biologically plausible approaches to attack these challenging problems. This dissertation provides a comprehensive study of human detection, tracking and segmentation that covers several research issues ranging from low, middle, and high-level vision.In low-level vision, we investigate video segmentation where the main challenge is the non-convex classification problem, and we develop a cascaded multi-layer segmentation framework where no-convex classification problems are addressed in a split-and-merge paradigm by combining merits of both statistical modeling and graph theory.In middle-level vision, we propose a segmentation based hypothesis-and-test paradigm to achieve joint localization and segmentation that exploits the complementary nature of region-based and edge-based shape priors. In addition, we integrate both priors into a Graph-cut framework to improve the segmentation results.In high-level vision, our research has two related parts. First, we propose a hybrid body representation that embraces part-whole shape priors and part-based spatial prior for integrated pose recognition, localization and segmentation in a given image. Second, we further combine spatial and temporal priors in an integrated online learning and inference framework, where body parts can be detected, localized and segmented simultaneously from a video sequence. Both of them are supported by previous low-level and mid-level vision tasks.Experimental results show that the proposed algorithms can achieve accurate and robust tracking, localization and segmentation results for different walking subjects with significant appearance and motion variability and under cluttered background

User-initialized active contour segmentation and golden-angle real-time cardiovascular magnetic resonance enable accurate assessment of LV function in patients with sinus rhythm and arrhythmias.

BackgroundData obtained during arrhythmia is retained in real-time cardiovascular magnetic resonance (rt-CMR), but there is limited and inconsistent evidence to show that rt-CMR can accurately assess beat-to-beat variation in left ventricular (LV) function or during an arrhythmia.MethodsMulti-slice, short axis cine and real-time golden-angle radial CMR data was collected in 22 clinical patients (18 in sinus rhythm and 4 patients with arrhythmia). A user-initialized active contour segmentation (ACS) software was validated via comparison to manual segmentation on clinically accepted software. For each image in the 2D acquisitions, slice volume was calculated and global LV volumes were estimated via summation across the LV using multiple slices. Real-time imaging data was reconstructed using different image exposure times and frame rates to evaluate the effect of temporal resolution on measured function in each slice via ACS. Finally, global volumetric function of ectopic and non-ectopic beats was measured using ACS in patients with arrhythmias.ResultsACS provides global LV volume measurements that are not significantly different from manual quantification of retrospectively gated cine images in sinus rhythm patients. With an exposure time of 95.2 ms and a frame rate of > 89 frames per second, golden-angle real-time imaging accurately captures hemodynamic function over a range of patient heart rates. In four patients with frequent ectopic contractions, initial quantification of the impact of ectopic beats on hemodynamic function was demonstrated.ConclusionUser-initialized active contours and golden-angle real-time radial CMR can be used to determine time-varying LV function in patients. These methods will be very useful for the assessment of LV function in patients with frequent arrhythmias

TransNet: A Transfer Learning-Based Network for Human Action Recognition

Human action recognition (HAR) is a high-level and significant research area in computer vision due to its ubiquitous applications. The main limitations of the current HAR models are their complex structures and lengthy training time. In this paper, we propose a simple yet versatile and effective end-to-end deep learning architecture, coined as TransNet, for HAR. TransNet decomposes the complex 3D-CNNs into 2D- and 1D-CNNs, where the 2D- and 1D-CNN components extract spatial features and temporal patterns in videos, respectively. Benefiting from its concise architecture, TransNet is ideally compatible with any pretrained state-of-the-art 2D-CNN models in other fields, being transferred to serve the HAR task. In other words, it naturally leverages the power and success of transfer learning for HAR, bringing huge advantages in terms of efficiency and effectiveness. Extensive experimental results and the comparison with the state-of-the-art models demonstrate the superior performance of the proposed TransNet in HAR in terms of flexibility, model complexity, training speed and classification accuracy

Load-Independent And Regional Measures Of Cardiac Function Via Real-Time Mri

LOAD-INDEPENDENT AND REGIONAL MEASURES OF CARDIAC FUNCTION VIA REAL-TIME MRI Francisco Jose Contijoch Robert C Gorman, MD Expansion of infarcted tissue during left ventricular (LV) remodeling after a myocardial infarction is associated with poor long-term prognosis. Several interventions have been developed to limit infarct expansion by modifying the material properties of the infarcted or surrounding borderzone tissue. Measures of myocardial function and material properties can be obtained non-invasively via imaging. However, these measures are sensitive to variations in loading conditions and acquisition of load-independent measures have been limited by surgically invasive procedures and limited spatial resolution. In this dissertation, a real-time magnetic resonance imaging (MRI) technique was validated in clinical patients and instrumented animals, several technical improvements in MRI acquisition and reconstruction were presented for improved imaging resolution, load-independent measures were obtained in animal studies via non-invasive imaging, and regional variations in function were measured in both na�ve and post-infarction animals. Specifically, a golden-angle radial MRI acquisition with non-Cartesian SENSE-based reconstruction with an exposure time less than 95 ms and a frame rate above 89 fps allows for accurate estimation of LV slice volume in clinical patients and instrumented animals. Two technical developments were pursued to improve image quality and spatial resolution. First, the slice volume obtained can be used as a self-navigator signal to generate retrospectively-gated, high-resolution datasets of multiple beat morphologies. Second, cross-correlation of the ECG with previously observed values resulted in accurate interpretation of cardiac phase in patients with arrhythmias and allowed for multi-shot imaging of dynamic scenarios. Synchronizing the measured LV slice volume with an LV pressure signal allowed for pressure-volume loops and corresponding load-independent measures of function to be obtained in instrumented animals. Acquiring LV slice volume at multiple slice locations revealed regional differences in contractile function. Motion-tracking of the myocardium during real-time imaging allowed for differences in contractile function between normal, borderzone, and infarcted myocardium to be measured. Lastly, application of real-time imaging to patients with arrhythmias revealed the variable impact of ectopic beats on global hemodynamic function, depending on frequency and ectopic pattern. This work established the feasibility of obtaining load-independent measures of function via real-time MRI and illustrated regional variations in cardiac function