Hidden Two-Stream Convolutional Networks for Action Recognition

Analyzing videos of human actions involves understanding the temporal relationships among video frames. State-of-the-art action recognition approaches rely on traditional optical flow estimation methods to pre-compute motion information for CNNs. Such a two-stage approach is computationally expensive, storage demanding, and not end-to-end trainable. In this paper, we present a novel CNN architecture that implicitly captures motion information between adjacent frames. We name our approach hidden two-stream CNNs because it only takes raw video frames as input and directly predicts action classes without explicitly computing optical flow. Our end-to-end approach is 10x faster than its two-stage baseline. Experimental results on four challenging action recognition datasets: UCF101, HMDB51, THUMOS14 and ActivityNet v1.2 show that our approach significantly outperforms the previous best real-time approaches.Comment: Accepted at ACCV 2018, camera ready. Code available at https://github.com/bryanyzhu/Hidden-Two-Strea

A Closer Look at Spatiotemporal Convolutions for Action Recognition

In this paper we discuss several forms of spatiotemporal convolutions for video analysis and study their effects on action recognition. Our motivation stems from the observation that 2D CNNs applied to individual frames of the video have remained solid performers in action recognition. In this work we empirically demonstrate the accuracy advantages of 3D CNNs over 2D CNNs within the framework of residual learning. Furthermore, we show that factorizing the 3D convolutional filters into separate spatial and temporal components yields significantly advantages in accuracy. Our empirical study leads to the design of a new spatiotemporal convolutional block "R(2+1)D" which gives rise to CNNs that achieve results comparable or superior to the state-of-the-art on Sports-1M, Kinetics, UCF101 and HMDB51

Implicit cognitions in awareness: Three empirical examples and implications for conscious identity.

open accessAcross psychological science the prevailing view of mental events includes unconscious mental representations that result from a separate implicit system outside of awareness. Recently, scientific interest in consciousness of self and the widespread application of mindfulness practice have made necessary innovative methods of assessing awareness during cognitive tasks and validating those assessments wherever they are researched. Studies from three areas of psychology, self-esteem, sustainability thinking, and the learning of control systems questioned the unconscious status of implicit cognitions. The studies replicated published results using methods of investigating (a) unselective learning of a control task (b) implicit attitudes using IAT, and (c) the Name-letter effect. In addition, a common analytic method of awareness assessment and its validation was used. Study 1 demonstrated that learned control of a dynamic system was predicted by the validity of rules of control in awareness. In Study 2, verbal reports of hesitations and trial difficulty predicted IAT scores for 34 participants’ environmental attitudes. In Study 3, the famous Name-letter effect was predicted by the validity of university students’ reported awareness of letter preference reasons. The repeated finding that self knowledge in awareness predicted what should be cognitions outside of awareness, according to the dual processing view, suggests an alternative model of implicit mental events in which associative relations evoke conscious symbolic representations. The analytic method of validating phenomenal reports will be discussed along with its potential contribution to research involving implicit cognitions