Artificial Intelligence Enabled Methods for Human Action Recognition using Surveillance Videos

Computer vision applications have been attracting researchers and academia. It is more so with cloud computing resources enabling such applications. Analysing video surveillance applications became an important research area due to its widespread applications. For instance, CCTV camera are used in public places in order to monitor situations, identify any theft or crime instances. In presence of thousands of such surveillance videos streaming simultaneously, manual analysis is very tedious and time consuming task. There is need for automated approach for analysis and giving notifications or findings to officers concerned. It is very useful to police and investigation agencies to ascertain facts, recover evidences and even exploit digital forensics. In this context, this paper throws light on different methods of human action recognition (HAR) using machine learning (ML) and deep learning (DL) that come under Artificial Intelligence (AI). It also reviews methods on privacy preserving action recognition and Generative Adversarial Networks (GANs). This paper also provides different datasets being used for human action recognition research besides giving an account of research gaps that help in pursuing further research in the area of human action recognition

시공간 주의집중을 갖는 이중 흐름 행동인식 신경망

학위논문(석사)--서울대학교 대학원 :공과대학 컴퓨터공학부,2019. 8. 전화숙.오늘날 활발한 심층 신경망 연구와 데이터 저장 및 처리 기술 발달로 인해 이 미지 뿐만 아니라 비디오와 같은 시간 흐름을 가진 대용량 데이터에서 다양한 인식 문제를 수행하는 연구가 더욱 더 많은 관심을 받고 있다. 그 중에서도 이중 흐름 신경망은 처음으로 신경망을 통한 학습이 기존의 수작업으로 뽑은 특징보다 (hand- crafted features) 좋은 성능을 보여준 이후로, 비디오 행동 인식에서 주류 아키텍쳐로 자리잡았다. 본 논문에서는 해당 아키텍쳐를 확장하여 비디오에서 동작 인식을 위해 독립적으로 훈련된 이중 흐름 신경망에 시공간 주의집중을 주는 아키텍쳐를 제안했 다. 본 논문에서는 cross attention을 통해 기존의 독립적인 신경망에 상호 보완적인 학습으로 성능 향상을 유도했다. HMDB-51의 표준 비디오 행동인식 벤치 마크에서 본 논문의 아키텍쳐의 성능을 실험하였으며, 기존의 아키텍쳐보다 개선된 성능을 얻을 수 있었다.Two-stream architecture has been mainstream since the success of [1], but two important information is processed independently and not interacted until the late fusion. We investigate a different spatio-temporal attention architecture based on two separate recognition streams (spatial and temporal), which interact with each other by cross attention. The spatial stream performs action recognition from still video frames, whilst the temporal stream is trained to recognise action from motion in the form of dense optical flow. Both streams convey their learned knowledge to the other stream in the form of attention maps. Cross attentions allow us to exploit the availability of supplemental information and enhance learning of the streams. To demonstrate the benefits of our proposed cross-stream spatio-temporal attention architecture, it has been evaluated on two standard action recognition benchmarks where it boosts the previous performance.요 약 제 1 장 서론 제 2 장 관련 연구 2.1 행동 인식에서의 이중 흐름 신경망 2.2 행동인식에서의 주의 집중(Attention) 제 3 장 시공간 주의집중을 갖는 이중 흐름 행동인식 신경망 3.1 효과적인 주의집중 추출 3.2 행동패턴 학습과정 제 4 장 실험 4.1 데이터셋과 구현 세부사항 4.2 성능 비교 제 5 장 결론 ABSTRACTMaste

Discriminative Video Representation Learning

Representation learning is a fundamental research problem in the area of machine learning, refining the raw data to discover representations needed for various applications. However, real-world data, particularly video data, is neither mathematically nor computationally convenient to process due to its semantic redundancy and complexity. Video data, as opposed to images, includes temporal correlation and motion dynamics, but the ground truth label is normally limited to category labels, which makes the video representation learning a challenging problem. To this end, this thesis addresses the problem of video representation learning, specifically discriminative video representation learning, which focuses on capturing useful data distributions and reliable feature representations improving the performance of varied downstream tasks. We argue that neither all frames in one video nor all dimensions in one feature vector are useful and should be equally treated for video representation learning. Based on this argument, several novel algorithms are investigated in this thesis under multiple application scenarios, such as action recognition, action detection and one-class video anomaly detection. These proposed video representation learning methods produce discriminative video features in both deep and non-deep learning setups. Specifically, they are presented in the form of: 1) an early fusion layer that adopts a temporal ranking SVM formulation, agglomerating several optical flow images from consecutive frames into a novel compact representation, named as dynamic optical flow images; 2) an intermediate feature aggregation layer that applies weakly-supervised contrastive learning techniques, learning discriminative video representations via contrasting positive and negative samples from a sequence; 3) a new formulation for one-class feature learning that learns a set of discriminative subspaces with orthonormal hyperplanes to flexibly bound the one-class data distribution using Riemannian optimisation methods. We provide extensive experiments to gain intuitions into why the learned representations are discriminative and useful. All the proposed methods in this thesis are evaluated on standard publicly available benchmarks, demonstrating state-of-the-art performance