CNN-ViT Supported Weakly-Supervised Video Segment Level Anomaly Detection

Video anomaly event detection (VAED) is one of the key technologies in computer vision for smart surveillance systems. With the advent of deep learning, contemporary advances in VAED have achieved substantial success. Recently, weakly supervised VAED (WVAED) has become a popular VAED technical route of research. WVAED methods do not depend on a supplementary self-supervised substitute task, yet they can assess anomaly scores straightway. However, the performance of WVAED methods depends on pretrained feature extractors. In this paper, we first address taking advantage of two pretrained feature extractors for CNN (e.g., C3D and I3D) and ViT (e.g., CLIP), for effectively extracting discerning representations. We then consider long-range and short-range temporal dependencies and put forward video snippets of interest by leveraging our proposed temporal self-attention network (TSAN). We design a multiple instance learning (MIL)-based generalized architecture named CNN-ViT-TSAN, by using CNN- and/or ViT-extracted features and TSAN to specify a series of models for the WVAED problem. Experimental results on publicly available popular crowd datasets demonstrated the effectiveness of our CNN-ViT-TSAN.publishedVersio

Machine learning based small bowel video capsule endoscopy analysis: Challenges and opportunities

YesVideo capsule endoscopy (VCE) is a revolutionary technology for the early diagnosis of gastric disorders. However, owing to the high redundancy and subtle manifestation of anomalies among thousands of frames, the manual construal of VCE videos requires considerable patience, focus, and time. The automatic analysis of these videos using computational methods is a challenge as the capsule is untamed in motion and captures frames inaptly. Several machine learning (ML) methods, including recent deep convolutional neural networks approaches, have been adopted after evaluating their potential of improving the VCE analysis. However, the clinical impact of these methods is yet to be investigated. This survey aimed to highlight the gaps between existing ML-based research methodologies and clinically significant rules recently established by gastroenterologists based on VCE. A framework for interpreting raw frames into contextually relevant frame-level findings and subsequently merging these findings with meta-data to obtain a disease-level diagnosis was formulated. Frame-level findings can be more intelligible for discriminative learning when organized in a taxonomical hierarchy. The proposed taxonomical hierarchy, which is formulated based on pathological and visual similarities, may yield better classification metrics by setting inference classes at a higher level than training classes. Mapping from the frame level to the disease level was structured in the form of a graph based on clinical relevance inspired by the recent international consensus developed by domain experts. Furthermore, existing methods for VCE summarization, classification, segmentation, detection, and localization were critically evaluated and compared based on aspects deemed significant by clinicians. Numerous studies pertain to single anomaly detection instead of a pragmatic approach in a clinical setting. The challenges and opportunities associated with VCE analysis were delineated. A focus on maximizing the discriminative power of features corresponding to various subtle lesions and anomalies may help cope with the diverse and mimicking nature of different VCE frames. Large multicenter datasets must be created to cope with data sparsity, bias, and class imbalance. Explainability, reliability, traceability, and transparency are important for an ML-based diagnostics system in a VCE. Existing ethical and legal bindings narrow the scope of possibilities where ML can potentially be leveraged in healthcare. Despite these limitations, ML based video capsule endoscopy will revolutionize clinical practice, aiding clinicians in rapid and accurate diagnosis

Deep Learning for Crowd Anomaly Detection

Today, public areas across the globe are monitored by an increasing amount of surveillance cameras. This widespread usage has presented an ever-growing volume of data that cannot realistically be examined in real-time. Therefore, efforts to understand crowd dynamics have brought light to automatic systems for the detection of anomalies in crowds. This thesis explores the methods used across literature for this purpose, with a focus on those fusing dense optical flow in a feature extraction stage to the crowd anomaly detection problem. To this extent, five different deep learning architectures are trained using optical flow maps estimated by three deep learning-based techniques. More specifically, a 2D convolutional network, a 3D convolutional network, and LSTM-based convolutional recurrent network, a pre-trained variant of the latter, and a ConvLSTM-based autoencoder is trained using both regular frames and optical flow maps estimated by LiteFlowNet3, RAFT, and GMA on the UCSD Pedestrian 1 dataset. The experimental results have shown that while prone to overfitting, the use of optical flow maps may improve the performance of supervised spatio-temporal architectures

Task adapted reconstruction for inverse problems

The paper considers the problem of performing a task defined on a model parameter that is only observed indirectly through noisy data in an ill-posed inverse problem. A key aspect is to formalize the steps of reconstruction and task as appropriate estimators (non-randomized decision rules) in statistical estimation problems. The implementation makes use of (deep) neural networks to provide a differentiable parametrization of the family of estimators for both steps. These networks are combined and jointly trained against suitable supervised training data in order to minimize a joint differentiable loss function, resulting in an end-to-end task adapted reconstruction method. The suggested framework is generic, yet adaptable, with a plug-and-play structure for adjusting both the inverse problem and the task at hand. More precisely, the data model (forward operator and statistical model of the noise) associated with the inverse problem is exchangeable, e.g., by using neural network architecture given by a learned iterative method. Furthermore, any task that is encodable as a trainable neural network can be used. The approach is demonstrated on joint tomographic image reconstruction, classification and joint tomographic image reconstruction segmentation

Graph Learning for Anomaly Analytics: Algorithms, Applications, and Challenges

Anomaly analytics is a popular and vital task in various research contexts, which has been studied for several decades. At the same time, deep learning has shown its capacity in solving many graph-based tasks like, node classification, link prediction, and graph classification. Recently, many studies are extending graph learning models for solving anomaly analytics problems, resulting in beneficial advances in graph-based anomaly analytics techniques. In this survey, we provide a comprehensive overview of graph learning methods for anomaly analytics tasks. We classify them into four categories based on their model architectures, namely graph convolutional network (GCN), graph attention network (GAT), graph autoencoder (GAE), and other graph learning models. The differences between these methods are also compared in a systematic manner. Furthermore, we outline several graph-based anomaly analytics applications across various domains in the real world. Finally, we discuss five potential future research directions in this rapidly growing field

Graph learning for anomaly analytics : algorithms, applications, and challenges

Anomaly analytics is a popular and vital task in various research contexts that has been studied for several decades. At the same time, deep learning has shown its capacity in solving many graph-based tasks, like node classification, link prediction, and graph classification. Recently, many studies are extending graph learning models for solving anomaly analytics problems, resulting in beneficial advances in graph-based anomaly analytics techniques. In this survey, we provide a comprehensive overview of graph learning methods for anomaly analytics tasks. We classify them into four categories based on their model architectures, namely graph convolutional network, graph attention network, graph autoencoder, and other graph learning models. The differences between these methods are also compared in a systematic manner. Furthermore, we outline several graph-based anomaly analytics applications across various domains in the real world. Finally, we discuss five potential future research directions in this rapidly growing field. © 2023 Association for Computing Machinery

The search for the slave ship Meermin : developing a methodology for finding inter tidal shipwrecks

Text in English with abstracts in English, Afrikaans and isiXhosaThis thesis describes the development of a methodology to find inter tidal shipwrecks. The discussion revolves around finding a particular shipwreck – that of the Dutch slaver Meermin. The story of the revolt on the Meermin helps to focus the search and development of the methodology to find inter tidal shipwrecks as the Meermin was wrecked in this zone. The thesis contextualises the search and the story by discussing not only maritime archaeology in South Africa, but also looking at slave ship archaeology and the history of slavery at the Cape. One of the key techniques for finding shipwrecks is the use of magnetometers. The discussion defines the types of magnetometers available to archaeologists and how magnetometry was applied during the search for the Meermin. This inevitably includes an examination of the shipwrecks wrecked in the area of the Meermin episode as well as the way this region has changed over time. The results of the magnetometer searches (which included airborne, handheld and marine magnetometers) are discussed as well as the ground truthing of the results. The latter involved excavation and the development of excavation strategies, and excavation results are scrutinized. In the final analysis the search for the Meermin is further contextualised by considering the various impacts the project has had in other spheres.Hierdie tesis beskryf die ontwikkeling van ‘n metodologie waarmee skeepswrakke in die inter-gety sone opgespoor kan word. Die Hollandse slaweskip, Meermin, is die fokus van die diskussie. Die storie van die slawe opstand op die Meermin help om die ontwikkeling en soektog na skeepswrakke in die inter-gety sone te verskerp, aangesien dit in hierdie sone was waarin die Meermin gestrand het. Die soektog en storie van die Meermin word gekontekstualiseer deur die bespreking van die ontwikkeling van maritieme argeologie in Suid Afrika, die argeologie van slawe skepe en ‘n kort geskiedenis van slawerny aan die Kaap. Magnetometers is een van die belangrikste tegnieke gebruik vir die opspoor van skeepswrakke. Die tipes magnetometers wat deur argeoloë gebruik word, word gedefinieër asook hoe magnetometers gedurende die soektog na die Meermin gebruik is. Daar word ook gekyk na die ander skepe wat in die area van die Meermin gestrand het en die veranderinge wat deur die jare in die streek plaasgevind het. Die resultate van die magnetometer soektogte (insluitend vliegtuig, draagbare en mariene magnetometers) word bespreek so wel as die opgrawings van die resultate. Hierdie opgrawings het noodwendig gelei tot die ontwikkeling van opgrawings tegnieke. Die resultate van die opgrawings word bespreek. Die finale analise kontekstualiseer die soektog na die Meermin met ‘n bepeinsing van die menige impakte wat die projek gehad het.Le thisisi icacisa ngenkqubela kulwazi-nkqubo lokufumana iinqanawa ezaphuka phakathi kokuzala nokurhoxa kolwandle. Ingxoxo zimalunga nokufunyanwa kwenqanawa ethile eyaphukayo – kanye leyo yayithutha amakhoboka amaHolani i-Meermin. Ibali lovukelo kwi-Meermin liyasinceda siqwalasele uphando nenkqubela kulwazi-nkqubo lokufumana iinqanawa ezaphuka phakathi kokuzala nokurhoxa kolwandle njengoko i-Meermin yaqhekeka kanye kulo mmandla. Ithisisi le isicacisela kanye ngophando nembali ngokuxoxa hayi ngobunzululwazi ngezakudala emanzini eMzantsi Afrika nje kuphela, koko iphinde ijonge ngobunzululwazi ngezakudala kwinqanawa yokuthutha amakhoboka nembali yobukhoboka eKapa. Obunye bobuqili obuphambili ekufumaneni iinqanawa eziqhekekileyo kukusetyenziswa kwezixhobo zokulinganisa iintshukumo. Ingxoxo ibalula iindidi zezixhobo zokulinganisa iintshukumo ezisetyenziswa ziinzululwazi ngezakudala nendlela ekwasetyenziswa ngayo ukulinganiswa kwentshukumo ngethuba kuphandwa i-Meermin. Ngokuqhelekileyo oku kuquka ukucutyungulwa kweenqanawa ezaqhekekayo ziqhekeka kummandla wesehlo esisodwa se-Meermin kunye nendlela le ngingqi eguquke ngayo emveni koko. Iziphumo zophando ngezixhobo zokulinganisa iintshukumo (ziquka ezo zasesibhakabhakeni, ezibanjwa ngesandla nezasemanzini) ziyaxoxwa kunye neziphumo zenyani yenene. Le yokugqibela iquka ukwembiwa nenkqubela kwindlela zokomba, iziphumo zokomba nazo ziqwalaselwe. Kuye kwaphinda kwacaciswa kwintlahlela yokugqibela kuphando lwe-Meermin kuqwalaselwa iimpembelelo ezithile umsebenzi othe wangquzulena nazo nakwezinye iindawo.Anthropology and ArchaeologyM.A. (Archaeology

INTELLIGENT VIDEO SURVEILLANCE OF HUMAN MOTION: ANOMALY DETECTION

Intelligent video surveillance is a system that can highlight extraction and video summarization that require recognition of the activities occurring in the video without any human supervision. Surveillance systems are extremely helpful to guard or protect you from any dangerous condition. In this project, we propose a system that can track and detect abnormal behavior in indoor environment. By concentrating on inside house enviromnent, we want to detect any abnormal behavior between adult and toddler to avoid abusing to happen. In general, the frameworks of a video surveillance system include the following stages: background estimator, segmentation, detection, tracking, behavior understanding and description. We use training behavior profile to collect the description and generate statistically behavior to perform anomaly detection later. We begin with modeling the simplest actions like: stomping, slapping, kicking, pointed sharp or blunt object that do not require sophisticated modeling. A method to model actions with more complex dynamic are then discussed. The results of the system manage to track adult figure, toddler figure and harm object as third subject. With this system, it can bring attention of human personnel security. For future work, we recommend to continue design methods for higher level representation of complex activities to do the matching anomaly detection with real-time video surveillance. We also propose the system to embed with hardware solution for triggered the matching detection as output