A Hybrid Templated-Based Composite Classification System

An automatic target classification system contains a classifier which reads a feature as an input and outputs a class label. Typically, the feature is a vector of real numbers. Other features can be non-numeric, such as a string of symbols or alphabets. One method of improving the performance of an automatic classification system is through combining two or more independent classifiers that are complementary in nature. Complementary classifiers are observed by finding an optimal method for partitioning the problem space. For example, the individual classifiers may operate to identify specific objects. Another method may be to use classifiers that operate on different features. We propose a design for a hybrid composite classification system, which exploits both real-numbered and non-numeric features with a template matching classification scheme. This composite classification system is made up of two independent classification systems.These two independent classification systems, which receive input from two separate sensors are then combined over various fusion methods for the purpose of target identification. By using these two separate classifiers, we explore conditions that allow the two techniques to be complementary in nature, thus improving the overall performance of the classification system. We examine various fusion techniques, in search of the technique that generates the best results. We investigate different parameter spaces and fusion rules on example problems to demonstrate our classification system. Our examples consider various application areas to help further demonstrate the utility of our classifier. Optimal classifier performance is obtained using a mathematical framework, which takes into account decision variables based on decision-maker preferences and/or engineering specifications, depending upon the classification problem at hand

Action Recognition in Still Images: Confluence of Multilinear Methods and Deep Learning

Motion is a missing information in an image, however, it is a valuable cue for action recognition. Thus, lack of motion information in a single image makes action recognition for still images inherently a very challenging problem in computer vision. In this dissertation, we show that both spatial and temporal patterns provide crucial information for recognizing human actions. Therefore, action recognition depends not only on the spatially-salient pixels, but also on the temporal patterns of those pixels. To address the challenge caused by the absence of temporal information in a single image, we introduce five effective action classification methodologies along with a new still image action recognition dataset. These include (1) proposing a new Spatial-Temporal Convolutional Neural Network, STCNN, trained by fine-tuning a CNN model, pre-trained on appearance-based classification only, over a novel latent space-time domain, named Ranked Saliency Map and Predicted Optical Flow, or RankSM-POF for short, (2) introducing a novel unsupervised Zero-shot approach based on low-rank Tensor Decomposition, named ZTD, (3) proposing the concept of temporal image, a compact representation of hypothetical sequence of images and then using it to design a new hierarchical deep learning network, TICNN, for still image action recognition, (4) introducing a dataset for STill image Action Recognition (STAR), containing over 1M images across 50 different human body-motion action categories. UCF-STAR is the largest dataset in the literature for action recognition in still images, exposing the intrinsic difficulty of action recognition through its realistic scene and action complexity. Moreover, TSSTN, a two-stream spatiotemporal network, is introduced to model the latent temporal information in a single image, and using it as prior knowledge in a two-stream deep network, (5) proposing a parallel heterogeneous meta- learning method to combine STCNN and ZTD through a stacking approach into an ensemble classifier of the proposed heterogeneous base classifiers. Altogether, this work demonstrates benefits of UCF-STAR as a large-scale still images dataset, and show the role of latent motion information in recognizing human actions in still images by presenting approaches relying on predicting temporal information, yielding higher accuracy on widely-used datasets

Unmasking Clever Hans Predictors and Assessing What Machines Really Learn

Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly "intelligent" behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem solving behaviors. Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for. Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication

Incremental learning algorithms and applications

International audienceIncremental learning refers to learning from streaming data, which arrive over time, with limited memory resources and, ideally, without sacrificing model accuracy. This setting fits different application scenarios where lifelong learning is relevant, e.g. due to changing environments , and it offers an elegant scheme for big data processing by means of its sequential treatment. In this contribution, we formalise the concept of incremental learning, we discuss particular challenges which arise in this setting, and we give an overview about popular approaches, its theoretical foundations, and applications which emerged in the last years

Modelos de aprendizaje automático en la detección e identificación de personas: una revisión de literatura

Introduction: This article is the result of research entitled "Development of a prototype to optimize access conditions to the SENA-Pescadero using artificial intelligence and open-source tools", developed at the Servicio Nacional de Aprendizaje in 2020.   Problem: How to identify Machine Learning Techniques applied to computer vision processes through a literature review? Objective: Determine the application, as well as advantages and disadvantages of machine learning techniques focused on the detection and identification of people. Methodology: Systematic literature review in 4 high-impact bibliographic and scientific databases, using search filters and information selection criteria. Results: Machine Learning techniques defined as Principal Component Analysis, Weak Label Regularized Local Coordinate Coding, Support Vector Machines, Haar Cascade Classifiers and EigenFaces and FisherFaces, as well as their applicability in detection and identification processes.   Conclusion: The research led to the identification of the main computational intelligence techniques based on machine learning, applied to the detection and identification of people. Their influence was shown in several application cases, but most of them were focused on the implementation and optimization of access control systems, or tasks in which the identification of people was required for the execution of processes. Originality: Through this research, we studied and defined the main machine learning techniques currently used for the detection and identification of people. Limitations: The systematic review is limited to information available in the 4 databases consulted, and the amount of information is variable as articles are deposited in the databases.Introducción: Este artículo es el resultado de la investigación titulada " Desarrollo de un prototipo para optimizar las condiciones de acceso al SENA-Pescadero utilizando inteligencia artificial y herramientas de código abierto", desarrollada en el Servicio Nacional de Aprendizaje en 2020. Problema: ¿Cómo identificar las técnicas de aprendizaje automático aplicadas a los procesos de visión por computador a través de una revisión bibliográfica? Objetivo: Determinar la aplicación, así como las ventajas y desventajas de las técnicas de aprendizaje automático enfocadas a la detección e identificación de personas. Metodología: Revisión sistemática de la literatura en 4 bases de datos bibliográficas y científicas de alto impacto, utilizando filtros de búsqueda y criterios de selección de información. Resultados: Técnicas de aprendizaje automático definidas como Análisis de Componentes Principales, Codificación Local de Coordenadas Regularizada de Etiquetas Débiles, Máquinas de Vectores de Soporte, Clasificadores en Cascada de Haar y EigenFaces y FisherFaces, así como su aplicabilidad en procesos de detección e identificación. Conclusiones: La investigación permitió identificar las principales técnicas de inteligencia computacional basadas en machine learning aplicadas a la detección e identificación de personas. Su influencia se mostró en varios casos de aplicación, pero la mayoría de ellos se centraron en la implementación y optimización de sistemas de control de acceso, o tareas en las que se requería la identificación de personas para la ejecución de procesos Originalidad: A través de esta investigación se estudiaron y definieron las principales técnicas de machine learning utilizadas actualmente para la detección e identificación de personas

Latent Structured Models for Video Understanding

The proliferation of videos in recent years has spurred a surge of interest in developing efficient techniques for automatic video interpretation. The thesis improves the understanding of videos by building structured models that use latent information to detect and recognize instances of actions or abnormalities in videos. The thesis also proposes efficient algorithms for inference in and learning of the proposed latent structured models that are appropriate for learning with weak supervision. An important class of latent variable models is the multiple instance learning where the training labels are provided only for bags of instances, but not for instances themselves. As inference of latent instance labels is performed jointly with training of a classifier on the same data, multiple-instance learning is very susceptible to overfitting. To increase the robustness of popular methods for multiple instance learning, the thesis introduces a novel concept of superbags (ensemble of bags of bags) that allows for decoupling of classifier training and latent label inference steps. In the thesis, a novel latent structured representation is proposed to discover instances of action classes in videos and jointly train an action classifier on them. Action class instances typically occupy only a part of the whole video that is not annotated in weakly labeled training videos. Therefore, multiple instance learning is proposed to find these latent action instances in training videos and jointly train the action classifier. The thesis proposes a sequential method to multiple instance learning to increase the robustness of the training. For the interpretation of crowded scenes, it is important to detect all irregular objects or actions in a video. However, the abnormality detection is hindered by the fact that the training set does not contain any abnormal sample, thus it is necessary to find abnormalities in a test video without actually knowing what they are. To address this problem, the thesis proposes a probabilistic graphical model for video parsing that searches for latent object hypotheses to jointly explain all the foreground pixels, which are, at the same time, well matched to the normal training samples. By inferring all latent normal hypotheses in a video, the model indirectly finds abnormalities as those hypotheses that are not supported by normal samples but still need to be used to explain the foreground. Video parsing is applied sequentially on individual video frames, where hypotheses are jointly inferred by a local search in a graphical model. The thesis then proposes a spatio-temporal extension of the video parsing, where an efficient inference method based on convex optimization is developed to find abnormal/normal spatio-temporal hypotheses in the video

State of the Art in Face Recognition

Notwithstanding the tremendous effort to solve the face recognition problem, it is not possible yet to design a face recognition system with a potential close to human performance. New computer vision and pattern recognition approaches need to be investigated. Even new knowledge and perspectives from different fields like, psychology and neuroscience must be incorporated into the current field of face recognition to design a robust face recognition system. Indeed, many more efforts are required to end up with a human like face recognition system. This book tries to make an effort to reduce the gap between the previous face recognition research state and the future state