Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams

Devising intelligent agents able to live in an environment and learn by observing the surroundings is a longstanding goal of Artificial Intelligence. From a bare Machine Learning perspective, challenges arise when the agent is prevented from leveraging large fully-annotated dataset, but rather the interactions with supervisory signals are sparsely distributed over space and time. This paper proposes a novel neural-network-based approach to progressively and autonomously develop pixel-wise representations in a video stream. The proposed method is based on a human-like attention mechanism that allows the agent to learn by observing what is moving in the attended locations. Spatio-temporal stochastic coherence along the attention trajectory, paired with a contrastive term, leads to an unsupervised learning criterion that naturally copes with the considered setting. Differently from most existing works, the learned representations are used in open-set class-incremental classification of each frame pixel, relying on few supervisions. Our experiments leverage 3D virtual environments and they show that the proposed agents can learn to distinguish objects just by observing the video stream. Inheriting features from state-of-the art models is not as powerful as one might expect

Deep Constraint-based Propagation in Graph Neural Networks

The popularity of deep learning techniques renewed the interest in neural architectures able to process complex structures that can be represented using graphs, inspired by Graph Neural Networks (GNNs). We focus our attention on the originally proposed GNN model of Scarselli et al. 2009, which encodes the state of the nodes of the graph by means of an iterative diffusion procedure that, during the learning stage, must be computed at every epoch, until the fixed point of a learnable state transition function is reached, propagating the information among the neighbouring nodes. We propose a novel approach to learning in GNNs, based on constrained optimization in the Lagrangian framework. Learning both the transition function and the node states is the outcome of a joint process, in which the state convergence procedure is implicitly expressed by a constraint satisfaction mechanism, avoiding iterative epoch-wise procedures and the network unfolding. Our computational structure searches for saddle points of the Lagrangian in the adjoint space composed of weights, nodes state variables and Lagrange multipliers. This process is further enhanced by multiple layers of constraints that accelerate the diffusion process. An experimental analysis shows that the proposed approach compares favourably with popular models on several benchmarks

Learning with convex constraints

In this paper, we focus on multitask learning and discuss the notion of learning from constraints, in which they limit the space of admissible real values of the task functions. We formulate learning as a variational problem and analyze convex constraints, with special attention to the case of linear bilateral and unilateral constraints. Interestingly, we show that the solution is not always an analytic function and that it cannot be expressed by the classic kernel expansion on the training examples. We provide exact and approximate solutions and report experimental evidence of the improvement with respect to classic kernel machines

Unsupervised Learning by Minimal Entropy Encoding

Following basic principles of information-theoretic learning, in this paper, we propose a novel approach to data clustering, referred to as minimal entropy encoding (MEE), which is based on a set of functions (features) projecting each input onto a minimum entropy configuration (code). Inspired by traditional parsimony principles, we seek solutions in reproducing kernel Hilbert spaces and then we prove that the encoding functions are expressed in terms of kernel expansion. In order to avoid trivial solutions, the developed features must be as different as possible by means of a soft constraint on the empirical estimation of the entropy associated with the encoding functions. This leads to an unconstrained optimization problem that can be efficiently solved by conjugate gradient. We also investigate an optimization strategy based on concave-convex algorithms. The relationships with maximum margin clustering are studied, showing that MEE overcomes some of its critical issues, such as the lack of a multiclass extension and the need to face problems with a large number of constraints. A massive evaluation on several benchmarks of the proposed approach shows improvements over state-of-the-art techniques, both in terms of accuracy and computational complexity

Constraint Verification With Kernel Machines

Based on a recently proposed framework of learning from constraints using kernel-based representations, in this brief, we naturally extend its application to the case of inferences on new constraints. We give examples for polynomials and first-order logic by showing how new constraints can be checked on the basis of given premises and data samples. Interestingly, this gives rise to a perceptual logic scheme in which the inference mechanisms do not rely only on formal schemes, but also on the data probability distribution. It is claimed that when using a properly relaxed computational checking approach, the complementary role of data samples makes it possible to break the complexity barriers of related formal checking mechanisms

Semi-supervised multiclass Kernel machines with probabilistic constraints

The extension of kernel-based binary classifiers to multiclass problems has been approached with different strategies in the last decades. Nevertheless, the most frequently used schemes simply rely on different criteria to combine the decisions of a set of independently trained binary classifiers. In this paper we propose an approach that aims at establishing a connection in the training stage of the classifiers using an innovative criterion. Motivated by the increasing interest in the semi-supervised learning framework, we describe a soft-constraining scheme that allows us to include probabilistic constraints on the outputs of the classifiers, using the unlabeled training data. Embedding this knowledge in the learning process can improve the generalization capabilities of the multiclass classifier, and it leads to a more accurate approximation of a probabilistic output without an explicit post-processing. We investigate our intuition on a face identification problem with 295 classes

Coherence constraints in facial expression recognition

This paper investigates the role of coherence constraints in recognizing facial expressions from images and video sequences. A set of constraints are introduced to bridge a pool of Convolutional Neural Networks (CNNs) during their training stage. Constraints are inspired by practical considerations on the regularity of the temporal evolution of the predictions, and by the idea of connecting the information extracted from multiple representations. We study CNNs with the aim of building a versatile recognizer of expressions in static images that can be further applied to video sequences. First, the importance of different face parts in the recognition task is studied, considering appearance and shape-related features. Then we focus on the Semi-Supervised learning setting, exploiting video data, where only a few frames are supervised. The unsupervised portion of the training data is used to enforce three types of coherence, namely temporal coherence, coherence among the predictions on the face parts and coherence between appearance and shape-based representation. Our experimental analysis shows that coherence constraints improve the quality of the expression recognizer, thus offering a suitable basis to profitably exploit unsupervised video sequences, also in cases in which some portions of the input face are not visible

Machine Learning: A Constraint-Based Approach

Machine Learning: A Constraint-Based Approach, Second Edition provides readers with a refreshing look at the basic models and algorithms of machine learning, with an emphasis on current topics of interest that include neural networks and kernel machines. The book presents the information in a truly unified manner that is based on the notion of learning from environmental constraints. It draws a path towards deep integration with machine learning that relies on the idea of adopting multivalued logic formalisms, such as in fuzzy systems. Special attention is given to deep learning, which nicely fits the constrained-based approach followed in this book. The book presents a simpler unified notion of regularization, which is strictly connected with the parsimony principle, including many solved exercises that are classified according to the Donald Knuth ranking of difficulty, which essentially consists of a mix of warm-up exercises that lead to deeper research problems. A software simulator is also included

Representation of facial features by Catmull-Rom splines

This paper describes a technique for the representation of the 2D frontal view of faces, based on Catmull-Rom splines. It takes advantage of the a priori knowledge about the face structure and of the proprieties of Catmull-Rom splines, like interpolation, smoothness and local control, in order to define a set of key points that correspond among different faces. Moreover, it can compactly describe the whole face even if the face features have not been completely localized. The proposed model has been tested in practical contexts of face analysis and promising qualitative results are included to illustrate its versatility and accuracy