Unsupervised Segmentation of Action Segments in Egocentric Videos using Gaze

Unsupervised segmentation of action segments in egocentric videos is a desirable feature in tasks such as activity recognition and content-based video retrieval. Reducing the search space into a finite set of action segments facilitates a faster and less noisy matching. However, there exist a substantial gap in machine understanding of natural temporal cuts during a continuous human activity. This work reports on a novel gaze-based approach for segmenting action segments in videos captured using an egocentric camera. Gaze is used to locate the region-of-interest inside a frame. By tracking two simple motion-based parameters inside successive regions-of-interest, we discover a finite set of temporal cuts. We present several results using combinations (of the two parameters) on a dataset, i.e., BRISGAZE-ACTIONS. The dataset contains egocentric videos depicting several daily-living activities. The quality of the temporal cuts is further improved by implementing two entropy measures.Comment: To appear in 2017 IEEE International Conference On Signal and Image Processing Application

Segmentation and classification of individual tree crowns

By segmentation and classification of individual tree crowns in high spatial resolution aerial images, information about the forest can be automatically extracted. Segmentation is about finding the individual tree crowns and giving each of them a unique label. Classification, on the other hand, is about recognising the species of the tree. The information of each individual tree in the forest increases the knowledge about the forest which can be useful for managements, biodiversity assessment, etc. Different algorithms for segmenting individual tree crowns are presented and also compared to each other in order to find their strengths and weaknesses. All segmentation algorithms developed in this thesis focus on preserving the shape of the tree crown. Regions, representing the segmented tree crowns, grow according to certain rules from seed points. One method starts from many regions for each tree crown and searches for the region that fits the tree crown best. The other methods start from a set of seed points, representing the locations of the tree crowns, to create the regions. The segmentation result varies from 73 to 95 % correctly segmented visual tree crowns depending on the type of forest and the method. The former value is for a naturally generated mixed forest and the latter for a non-mixed forest. The classification method presented uses shape information of the segments and colour information of the corresponding tree crown in order to decide the species. The classification method classifies 77 % of the visual trees correctly in a naturally generated mixed forest, but on a forest stand level the classification is over 90 %

Brain‑correlates of processing local dependencies within a statistical learning paradigm

Statistical learning refers to the implicit mechanism of extracting regularities in our environment. Numerous studies have investigated the neural basis of statistical learning. However, how the brain responds to violations of auditory regularities based on prior (implicit) learning requires further investigation. Here, we used functional magnetic resonance imaging (fMRI) to investigate the neural correlates of processing events that are irregular based on learned local dependencies. A stream of consecutive sound triplets was presented. Unbeknown to the subjects, triplets were either (a) standard, namely triplets ending with a high probability sound or, (b) statistical deviants, namely triplets ending with a low probability sound. Participants (n = 33) underwent a learning phase outside the scanner followed by an fMRI session. Processing of statistical deviants activated a set of regions encompassing the superior temporal gyrus bilaterally, the right deep frontal operculum including lateral orbitofrontal cortex, and the right premotor cortex. Our results demonstrate that the violation of local dependencies within a statistical learning paradigm does not only engage sensory processes, but is instead reminiscent of the activation pattern during the processing of local syntactic structures in music and language, reflecting the online adaptations required for predictive coding in the context of statistical learning.publishedVersio

Neural bases of learning and recognition of statistical regularities

First published: 09 January 2020Statistical learning is a set of cognitive mechanisms allowing for extracting regularities from the environment and segmenting continuous sensory input into discrete units. The current study used functional magnetic resonance imaging (fMRI) (N = 25) in conjunction with an artificial language learning paradigm to provide new insight into the neural mechanisms of statistical learning, considering both the online process of extracting statistical regularities and the subsequent offline recognition of learned patterns. Notably, prior fMRI studies on statistical learning have not contrasted neural activation during the learning and recognition experimental phases. Here, we found that learning is supported by the superior temporal gyrus and the anterior cingulate gyrus, while subsequent recognition relied on the left inferior frontal gyrus. Besides, prior studies only assessed the brain response during the recognition of trained words relative to novel nonwords. Hence, a further key goal of this study was to understand how the brain supports recognition of discrete constituents from the continuous input versus recognition of mere statistical structure that is used to build new constituents that are statistically congruent with the ones from the input. Behaviorally, recognition performance indicated that statistically congruent novel tokens were less likely to be endorsed as parts of the familiar environment than discrete constituents. fMRI data showed that the left intraparietal sulcus and angular gyrus support the recognition of old discrete constituents relative to novel statistically congruent items, likely reflecting an additional contribution from memory representations for trained items.The research was supported by the Spanish Ministry of Economy and Competitiveness (MINECO) through the “Severo Ochoa” Programme for Centres/Units of Excellence in R&D (SEV-2015- 490), and project Grant RTI2018-098317-B-I00 awarded to M.O., by the Basque Government through project Grant PI-2017-25 awarded to D.S., and by the European Commission as Marie Skłodowska-Curie Fellowship DLV-792331 to L.P

Geometrical-based approach for robust human image detection

In recent years, object detection and classification has been gaining more attention, thus, there are several human object detection algorithms being used to locate and recognize human objects in images. The research of image processing and analyzing based on human shape is one of the hot topic due to the wide applicability in real applications. In this paper, we present a new object classification approach. The new approach will use a simple and robust geometrical model to classify the detected object as human or non-human in the images. In the proposed approach, the object is detected. Then the detected object under different conditions can be accurately classified (i.e. human, non-human) by combining the features that are extracted from the upper portion of the contour and the proposed geometrical model parameters. A software-based simulation using Matlab was performed using INRIA dataset and the obtained results are validated by comparing with five state-of-art approaches in literature and some of the machine learning approaches such as artificial neural networks (ANN), support vector machine (SVM), and random forest (RF). The experimental results show that the proposed object classification approach is efficient and achieved a comparable accuracy to other machine learning approaches and other state-of-art approaches. Keywords: Human classification, Geometrical model, INRIA, Machine learning, SVM, ANN, Random forest