Symbolic and Deep Learning Based Data Representation Methods for Activity Recognition and Image Understanding at Pixel Level

Efficient representation of large amount of data particularly images and video helps in the analysis, processing and overall understanding of the data. In this work, we present two frameworks that encapsulate the information present in such data. At first, we present an automated symbolic framework to recognize particular activities in real time from videos. The framework uses regular expressions for symbolically representing (possibly infinite) sets of motion characteristics obtained from a video. It is a uniform framework that handles trajectory-based and periodic articulated activities and provides polynomial time graph algorithms for fast recognition. The regular expressions representing motion characteristics can either be provided manually or learnt automatically from positive and negative examples of strings (that describe dynamic behavior) using offline automata learning frameworks. Confidence measures are associated with recognitions using Levenshtein distance between a string representing a motion signature and the regular expression describing an activity. We have used our framework to recognize trajectory-based activities like vehicle turns (U-turns, left and right turns, and K-turns), vehicle start and stop, person running and walking, and periodic articulated activities like digging, waving, boxing, and clapping in videos from the VIRAT public dataset, the KTH dataset, and a set of videos obtained from YouTube. Next, we present a core sampling framework that is able to use activation maps from several layers of a Convolutional Neural Network (CNN) as features to another neural network using transfer learning to provide an understanding of an input image. The intermediate map responses of a Convolutional Neural Network (CNN) contain information about an image that can be used to extract contextual knowledge about it. Our framework creates a representation that combines features from the test data and the contextual knowledge gained from the responses of a pretrained network, processes it and feeds it to a separate Deep Belief Network. We use this representation to extract more information from an image at the pixel level, hence gaining understanding of the whole image. We experimentally demonstrate the usefulness of our framework using a pretrained VGG-16 model to perform segmentation on the BAERI dataset of Synthetic Aperture Radar (SAR) imagery and the CAMVID dataset. Using this framework, we also reconstruct images by removing noise from noisy character images. The reconstructed images are encoded using Quadtrees. Quadtrees can be an efficient representation in learning from sparse features. When we are dealing with handwritten character images, they are quite susceptible to noise. Hence, preprocessing stages to make the raw data cleaner can improve the efficacy of their use. We improve upon the efficiency of probabilistic quadtrees by using a pixel level classifier to extract the character pixels and remove noise from the images. The pixel level denoiser uses a pretrained CNN trained on a large image dataset and uses transfer learning to aid the reconstruction of characters. In this work, we primarily deal with classification of noisy characters and create the noisy versions of handwritten Bangla Numeral and Basic Character datasets and use them and the Noisy MNIST dataset to demonstrate the usefulness of our approach

Applications of complex adaptive systems approaches to coastal systems

This thesis investigatesth e application of complex adaptives ystemsa pproaches (e. g. Artificial Neural Networks and Evolutionary Computation) to the study of coastal hydrodynamica nd morphodynamicb ehaviour.T raditionally, nearshorem orphologicalc oastal systems tudiesh ave developeda n understandingo f thosep hysicalp rocesseso ccurringo n both short temporal, and small spatial scales with a large degree of success. The associated approachesa nd conceptsu sedt o study the coastals ystema t theses calesh ave Primarily been linear in nature.H owever,w hent hesea pproachetso studyingt he coastals ystema re extendedto investigating larger temporal and spatial scales,w hich are commensuratew ith the aims of coastal managementr, esults have had less success.T he lack of successi n developing an understandingo f large scalec oastalb ehaviouri s to a large extent attributablet o the complex behavioura ssociatedw ith the coastals ystem.I bis complexity arises as a result of both the stochastic and chaotic nature of the coastal system. This allows small scale system understandingto be acquiredb ut preventst he Largers caleb ehaviourt o be predictede ffectively. This thesis presentsf our hydro-morphodynamicc ase studies to demonstratet he utility of complex adaptives ystema pproachesfo r studying coastals ystems.T he first two demonstrate the application of Artificial Neural Networks, whilst the latter two illustrate the application of EvolutionaryC omputation.C aseS tudy #I considerst he natureo f the discrepancyb etweent he observedl ocation of wave breakingp atternso ver submergeds andbarsa nd the actual sandbar locations.A rtificial Neural Networks were able to quantitativelyc orrectt he observedlo cations to produce reliable estimates of the actual sand bar locations. Case Study #2 considers the developmenot f an approachf or the discriminationo f shorelinel ocation in video imagesf or the productiono f intertidal mapso f the nearshorer egion. In this caset he systemm odelledb y the Artificial Neural Network is the nature of the discrimination model carried out by the eye in delineating a shoreline feature between regions of sand and water. The Artificial Neural Network approachw as shownt o robustly recognisea rangeo f shorelinef eaturesa t a variety of beaches and hydrodynamic settings. Case Study #3 was the only purely hydrodynamic study consideredin the thesis.I t investigatedth e use of Evolutionary Computationt o provide means of developing a parametric description of directional wave spectra in both reflective and nonreflective conditions. It is shown to provide a unifying approach which produces results which surpassedth ose achievedb y traditional analysisa pproachese vent hough this may not strictly have been considered as a fidly complex system. Case Study #4 is the most ambitious applicationa nd addressetsh e needf or data reductiona s a precursorw hen trying to study large scalem orphodynamicd ata sets.I t utilises EvolutionaryC omputationa pproachesto extractt he significant morphodynamic variability evidenced in both directly and remotely sampled nearshorem orphologiesS. ignificantd atar eductioni s achievedw hilst reWning up to 90% of the original variability in the data sets. These case studies clearly demonstrate the ability of complex adaptive systems to be successfidly applied to coastal system studies. This success has been shown to equal and sometimess urpasst he results that may be obtained by traditional approachesT. he strong performance of Complex Adaptive System approaches is closely linked to the level of complexity or non-linearity of the system being studied. Based on a qualitative evaluation, Evolutionary Computation was shown to demonstrate an advantage over Artificial Neural Networks in terms of the level of new insights which may be obtained. However, utility also needs to consider general ease of applicability and ease of implementation of the study approach.I n this sense,A rtificial Neural Networks demonstratem ore utility for the study of coastals ystems.T he qualitative assessmenatp proachu sedt o evaluatet he cases tudiesi n this thesis, may be used as a guide for choosingt he appropriatenesso f either Artificial Neural Networks or Evolutionary Computation for future coastal system studies

Local Binary Patterns in Focal-Plane Processing. Analysis and Applications

Feature extraction is the part of pattern recognition, where the sensor data is transformed into a more suitable form for the machine to interpret. The purpose of this step is also to reduce the amount of information passed to the next stages of the system, and to preserve the essential information in the view of discriminating the data into different classes. For instance, in the case of image analysis the actual image intensities are vulnerable to various environmental effects, such as lighting changes and the feature extraction can be used as means for detecting features, which are invariant to certain types of illumination changes. Finally, classification tries to make decisions based on the previously transformed data. The main focus of this thesis is on developing new methods for the embedded feature extraction based on local non-parametric image descriptors. Also, feature analysis is carried out for the selected image features. Low-level Local Binary Pattern (LBP) based features are in a main role in the analysis. In the embedded domain, the pattern recognition system must usually meet strict performance constraints, such as high speed, compact size and low power consumption. The characteristics of the final system can be seen as a trade-off between these metrics, which is largely affected by the decisions made during the implementation phase. The implementation alternatives of the LBP based feature extraction are explored in the embedded domain in the context of focal-plane vision processors. In particular, the thesis demonstrates the LBP extraction with MIPA4k massively parallel focal-plane processor IC. Also higher level processing is incorporated to this framework, by means of a framework for implementing a single chip face recognition system. Furthermore, a new method for determining optical flow based on LBPs, designed in particular to the embedded domain is presented. Inspired by some of the principles observed through the feature analysis of the Local Binary Patterns, an extension to the well known non-parametric rank transform is proposed, and its performance is evaluated in face recognition experiments with a standard dataset. Finally, an a priori model where the LBPs are seen as combinations of n-tuples is also presentedSiirretty Doriast