Towards On-line Domain-Independent Big Data Learning: Novel Theories and Applications

Feature extraction is an extremely important pre-processing step to pattern recognition, and machine learning problems. This thesis highlights how one can best extract features from the data in an exhaustively online and purely adaptive manner. The solution to this problem is given for both labeled and unlabeled datasets, by presenting a number of novel on-line learning approaches. Specifically, the differential equation method for solving the generalized eigenvalue problem is used to derive a number of novel machine learning and feature extraction algorithms. The incremental eigen-solution method is used to derive a novel incremental extension of linear discriminant analysis (LDA). Further the proposed incremental version is combined with extreme learning machine (ELM) in which the ELM is used as a preprocessor before learning. In this first key contribution, the dynamic random expansion characteristic of ELM is combined with the proposed incremental LDA technique, and shown to offer a significant improvement in maximizing the discrimination between points in two different classes, while minimizing the distance within each class, in comparison with other standard state-of-the-art incremental and batch techniques. In the second contribution, the differential equation method for solving the generalized eigenvalue problem is used to derive a novel state-of-the-art purely incremental version of slow feature analysis (SLA) algorithm, termed the generalized eigenvalue based slow feature analysis (GENEIGSFA) technique. Further the time series expansion of echo state network (ESN) and radial basis functions (EBF) are used as a pre-processor before learning. In addition, the higher order derivatives are used as a smoothing constraint in the output signal. Finally, an online extension of the generalized eigenvalue problem, derived from James Stone’s criterion, is tested, evaluated and compared with the standard batch version of the slow feature analysis technique, to demonstrate its comparative effectiveness. In the third contribution, light-weight extensions of the statistical technique known as canonical correlation analysis (CCA) for both twinned and multiple data streams, are derived by using the same existing method of solving the generalized eigenvalue problem. Further the proposed method is enhanced by maximizing the covariance between data streams while simultaneously maximizing the rate of change of variances within each data stream. A recurrent set of connections used by ESN are used as a pre-processor between the inputs and the canonical projections in order to capture shared temporal information in two or more data streams. A solution to the problem of identifying a low dimensional manifold on a high dimensional dataspace is then presented in an incremental and adaptive manner. Finally, an online locally optimized extension of Laplacian Eigenmaps is derived termed the generalized incremental laplacian eigenmaps technique (GENILE). Apart from exploiting the benefit of the incremental nature of the proposed manifold based dimensionality reduction technique, most of the time the projections produced by this method are shown to produce a better classification accuracy in comparison with standard batch versions of these techniques - on both artificial and real datasets

From Frequency to Meaning: Vector Space Models of Semantics

Computers understand very little of the meaning of human language. This profoundly limits our ability to give instructions to computers, the ability of computers to explain their actions to us, and the ability of computers to analyse and process text. Vector space models (VSMs) of semantics are beginning to address these limits. This paper surveys the use of VSMs for semantic processing of text. We organize the literature on VSMs according to the structure of the matrix in a VSM. There are currently three broad classes of VSMs, based on term-document, word-context, and pair-pattern matrices, yielding three classes of applications. We survey a broad range of applications in these three categories and we take a detailed look at a specific open source project in each category. Our goal in this survey is to show the breadth of applications of VSMs for semantics, to provide a new perspective on VSMs for those who are already familiar with the area, and to provide pointers into the literature for those who are less familiar with the field

Encoding Sequential Information in Semantic Space Models: Comparing Holographic Reduced Representation and Random Permutation

Circular convolution and random permutation have each been proposed as neurally plausible binding operators capable of encoding sequential information in semantic memory. We perform several controlled comparisons of circular convolution and random permutation as means of encoding paired associates as well as encoding sequential information. Random permutations outperformed convolution with respect to the number of paired associates that can be reliably stored in a single memory trace. Performance was equal on semantic tasks when using a small corpus, but random permutations were ultimately capable of achieving superior performance due to their higher scalability to large corpora. Finally, “noisy” permutations in which units are mapped to other units arbitrarily (no one-to-one mapping) perform nearly as well as true permutations. These findings increase the neurological plausibility of random permutations and highlight their utility in vector space models of semantics

Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks

Biological plastic neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. The interplay of these elements leads to the emergence of adaptive behavior and intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks with a large variety of dynamics, architectures, and plasticity rules: these artificial systems are composed of inputs, outputs, and plastic components that change in response to experiences in an environment. These systems may autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. EPANNs have seen considerable progress over the last two decades. Current scientific and technological advances in artificial neural networks are now setting the conditions for radically new approaches and results. In particular, the limitations of hand-designed networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and developments are presented

Fully Distributed Robust Singular Value Decomposition

Articulated Funiculator is a new and innovative concept developed by Tyréns forachieving a more efficient vertical transportation with a higher space utilization.Having a variety of merits, i.e.: simple construction, direct electromagneticthrust propulsion, and high safety and reliability in contrast to rotary inductionmotor, linear induction motor (LIM) is considered to be one of the cases as thepropulsion system for Articulated Funiculator. The thesis is then carried outwith the purpose of determining the feasibility of this study case by designing theLIMs meeting some specific requirements. The detailed requirements include: aset of identical LIMs are required to jointly produce the thrust that is sufficientto vertically raise the moving system up to 2 m/s2; the size of the LIMs cannotexceed the specification of the funiculator; the maximum flux density in the airgap for each LIM is kept slightly below 0.6 T; no iron saturation of any part ofthe LIMs is allowed.In this thesis report, an introduction of LIM is firstly presented. Followingthe introduction, relevant literature has been reviewed for a strengthenedtheoretical fundamentals and a better understanding of LIM’s history and applications. A general classification of LIMs is subsequently introduced. In addtion,an analytical model of the single-sided linear induction motor (SLIM) has beenbuilt based on an approximate equivalent circuit, and the preliminary geometryof the SLIM is thereby obtained. In order to acquire a more comprehensiveunderstanding of the machine characteristics and a more precise SLIM design, atwo-dimensional finite element method (2D-FEM) analysis is performed initiallyaccording to the preliminary geometry. The results, unfortunately, turn out tobe iron severely saturated in the teeth and yoke, and a excessive maximumvalue of air-gap flux density. Specific to the problems, different parameters ofthe SLIM are marginally adjusted and a series of design scenarios are run inFlux2D for 8-pole and 6-pole SLIM. The comparisons between the results areconducted and the final solution is lastly chosen among them.Articulated Funiculator är ett nytt och innovativt koncept som utvecklats av Tyréns för att möjilggöra en mer effektiv vertikal transport och bättre utnyttjautrymme. Tack vare fördelar såsom en enkel konstruktion, direkt elektromagnetiskdragkraftsframdrivning, samt hög säkerhet och tillförlitlighet i motsatstill roterande induktionsmotor, är en linjär induktionsmotor (LIM) aktuell somframdrivningssystem. Detta examensarbete är utfört med syfte att utforma enLIM för att uppfylla vissa specifika krav. De detaljerade kraven inkluderar: enuppsättning identiska LIM krävs för att gemensamt producera tillräcklig dragkraftför att vertikalt höja det rörliga systemet upp till 2 m/s2; storleken påLIM får inte överstiga specifikation; den maximala flödestätheten i luftgapet förvarje LIM hålls är begränsad till knappt 0.6 T; ingen järnmättnad av någon delav LIM är tillåtet. I denna rapport ges först en introduktion av LIM-konceptet. Efter introduktionenhar relevant litteratur granskats för att stärka teoretiska grundkunskapersamt ge en bättre belysning av historiken kring LIMs samt dess applikationer. Utöver detta har en analytisk modell av den ensidiga linjära induktionsmotorn(SLIM) byggts, baserat på en ungefärlig ekvivalent krets med vilket den preliminärageometrin för SLIM. För att erhålla en mer grundläggande förståelse avmaskinens egenskaper är en tvådimensionell analys med finita elementmetoden(2D-FEM) utförd, initialt med användande av en preliminär geometri erhållenmed hjälp av analytisk dimensionering. Resultaten från dessa simuleringar visadedock att järnet mättats kraftigt i både tänderna och oket och ett överdrivetstort maximivärde av luftgapets flödestäthet erhålls. Specifikt för applikationenjusteras olika parametrar och en rad driftscenarier körs i Flux2D för en 8-poligoch en 6-polig SLIM. En slutgiltig jämförelse mellan de olika maskindesignernapresenteras och den rekommenderade lösningen väljs slutligen