Pattern Recognition

Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition

Probabilistic Models for Joint Segmentation, Detection and Tracking

Migrace buněk a buněčných částic hraje důležitou roli ve fungování živých organismů. Systematický výzkum buněčné migrace byl umožněn v posledních dvaceti letech rychlým rozvojem neinvazivních zobrazovacích technik a digitálních snímačů. Moderní zobrazovací systémy dovolují studovat chování buněčných populací složených z mnoha ticíců buněk. Manuální analýza takového množství dat by byla velice zdlouhavá, protože některé experimenty vyžadují analyzovat tvar, rychlost a další charakteristiky jednotlivých buněk. Z tohoto důvodu je ve vědecké komunitě velká poptávka po automatických metodách.Migration of cells and subcellular particles plays a crucial role in many processes in living organisms. Despite its importance a systematic research of cell motility has only been possible in last two decades due to rapid development of non-invasive imaging techniques and digital cameras. Modern imaging systems allow to study large populations with thousands of cells. Manual analysis of the acquired data is infeasible, because in order to gain insight into underlying biochemical processes it is sometimes necessary to determine shape, velocity and other characteristics of individual cells. Thus there is a high demand for automatic methods

Simulation Intelligence: Towards a New Generation of Scientific Methods

The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simulation, and artificial intelligence. We call this merger simulation intelligence (SI), for short. We argue the motifs of simulation intelligence are interconnected and interdependent, much like the components within the layers of an operating system. Using this metaphor, we explore the nature of each layer of the simulation intelligence operating system stack (SI-stack) and the motifs therein: (1) Multi-physics and multi-scale modeling; (2) Surrogate modeling and emulation; (3) Simulation-based inference; (4) Causal modeling and inference; (5) Agent-based modeling; (6) Probabilistic programming; (7) Differentiable programming; (8) Open-ended optimization; (9) Machine programming. We believe coordinated efforts between motifs offers immense opportunity to accelerate scientific discovery, from solving inverse problems in synthetic biology and climate science, to directing nuclear energy experiments and predicting emergent behavior in socioeconomic settings. We elaborate on each layer of the SI-stack, detailing the state-of-art methods, presenting examples to highlight challenges and opportunities, and advocating for specific ways to advance the motifs and the synergies from their combinations. Advancing and integrating these technologies can enable a robust and efficient hypothesis-simulation-analysis type of scientific method, which we introduce with several use-cases for human-machine teaming and automated science

Neural Networks for Personalized Recommender Systems

The recommender system is an essential tool for companies and users. A successful recommender system not only can help companies promote their products and services, but also benefit users by filtering out unwanted information. Thus, recommender systems are growing to be indispensable in a wide range of industries. Moreover, due to the fact that neural networks have been proved to be efficient and scalable, they are widely studied and applied to various fields. This thesis aims at developing methods for recommender systems by adapting neural networks. By exploring to adapt neural networks to recommender systems, this thesis investigates challenges that recommender systems are facing, and presents approaches to these challenges. Specifically, these challenges include: (1) data sparsity, (2) the complex relationships between users and items, (3) dynamic user preferences. To address the data sparsity, this thesis proposes to learn both collaborative features and content representations to generate recommendations in case of sparse data. Moreover, it proposes an architecture for the training process to further improve the quality of recommendations. To dynamically learn users' preferences, the thesis proposes to learn temporal features to capture dynamic changes of users' preferences. In this way, both the users' general preferences and the latest interactions are considered. To learn the complex relationships, this thesis also proposes a geometric method to measure nonlinear metric to learn the complex relationship among users and items. Moreover, the relationships between items are also considered to avoid potential problems

In-Material Processing of High Bandwidth Sensor Measurements Using Modular Neural Networks

Robotic materials are a novel class of materials that tightly integrate sensing, computing, and actuation into an engineered material or composite to allow the behavior of the material to be defined algorithmically. Robotic materials are constructed using an embedded network of computing nodes based on small, inexpensive microcontrollers. Examples of such materials include morphable airfoils which change shape in response to flight conditions or mission parameters, robotic skins with rich tactile sensing capabilities that recognize texture or touch gestures, clothing with tightly integrated sensing to assist with or augment the wearer's perception of the environment, or materials with dynamic camouflage capabilities.In this thesis, I develop a framework for in-material processing which tightly couples modularized deep neural networks and high-bandwidth sensors using a network of embedded, material-scale components. This framework enables materials to learn multiple desired responses to stimuli, avoiding the need for accurate modeling of the dynamics of the material and stimuli. I utilize a modular neural network design consisting of convolutional (CNN) and long short-term memory (LSTM) layers implemented in each node in the material as a computational approach for robotic materials. This network architecture allows for nodes in the material to process local sensor values, maintain local state information, and communicate with nodes in a local neighborhood in the materials. A multiobjective optimization approach is employed to automatically design the neural network architectures which maximizes the performance of the network while ensuring hardware budgets, such as memory requirements, are maintained. A communication network design is also developed to allow network modules to learn a communication protocol that limits communication to a desired rate, ensuring in-network bandwidth constraints are maintained.I demonstrate the suitability of this computational model for robotic materials using examples in several domains. An RF-based e-textile gesture input device capable of distinguishing between user control gestures is used to control arbitrary external devices. A tire with embedded piezoelectric sensing capabilites for use in high-performance autonomous vehicles performs state-of-the-art identification of terrains driven on. Two robotic skins are presented---one which is capable of detecting and localizing contact, and identifying the texture of the contacting objects; and a second which assists with avoiding collisions with obstacles and identifies affective touch gestures performed by a human collaborator. Finally, a distributed approach to human activity recognition is presented whose activity identification performance is comparable to a centralized approach, but can be implemented on hardware designed for wearable applications, as opposed to a GPU-enabled device. The examples shown demonstrate that robotic materials can perform significant in-material processing; are loosely coupled from a host system, communicating a minimal number of low-bandwidth events to the host; and can exhibit multifunctional behavior that is analyzed for safety or performance considerations

Quantum Methods for Neural Networks and Application to Medical Image Classification

Quantum machine learning techniques have been proposed as a way to potentially enhance performance in machine learning applications. In this paper, we introduce two new quantum methods for neural networks. The first one is a quantum orthogonal neural network, which is based on a quantum pyramidal circuit as the building block for implementing orthogonal matrix multiplication. We provide an efficient way for training such orthogonal neural networks; novel algorithms are detailed for both classical and quantum hardware, where both are proven to scale asymptotically better than previously known training algorithms. The second method is quantum-assisted neural networks, where a quantum computer is used to perform inner product estimation for inference and training of classical neural networks. We then present extensive experiments applied to medical image classification tasks using current state of the art quantum hardware, where we compare different quantum methods with classical ones, on both real quantum hardware and simulators. Our results show that quantum and classical neural networks generates similar level of accuracy, supporting the promise that quantum methods can be useful in solving visual tasks, given the advent of better quantum hardware.Comment: arXiv admin note: substantial text overlap with arXiv:2109.01831, arXiv:2106.0719

Automatic analysis of electronic drawings using neural network

Neural network technique has been found to be a powerful tool in pattern recognition. It captures associations or discovers regularities with a set of patterns, where the types, number of variables or diversity of the data are very great, the relationships between variables are vaguely understood, or the relationships are difficult to describe adequately with conventional approaches. In this dissertation, which is related to the research and the system design aiming at recognizing the digital gate symbols and characters in electronic drawings, we have proposed: (1) A modified Kohonen neural network with a shift-invariant capability in pattern recognition; (2) An effective approach to optimization of the structure of the back-propagation neural network; (3) Candidate searching and pre-processing techniques to facilitate the automatic analysis of the electronic drawings. An analysis and the system performance reveal that when the shift of an image pattern is not large, and the rotation is only by nx90°, (n = 1, 2, and 3), the modified Kohonen neural network is superior to the conventional Kohonen neural network in terms of shift-invariant and limited rotation-invariant capabilities. As a result, the dimensionality of the Kohonen layer can be reduced significantly compared with the conventional ones for the same performance. Moreover, the size of the subsequent neural network, say, back-propagation feed-forward neural network, can be decreased dramatically. There are no known rules for specifying the number of nodes in the hidden layers of a feed-forward neural network. Increasing the size of the hidden layer usually improves the recognition accuracy, while decreasing the size generally improves generalization capability. We determine the optimal size by simulation to attain a balance between the accuracy and generalization. This optimized back-propagation neural network outperforms the conventional ones designed by experience in general. In order to further reduce the computation complexity and save the calculation time spent in neural networks, pre-processing techniques have been developed to remove long circuit lines in the electronic drawings. This made the candidate searching more effective

Sequence-to-sequence learning for machine translation and automatic differentiation for machine learning software tools

Cette thèse regroupe des articles d'apprentissage automatique et s'articule autour de deux thématiques complémentaires. D'une part, les trois premiers articles examinent l'application des réseaux de neurones artificiels aux problèmes du traitement automatique du langage naturel (TALN). Le premier article introduit une structure codificatrice-décodificatrice avec des réseaux de neurones récurrents pour traduire des segments de phrases de longueur variable. Le deuxième article analyse la performance de ces modèles de `traduction neuronale automatique' de manière qualitative et quantitative, tout en soulignant les difficultés posées par les phrases longues et les mots rares. Le troisième article s'adresse au traitement des mots rares et hors du vocabulaire commun en combinant des algorithmes de compression par dictionnaire et des réseaux de neurones récurrents. D'autre part, la deuxième partie de cette thèse fait abstraction de modèles particuliers de réseaux de neurones afin d'aborder l'infrastructure logicielle nécessaire à leur définition et entraînement. Les infrastructures modernes d'apprentissage profond doivent avoir la capacité d'exécuter efficacement des programmes d'algèbre linéaire et par tableaux, tout en étant capable de différentiation automatique (DA) pour calculer des dérivées multiples. Le premier article aborde les défis généraux posés par la conciliation de ces deux objectifs et propose la solution d'une représentation intermédiaire fondée sur les graphes. Le deuxième article attaque le même problème d'une manière différente: en implémentant un code source par bande dans un langage de programmation dynamique par tableau (Python et NumPy).This thesis consists of a series of articles that contribute to the field of machine learning. In particular, it covers two distinct and loosely related fields. The first three articles consider the use of neural network models for problems in natural language processing (NLP). The first article introduces the use of an encoder-decoder structure involving recurrent neural networks (RNNs) to translate from and to variable length phrases and sentences. The second article contains a quantitative and qualitative analysis of the performance of these `neural machine translation' models, laying bare the difficulties posed by long sentences and rare words. The third article deals with handling rare and out-of-vocabulary words in neural network models by using dictionary coder compression algorithms and multi-scale RNN models. The second half of this thesis does not deal with specific neural network models, but with the software tools and frameworks that can be used to define and train them. Modern deep learning frameworks need to be able to efficiently execute programs involving linear algebra and array programming, while also being able to employ automatic differentiation (AD) in order to calculate a variety of derivatives. The first article provides an overview of the difficulties posed in reconciling these two objectives, and introduces a graph-based intermediate representation that aims to tackle these difficulties. The second article considers a different approach to the same problem, implementing a tape-based source-code transformation approach to AD on a dynamically typed array programming language (Python and NumPy)