Using generative models for handwritten digit recognition

We describe a method of recognizing handwritten digits by fitting generative models that are built from deformable B-splines with Gaussian ``ink generators'' spaced along the length of the spline. The splines are adjusted using a novel elastic matching procedure based on the Expectation Maximization (EM) algorithm that maximizes the likelihood of the model generating the data. This approach has many advantages. (1) After identifying the model most likely to have generated the data, the system not only produces a classification of the digit but also a rich description of the instantiation parameters which can yield information such as the writing style. (2) During the process of explaining the image, generative models can perform recognition driven segmentation. (3) The method involves a relatively small number of parameters and hence training is relatively easy and fast. (4) Unlike many other recognition schemes it does not rely on some form of pre-normalization of input images, but can handle arbitrary scalings, translations and a limited degree of image rotation. We have demonstrated our method of fitting models to images does not get trapped in poor local minima. The main disadvantage of the method is it requires much more computation than more standard OCR techniques

Towards robust convolutional neural networks in challenging environments

Image classification is one of the fundamental tasks in the field of computer vision. Although Artificial Neural Network (ANN) showed a lot of promise in this field, the lack of efficient computer hardware subdued its potential to a great extent. In the early 2000s, advances in hardware coupled with better network design saw the dramatic rise of Convolutional Neural Network (CNN). Deep CNNs pushed the State-of-The-Art (SOTA) in a number of vision tasks, including image classification, object detection, and segmentation. Presently, CNNs dominate these tasks. Although CNNs exhibit impressive classification performance on clean images, they are vulnerable to distortions, such as noise and blur. Fine-tuning a pre-trained CNN on mutually exclusive or a union set of distortions is a brute-force solution. This iterative fine-tuning process with all known types of distortion is, however, exhaustive and the network struggles to handle unseen distortions. CNNs are also vulnerable to image translation or shift, partly due to common Down-Sampling (DS) layers, e.g., max-pooling and strided convolution. These operations violate the Nyquist sampling rate and cause aliasing. The textbook solution is low-pass filtering (blurring) before down-sampling, which can benefit deep networks as well. Even so, non-linearity units, such as ReLU, often re-introduce the problem, suggesting that blurring alone may not suffice. Another important but under-explored issue for CNNs is unknown or Open Set Recognition (OSR). CNNs are commonly designed for closed set arrangements, where test instances only belong to some ‘Known Known’ (KK) classes used in training. As such, they predict a class label for a test sample based on the distribution of the KK classes. However, when used under the OSR setup (where an input may belong to an ‘Unknown Unknown’ or UU class), such a network will always classify a test instance as one of the KK classes even if it is from a UU class. Historically, CNNs have struggled with detecting objects in images with large difference in scale, especially small objects. This is because the DS layers inside a CNN often progressively wipe out the signal from small objects. As a result, the final layers are left with no signature from these objects leading to degraded performance. In this work, we propose solutions to the above four problems. First, we improve CNN robustness against distortion by proposing DCT based augmentation, adaptive regularisation, and noise suppressing Activation Functions (AF). Second, to ensure further performance gain and robustness to image transformations, we introduce anti-aliasing properties inside the AF and propose a novel DS method called blurpool. Third, to address the OSR problem, we propose a novel training paradigm that ensures detection of UU classes and accurate classification of the KK classes. Finally, we introduce a novel CNN that enables a deep detector to identify small objects with high precision and recall. We evaluate our methods on a number of benchmark datasets and demonstrate that they outperform contemporary methods in the respective problem set-ups.Doctor of Philosoph

Reining in the Deep Generative Models

Diese Doktorarbeit untersucht die Kontrollierbarkeit generativer Modelle (insbesondere VAEs und GANs), angewandt hauptsächlich auf Bilder. Wir verbessern 1. die Qualität der generierten Bilder durch das Entfernen der willkürlichen Annahme über den Prior, 2. die Performanz der Klassifikation durch das wählen einer passenden Verteilung im latenten Raum und 3., die Inferenzperformanz durch die gleichzeitige Optimierung einer Kostenfunktion für die Generierung und Inferenz. Variationale Autoencoder (VAEs) sind ein sehr nützliches Werkzeug, da sie als Basis für eine Vielzahl von Aufgaben im Bereich „Maschinelles Lernen“ verwendet werden können, wie beispielsweise für teilüberwachtes Lernen, lernen von Repräsentationen, und unüberwachtem Lernen, usw. Die von VAEs generierten Bilder sind meist stark geglättet, was die praktische Anwendung deutlich limitiert. Als Erklärung hierfür dienen zwei Hypothesen: erstens, ein schlechtes Modell der Likelihood and zweitens, einen zu einfachen Prior. Wir untersuchen diese Hypothesen durch das Erstellen eines deterministischen Autoencoders, den wir regularisierten Autoencoder (RAE) nennen, von dem Stichproben gezogen werden können. Diese Ergänzung erlaubt es uns beliebige Prior-Verteilungen im latenten Raum vorzugeben, wodurch wir Hypothese Eins untersuchen. Diese Untersuchung führt zur Schlussfolgerung, dass der Hauptgrund für die verschwommenen Bilder eines VAEs ein schlecht gewähltes Prior Modell ist. Des Weiteren zeigen wir, dass die Kombination generativer (z.B. VAE-Objektiv) und diskriminativer (z.B. Klassifikatoren) Kostenfunktionen die Performanz für beide steigert. Dafür verwenden wir eine spezielle Variante eines RAE zum Erstellen eines Klassifikators, der robust gegen „Adversarial Attacks“ ist. Konditionierte generative Modelle haben das Potential die Animationsindustrie, neben anderer Industrien, zu revolutionieren. Um dies zu erreichen müssen zwei Schlüsselvoraussetzungen erfüllt werden: erstens eine hohe Qualität der generierten Daten (d.h. die Erzeugung von hoch auflösenden Bildern) und zweitens die generierten Daten müssen ihrer Konditionierung folgen (d.h. erzeugte Bilder müssen die durch die Konditionierung festgelegten Eigenschaften erfüllen). Wir verwenden die Pixel-lokalisierte Korrelation zwischen der Konditionierungsvariable und dem generierten Bild, um einen starken Zusammenhang zwischen beiden sicherzustellen. Dadurch erhalten wir präzise Kontrolle über die generierten Daten. Darüber hinaus zeigen wir, dass das Schließen des Generations-Inferenz Kreises (beide gemeinsam trainieren) von latenten Variablenmodellen zur Verbesserung von sowohl der Generierungskomponente als auch der Inferenzkomponente führt. Dies ermöglicht das gemeinsame Trainieren eines generativen Modells und eines Modells für Inferenz in einem einheitlichen Rahmen. Dies ist sowohl im überwachten, als auch im teilüberwachten Lernen, möglich. Mit diesem vorgeschlagenen Ansatz ist es möglich einen robusten Klassifikator zu trainieren, durch die Verwendung der Marginalen Likelihood eines Datenpunktes, der Entfernung der willkürlichen Annahme über den Prior, der Abmilderung der Diskrepanz zwischen Prior- und Posterior-Verteilung, und des Schließens des Generations-Inferenz Kreises. In dieser Arbeit untersuchen wir die Implikationen von jedem dieser Themen in vielfältigen Aufgaben der Bildklassifizierung und Bildgenerierung

Exploring the out-of-school writing practices of three children aged 9 - 10 years old and how these practices travel across and within the domains of home and school

This study explores the writing practices of children aged 9-10 years across the settings of home and school. It examines the nature of the out-of-school writing practices of three case study children, within and across these domains. Additionally, it seeks to understand the children’s relationship with writing and considers if and how their writing practices travel across both domains. In examining home and school writing practices, the study took a sociocultural perspective underpinned by Bronfenbrenner’s (1979) ecological systems theory framework. A review of the relevant literature led to the utilisation of a multimodal definition of writing and the framing of the research as a qualitative, bounded case study within an interpretive, iterative enquiry. The principal research methods were the collation of the children’s writing artefacts, together with video and photographic footage of in-action practices in home settings, school observations and writing conversations. The findings reveal the ways in which these developing young writers engaged and interacted with writing differently in both settings. The trajectory of writing practices across home and school are seen to be in a recursive relationship through the transformation of writing events from one setting to another. Three key themes developed and are presented as metaphors of travel: Places, spaces and local customs; Text souvenirs and local decisions; and Domain exchange and transaction. These themes indicate the range and versatility of the children’s home writing practices. They highlight the complexity in characterising a shared definition of writing across domains. On the basis of the data, the study argues for teachers to be more aware and welcoming of children’s home writing practices in classroom activities. In so doing, teachers would be better able to build on these experiences, leading to new and shared ways of conceptualising writing in English primary classrooms. Finally, the study considers avenues for future research