Advances in Character Recognition

This book presents advances in character recognition, and it consists of 12 chapters that cover wide range of topics on different aspects of character recognition. Hopefully, this book will serve as a reference source for academic research, for professionals working in the character recognition field and for all interested in the subject

System Abstractions for Scalable Application Development at the Edge

Recent years have witnessed an explosive growth of Internet of Things (IoT) devices, which collect or generate huge amounts of data. Given diverse device capabilities and application requirements, data processing takes place across a range of settings, from on-device to a nearby edge server/cloud and remote cloud. Consequently, edge-cloud coordination has been studied extensively from the perspectives of job placement, scheduling and joint optimization. Typical approaches focus on performance optimization for individual applications. This often requires domain knowledge of the applications, but also leads to application-specific solutions. Application development and deployment over diverse scenarios thus incur repetitive manual efforts. There are two overarching challenges to provide system-level support for application development at the edge. First, there is inherent heterogeneity at the device hardware level. The execution settings may range from a small cluster as an edge cloud to on-device inference on embedded devices, differing in hardware capability and programming environments. Further, application performance requirements vary significantly, making it even more difficult to map different applications to already heterogeneous hardware. Second, there are trends towards incorporating edge and cloud and multi-modal data. Together, these add further dimensions to the design space and increase the complexity significantly. In this thesis, we propose a novel framework to simplify application development and deployment over a continuum of edge to cloud. Our framework provides key connections between different dimensions of design considerations, corresponding to the application abstraction, data abstraction and resource management abstraction respectively. First, our framework masks hardware heterogeneity with abstract resource types through containerization, and abstracts away the application processing pipelines into generic flow graphs. Further, our framework further supports a notion of degradable computing for application scenarios at the edge that are driven by multimodal sensory input. Next, as video analytics is the killer app of edge computing, we include a generic data management service between video query systems and a video store to organize video data at the edge. We propose a video data unit abstraction based on a notion of distance between objects in the video, quantifying the semantic similarity among video data. Last, considering concurrent application execution, our framework supports multi-application offloading with device-centric control, with a userspace scheduler service that wraps over the operating system scheduler

Feature design and lexicon reduction for efficient offline handwriting recognition

This thesis establishes a pattern recognition framework for offline word recognition systems. It focuses on the image level features because they greatly influence the recognition performance. In particular, we consider two complementary aspects of prominent features impact: lexicon reduction and the actual recognition. The first aspect, lexicon reduction, consists in the design of a weak classifier which outputs a set of candidate word hypotheses given a word image. Its main purpose is to reduce the recognition computational time while maintaining (or even improving) the recognition rate. The second aspect is the actual recognition system itself. In fact, several features exist in the literature based on different fields of research, but no consensus exists concerning the most promising ones. The goal of the proposed framework is to improve our understanding of relevant features in order to build better recognition systems. For this purpose, we addressed two specific problems: 1) feature design for lexicon reduction (application to Arabic script), and 2) feature evaluation for cursive handwriting recognition (application to Latin and Arabic scripts). Few methods exist for lexicon reduction in Arabic script, unlike Latin script. Existing methods use salient features of Arabic words such as the number of subwords and diacritics, but totally ignore the shape of the subwords. Therefore, our first goal is to perform lexicon reductionn based on subwords shape. Our approach is based on shape indexing, where the shape of a query subword is compared to a labeled database of sample subwords. For efficient comparison with a low computational overhead, we proposed the weighted topological signature vector (W-TSV) framework, where the subword shape is modeled as a weighted directed acyclic graph (DAG) from which the W-TSV vector is extracted for efficient indexing. The main contributions of this work are to extend the existing TSV framework to weighted DAG and to propose a shape indexing approach for lexicon reduction. Good performance for lexicon reduction is achieved for Arabic subwords. Nevertheless, the performance remains modest for Arabic words. Considering the results of our first work on Arabic lexicon reduction, we propose to build a new index for better performance at the word level. The subword shape and the number of subwords and diacritics are all important components of Arabic word shape. We therefore propose the Arabic word descriptor (AWD) which integrates all the aforementioned components. It is built in two steps. First, a structural descriptor (SD) is computed for each connected component (CC) of the word image. It describes the CC shape using the bag-of-words model, where each visual word represents a different local shape structure. Then, the AWD is formed by concatenating the SDs using an efficient heuristic, implicitly discriminating between subwords and diacritics. In the context of lexicon reduction, the AWD is used to index a reference database. The main contribution of this work is the design of the AWD, which integrates lowlevel cues (subword shape structure) and symbolic information (subword counts and diacritics) into a single descriptor. The proposed method has a low computational overhead, it is simple to implement and it provides state-of-the-art performance for lexicon reduction on two Arabic databases, namely the Ibn Sina database of subwords and the IFN/ENIT database of words. The last part of this thesis focuses on features for word recognition. A large body of features exist in the literature, each of them being motivated by different fields, such as pattern recognition, computer vision or machine learning. Identifying the most promising approaches would improve the design of the next generation of features. Nevertheless, because they are based on different concepts, it is difficult to compare them on a theoretical ground and efficient empirical tools are needed. Therefore, the last objective of the thesis is to provide a method for feature evaluation that assesses the strength and complementarity of existing features. A combination scheme has been designed for this purpose, in which each feature is evaluated through a reference recognition system, based on recurrent neural networks. More precisely, each feature is represented by an agent, which is an instance of the recognition system trained with that feature. The decisions of all the agents are combined using a weighted vote. The weights are jointly optimized during a training phase in order to increase the weighted vote of the true word label. Therefore, they reflect the strength and complementarity of the agents and their features for the given task. Finally, they are converted into a numerical score assigned to each feature, which is easy to interpret under this combination model. To the best of our knowledge, this is the first feature evaluation method able to quantify the importance of each feature, instead of providing a ranking based on the recognition rate. Five state-of-the-art features have been tested, and our results provide interesting insight for future feature design

Optimization tools for non-asymptotic statistics in exponential families

Les familles exponentielles sont une classe de modèles omniprésente en statistique. D'une part, elle peut modéliser n'importe quel type de données. En fait la plupart des distributions communes en font partie : Gaussiennes, variables catégoriques, Poisson, Gamma, Wishart, Dirichlet. D'autre part elle est à la base des modèles linéaires généralisés (GLM), une classe de modèles fondamentale en apprentissage automatique. Enfin les mathématiques qui les sous-tendent sont souvent magnifiques, grâce à leur lien avec la dualité convexe et la transformée de Laplace. L'auteur de cette thèse a fréquemment été motivé par cette beauté. Dans cette thèse, nous faisons trois contributions à l'intersection de l'optimisation et des statistiques, qui tournent toutes autour de la famille exponentielle. La première contribution adapte et améliore un algorithme d'optimisation à variance réduite appelé ascension des coordonnées duales stochastique (SDCA), pour entraîner une classe particulière de GLM appelée champ aléatoire conditionnel (CRF). Les CRF sont un des piliers de la prédiction structurée. Les CRF étaient connus pour être difficiles à entraîner jusqu'à la découverte des technique d'optimisation à variance réduite. Notre version améliorée de SDCA obtient des performances favorables comparées à l'état de l'art antérieur et actuel. La deuxième contribution s'intéresse à la découverte causale. Les familles exponentielles sont fréquemment utilisées dans les modèles graphiques, et en particulier dans les modèles graphique causaux. Cette contribution mène l'enquête sur une conjecture spécifique qui a attiré l'attention dans de précédents travaux : les modèles causaux s'adaptent plus rapidement aux perturbations de l'environnement. Nos résultats, obtenus à partir de théorèmes d'optimisation, soutiennent cette hypothèse sous certaines conditions. Mais sous d'autre conditions, nos résultats contredisent cette hypothèse. Cela appelle à une précision de cette hypothèse, ou à une sophistication de notre notion de modèle causal. La troisième contribution s'intéresse à une propriété fondamentale des familles exponentielles. L'une des propriétés les plus séduisantes des familles exponentielles est la forme close de l'estimateur du maximum de vraisemblance (MLE), ou maximum a posteriori (MAP) pour un choix naturel de prior conjugué. Ces deux estimateurs sont utilisés presque partout, souvent sans même y penser. (Combien de fois calcule-t-on une moyenne et une variance pour des données en cloche sans penser au modèle Gaussien sous-jacent ?) Pourtant la littérature actuelle manque de résultats sur la convergence de ces modèles pour des tailles d'échantillons finis, lorsque l'on mesure la qualité de ces modèles avec la divergence de Kullback-Leibler (KL). Pourtant cette divergence est la mesure de différence standard en théorie de l'information. En établissant un parallèle avec l'optimisation, nous faisons quelques pas vers un tel résultat, et nous relevons quelques directions pouvant mener à des progrès, tant en statistiques qu'en optimisation. Ces trois contributions mettent des outil d'optimisation au service des statistiques dans les familles exponentielles : améliorer la vitesse d'apprentissage de GLM de prédiction structurée, caractériser la vitesse d'adaptation de modèles causaux, estimer la vitesse d'apprentissage de modèles omniprésents. En traçant des ponts entre statistiques et optimisation, cette thèse fait progresser notre maîtrise de méthodes fondamentales d'apprentissage automatique.Exponential families are a ubiquitous class of models in statistics. On the one hand, they can model any data type. Actually, the most common distributions are exponential families: Gaussians, categorical, Poisson, Gamma, Wishart, or Dirichlet. On the other hand, they sit at the core of generalized linear models (GLM), a foundational class of models in machine learning. They are also supported by beautiful mathematics thanks to their connection with convex duality and the Laplace transform. This beauty is definitely responsible for the existence of this thesis. In this manuscript, we make three contributions at the intersection of optimization and statistics, all revolving around exponential families. The first contribution adapts and improves a variance reduction optimization algorithm called stochastic dual coordinate ascent (SDCA) to train a particular class of GLM called conditional random fields (CRF). CRF are one of the cornerstones of structured prediction. CRF were notoriously hard to train until the advent of variance reduction techniques, and our improved version of SDCA performs favorably compared to the previous state-of-the-art. The second contribution focuses on causal discovery. Exponential families are widely used in graphical models, and in particular in causal graphical models. This contribution investigates a specific conjecture that gained some traction in previous work: causal models adapt faster to perturbations of the environment. Using results from optimization, we find strong support for this assumption when the perturbation is coming from an intervention on a cause, and support against this assumption when perturbation is coming from an intervention on an effect. These pieces of evidence are calling for a refinement of the conjecture. The third contribution addresses a fundamental property of exponential families. One of the most appealing properties of exponential families is its closed-form maximum likelihood estimate (MLE) and maximum a posteriori (MAP) for a natural choice of conjugate prior. These two estimators are used almost everywhere, often unknowingly -- how often are mean and variance computed for bell-shaped data without thinking about the Gaussian model they underly? Nevertheless, literature to date lacks results on the finite sample convergence property of the information (Kulback-Leibler) divergence between these estimators and the true distribution. Drawing on a parallel with optimization, we take some steps towards such a result, and we highlight directions for progress both in statistics and optimization. These three contributions are all using tools from optimization at the service of statistics in exponential families: improving upon an algorithm to learn GLM, characterizing the adaptation speed of causal models, and estimating the learning speed of ubiquitous models. By tying together optimization and statistics, this thesis is taking a step towards a better understanding of the fundamentals of machine learning

Machine learning in the real world with multiple objectives

Machine learning (ML) is ubiquitous in many real-world applications. Existing ML systems are based on optimizing a single quality metric such as prediction accuracy. These metrics typically do not fully align with real-world design constraints such as computation, latency, fairness, and acquisition costs that we encounter in real-world applications. In this thesis, we develop ML methods for optimizing prediction accuracy while accounting for such real-world constraints. In particular, we introduce multi-objective learning in two different setups: resource-efficient prediction and algorithmic fairness in language models. First, we focus on decreasing the test-time computational costs of prediction systems. Budget constraints arise in many machine learning problems. Computational costs limit the usage of many models on small devices such as IoT or mobile phones and increase the energy consumption in cloud computing. We design systems that allow on-the-fly modification of the prediction model for each input sample. These sample-adaptive systems allow us to leverage wide variability in sample complexity where we learn policies for selecting cheap models for low complexity instances and using descriptive models only for complex ones. We utilize multiple--objective approach where one minimizes the system cost while preserving predictive accuracy. We demonstrate significant speed-ups in the fields of computer vision, structured prediction, natural language processing, and deep learning. In the context of fairness, we first demonstrate that a naive application of ML methods runs the risk of amplifying social biases present in data. This danger is particularly acute for methods based on word embeddings, which are increasingly gaining importance in many natural language processing applications of ML. We show that word embeddings trained on Google News articles exhibit female/male gender stereotypes. We demonstrate that geometrically, gender bias is captured by unique directions in the word embedding vector space. To remove bias we formulate a empirical risk objective with fairness constraints to remove stereotypes from embeddings while maintaining desired associations. Using crowd-worker evaluation as well as standard benchmarks, we empirically demonstrate that our algorithms significantly reduces gender bias in embeddings, while preserving its useful properties such as the ability to cluster related concepts

APOE AS A METABOLIC REGULATOR IN HUMANS, MICE, AND ASTROCYTES

Altered metabolic pathways appear to play central roles in the pathophysiology of late-onset Alzheimer’s disease (AD). Carrier status of the E4 allele of the APOE gene is the strongest genetic risk factor for late-onset AD, and increasing evidence suggests that E4 carriers may be at an increased risk for neurodegeneration based on inherent metabolic impairments. A new appreciation is forming for the role of APOE in cerebral metabolism, and how nutritional factors may impact this role. In chapter 1, the literature on nutritional interventions in E4 carriers aimed at mitigating disease risk is reviewed. Studies investigating the mechanism by which E4 increases disease risk have focused primarily on the association of E4 with the neuropathological hallmarks. While these studies have aided in our understanding of the role of E4 in late-disease pathology, investigating metabolic signatures of E4 carriers who have not yet developed neuropathology gives insight into the potential earlier mechanisms of E4 as a risk factor for AD. For example, an early and consistent biological hallmark of AD is cerebral glucose hypometabolism as observed by fluorodeoxyglucose positron emission tomography (FDG-PET). Interestingly, E4 carriers also display an AD-like pattern of decreased glucose metabolism by FDG-PET far before clinical symptomology. Since glucose hypometabolism occurs early in AD and early in E4 carriers, it may represent a critical prodromal phase of AD. Beyond this brain imaging finding, it is unclear if APOE has any other discernable metabolic effects in cognitively unimpaired young people. In chapter 2 we bridge this knowledge gap in the field. We utilized indirect calorimetry (IC) as a method for assessing metabolism in young and middle aged volunteers with and without the E4 allele. While IC is commonly used in clinical settings to assess nutritional status, it has never been used to assess risk for cognitive decline. Thus, repurposing IC to study the metabolic effects of an AD risk factor such as E4 represents a simple, cost-effective, and innovative new approach. We found that young female E4 carriers show a lower resting energy expenditure compared to non-carriers. We also tested how E4 carriage affects response to a glucose challenge by administering a glucose rich beverage in conjunction with IC measurements and plasma metabolomics. We found that female E4 carriers were unable to increase oxygen consumption relative to non-carriers, reflecting an impairment in glucose oxidation. Additionally, the plasma metabolome of E4 carriers showed increased lactate and an overall metabolic profile consistent with aerobic glycolysis. We translated these findings to mice expressing the human alleles of APOE. We found that E4 mice on a normal chow diet have lower energy expenditure than E3 mice, a difference further exacerbated by high carbohydrate diet feeding. Stable isotope tracing in mice whole brains and astrocytes implicate increased utilization of aerobic glycolysis as a mechanism for altered glucose handling in E4 carriers. Another pathological feature of the Alzheimer’s brain is glial lipid accumulation. The mechanism for this is largely unknown. In chapter 3, the literature pertaining to lipid droplets (LD) in the brain is reviewed. We show that LDs are much more than simple fat depots, playing critical roles in metabolism, inflammation, and various neurodegenerative diseases. In chapter 4, the effect of the E4 allele on astrocyte LD accumulation and turnover is assessed. Using an in vitro model of APOE we probed the storage and oxidation capacity of fatty acids in E3 and E4 astrocytes. We observed that E4 astrocytes exhibit greater storage of fatty acids as LDs under control and lipid loaded conditions compared to E3 astrocytes. Furthermore, we found that E4 astrocytes rely on these LDs as a source of fuel for oxidation. Therefore, APOE appears to regulate whole body energy expenditure, cerebral glucose oxidation, astrocyte LD metabolism, and risk for a host of metabolic diseases. In chapter 5, the evolutionary history of APOE is presented to posit a hypothesis for why E4 may be disadvantageous in modern times compared to its prior advantages in the pre-historic era. These results point toward a larger role for APOE in the regulation of metabolism than previously understood and advocates for alternative nutritional approaches including calorie reduction and intermittent fasting as plausible interventions to mitigate disease risk in E4 carriers

The role of macrophage intracellular lipid partitioning in glucose and lipid homeostasis during obesity

Obesity-associated metabolic disorders are amongst the most prevalent causes of death worldwide. Understanding how obesity leads to the development of the Metabolic Syndrome (MetS) and cardiovascular disease (CVD) will enable the development of novel therapies that dissociate obesity from its cardiometabolic complications. Our laboratory views the functional capacity of white adipose tissue (WAT), the organ designed for safe lipid storage, as a key factor in the development of MetS and CVD. At a genetically-defined stage of the aberrant WAT expansion that occurs during obesity, adipocytes undergo a functional failure, resulting in an impaired control of serum free fatty acid (FFA) concentration. In such setting, FFAs and their metabolic derivatives accumulate in other organs, where they cause lipotoxicity, leading to the development of insulin resistance and CVD. We therefore aim to understand the pathophysiological mechanisms that induce adipocyte dysfunction. The past two decades of research have established the immune system as an important regulator of WAT function. The number of adipose tissue macrophages (ATMs), the most abundant immune cell type in WAT, increases during obesity, resulting in WAT inflammation. Multiple genetic and pharmacological intervention studies of murine models of obesity have assigned a causal link between ATM pro-inflammatory activation and WAT dysfunction. However, while the propagation of inflammation in ATMs during obesity has been extensively studied, factors triggering ATM inflammatory activation are less clear. Recently, our lab has observed lipid accumulation in the ATMs isolated from obese mice. Lipid-laden ATMs were pro-inflammatory, leading us to hypothesise that aberrant lipid build-up in macrophages triggers WAT inflammation during obesity. This thesis expands on the initial findings from our lab and describes two novel mechanisms that potentially contribute to lipid-induced inflammatory activation of ATMs. In chapter 3, the role of de novo phosphatidylcholine (PC) synthesis pathway during lipotoxicity in macrophages is addressed. The first part of the chapter demonstrates that lipotoxic environment increased de novo PC synthesis rate in bone marrow-derived macrophages (BMDMs) and ATMs, and that loss of rate-limiting enzyme in de novo PC synthesis pathway, CTP:phosphocholine cytidylyltransferase a (CCTa) diminished saturated FFA-induced inflammation in BMDMs. In the second part, I show that macrophage-specific CCTa deletion did not impact on the development of WAT inflammation or systemic insulin resistance, but had a minor benefitial effect on hepatic gene transcription during obesity. Chapter 4 develops on recent observations of interactions between sympathetic nerves and macrophages in WAT. In the first part of the chapter, I demonstrate that stimulating B2-adrenergic receptor (B2AR), the main receptor for sympathetic neurotransmitter norepinephrine in macrophages, enhanced intracellular triglyceride storage by up-regulating diacylglycerol O-acyltransferase 1 (Dgat1) gene expression in BMDMs. The second part of the chapter shows that macrophage-specific B2AR deletion did not modulate systemic glucose and lipid metabolism during obesity, but mice lacking B2ARs in macrophages demonstrated augmented hepatic glucose production on a chow diet. Furthermore, systemic B2AR blockade or macrophage-specific B2AR deletion in mice did not affect the thermogenic response to cold exposure. Chapter 5 includes the characterisation of B2AR stimulation-induced changes to the global cellular proteome of BMDMs, and a subsequent validation of the role of candidate transcription factors in regulating B2AR agonism-induced gene expression in BMDMs.Wellcome Trust 4 year PhD programm

Distributed Learning, Prediction and Detection in Probabilistic Graphs.

Critical to high-dimensional statistical estimation is to exploit the structure in the data distribution. Probabilistic graphical models provide an efficient framework for representing complex joint distributions of random variables through their conditional dependency graph, and can be adapted to many high-dimensional machine learning applications. This dissertation develops the probabilistic graphical modeling technique for three statistical estimation problems arising in real-world applications: distributed and parallel learning in networks, missing-value prediction in recommender systems, and emerging topic detection in text corpora. The common theme behind all proposed methods is a combination of parsimonious representation of uncertainties in the data, optimization surrogate that leads to computationally efficient algorithms, and fundamental limits of estimation performance in high dimension. More specifically, the dissertation makes the following theoretical contributions: (1) We propose a distributed and parallel framework for learning the parameters in Gaussian graphical models that is free of iterative global message passing. The proposed distributed estimator is shown to be asymptotically consistent, improve with increasing local neighborhood sizes, and have a high-dimensional error rate comparable to that of the centralized maximum likelihood estimator. (2) We present a family of latent variable Gaussian graphical models whose marginal precision matrix has a “low-rank plus sparse” structure. Under mild conditions, we analyze the high-dimensional parameter error bounds for learning this family of models using regularized maximum likelihood estimation. (3) We consider a hypothesis testing framework for detecting emerging topics in topic models, and propose a novel surrogate test statistic for the standard likelihood ratio. By leveraging the theory of empirical processes, we prove asymptotic consistency for the proposed test and provide guarantees of the detection performance.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/110499/1/mengzs_1.pd

Semantic knowledge integration for learning from semantically imprecise data

Low availability of labeled training data often poses a fundamental limit to the accuracy of computer vision applications using machine learning methods. While these methods are improved continuously, e.g., through better neural network architectures, there cannot be a single methodical change that increases the accuracy on all possible tasks. This statement, known as the no free lunch theorem, suggests that we should consider aspects of machine learning other than learning algorithms for opportunities to escape the limits set by the available training data. In this thesis, we focus on two main aspects, namely the nature of the training data, where we introduce structure into the label set using concept hierarchies, and the learning paradigm, which we change in accordance with requirements of real-world applications as opposed to more academic setups.Concept hierarchies represent semantic relations, which are sets of statements such as "a bird is an animal." We propose a hierarchical classifier to integrate this domain knowledge in a pre-existing task, thereby increasing the information the classifier has access to. While the hierarchy's leaf nodes correspond to the original set of classes, the inner nodes are "new" concepts that do not exist in the original training data. However, we pose that such "imprecise" labels are valuable and should occur naturally, e.g., as an annotator's way of expressing their uncertainty. Furthermore, the increased number of concepts leads to more possible search terms when assembling a web-crawled dataset or using an image search. We propose CHILLAX, a method that learns from semantically imprecise training data, while still offering precise predictions to integrate seamlessly into a pre-existing application