Task-specific and interpretable feature learning

Deep learning models have had tremendous impacts in recent years, while a question has been raised by many: Is deep learning just a triumph of empiricism? There has been emerging interest in reducing the gap between the theoretical soundness and interpretability, and the empirical success of deep models. This dissertation provides a comprehensive discussion on bridging traditional model-based learning approaches that emphasize problem-specific reasoning, and deep models that allow for larger learning capacity. The overall goal is to devise the next-generation feature learning architectures that are: 1) task-specific, namely, optimizing the entire pipeline from end to end while taking advantage of available prior knowledge and domain expertise; and 2) interpretable, namely, being able to learn a representation consisting of semantically sensible variables, and to display predictable behaviors. This dissertation starts by showing how the classical sparse coding models could be improved in a task-specific way, by formulating the entire pipeline as bi-level optimization. Then, it mainly illustrates how to incorporate the structure of classical learning models, e.g., sparse coding, into the design of deep architectures. A few concrete model examples are presented, ranging from the $\ell_0$ and $\ell_1$ sparse approximation models, to the $\ell_\infty$ constrained model and the dual-sparsity model. The analytic tools in the optimization problems can be translated to guide the architecture design and performance analysis of deep models. As a result, those customized deep models demonstrate improved performance, intuitive interpretation, and efficient parameter initialization. On the other hand, deep networks are shown to be analogous to brain mechanisms. They exhibit the ability to describe semantic content from the primitive level to the abstract level. This dissertation thus also presents a preliminary investigation of the synergy between feature learning with cognitive science and neuroscience. Two novel application domains, image aesthetics assessment and brain encoding, are explored, with promising preliminary results achieved

Understanding human-centric images : from geometry to fashion

Understanding humans from photographs has always been a fundamental goal of computer vision. Early works focused on simple tasks such as detecting the location of individuals by means of bounding boxes. As the field progressed, harder and more higher level tasks have been undertaken. For example, from human detection came the 2D and 3D human pose estimation in which the task consisted of identifying the location in the image or space of all different body parts, e.g., head, torso, knees, arms, etc. Human attributes also became a great source of interest as they allow recognizing individuals and other properties such as gender or age. Later, the attention turned to the recognition of the action being performed. This, in general, relies on the previous works on pose estimation and attribute classification. Currently, even higher level tasks are being conducted such as predicting the motivations of human behavior or identifying the fashionability of an individual from a photograph. In this thesis we have developed a hierarchy of tools that cover all these range of problems, from low level feature point descriptors to high level fashion-aware conditional random fields models, all with the objective of understanding humans from monocular, RGB images. In order to build these high level models it is paramount to have a battery of robust and reliable low and mid level cues. Along these lines, we have proposed two low-level keypoint descriptors: one based on the theory of the heat diffusion on images, and the other that uses a convolutional neural network to learn discriminative image patch representations. We also introduce distinct low-level generative models for representing human pose: in particular we present a discrete model based on a directed acyclic graph and a continuous model that consists of poses clustered on a Riemannian manifold. As mid level cues we propose two 3D human pose estimation algorithms: one that estimates the 3D pose given a noisy 2D estimation, and an approach that simultaneously estimates both the 2D and 3D pose. Finally, we formulate higher level models built upon low and mid level cues for human understanding. Concretely, we focus on two different tasks in the context of fashion: semantic segmentation of clothing, and predicting the fashionability from images with metadata to ultimately provide fashion advice to the user. In summary, to robustly extract knowledge from images with the presence of humans it is necessary to build high level models that integrate low and mid level cues. In general, using and understanding strong features is critical for obtaining reliable performance. The main contribution of this thesis is in proposing a variety of low, mid and high level algorithms for human-centric images that can be integrated into higher level models for comprehending humans from photographs, as well as tackling novel fashion-oriented problems.Siempre ha sido una meta fundamental de la visión por computador la comprensión de los seres humanos. Los primeros trabajos se fijaron en objetivos sencillos tales como la detección en imágenes de la posición de los individuos. A medida que la investigación progresó se emprendieron tareas mucho más complejas. Por ejemplo, a partir de la detección de los humanos se pasó a la estimación en dos y tres dimensiones de su postura por lo que la tarea consistía en identificar la localización en la imagen o el espacio de las diferentes partes del cuerpo, por ejemplo cabeza, torso, rodillas, brazos, etc...También los atributos humanos se convirtieron en una gran fuente de interés ya que permiten el reconocimiento de los individuos y de sus propiedades como el género o la edad. Más tarde, la atención se centró en el reconocimiento de la acción realizada. Todos estos trabajos reposan en las investigaciones previas sobre la estimación de las posturas y la clasificación de los atributos. En la actualidad, se llevan a cabo investigaciones de un nivel aún superior sobre cuestiones tales como la predicción de las motivaciones del comportamiento humano o la identificación del tallaje de un individuo a partir de una fotografía. En esta tesis desarrollamos una jerarquía de herramientas que cubre toda esta gama de problemas, desde descriptores de rasgos de bajo nivel a modelos probabilísticos de campos condicionales de alto nivel reconocedores de la moda, todos ellos con el objetivo de mejorar la comprensión de los humanos a partir de imágenes RGB monoculares. Para construir estos modelos de alto nivel es decisivo disponer de una batería de datos robustos y fiables de nivel bajo y medio. En este sentido, proponemos dos descriptores novedosos de bajo nivel: uno se basa en la teoría de la difusión de calor en las imágenes y otro utiliza una red neural convolucional para aprender representaciones discriminativas de trozos de imagen. También introducimos diferentes modelos de bajo nivel generativos para representar la postura humana: en particular presentamos un modelo discreto basado en un gráfico acíclico dirigido y un modelo continuo que consiste en agrupaciones de posturas en una variedad de Riemann. Como señales de nivel medio proponemos dos algoritmos estimadores de la postura humana: uno que estima la postura en tres dimensiones a partir de una estimación imprecisa en el plano de la imagen y otro que estima simultáneamente la postura en dos y tres dimensiones. Finalmente construimos modelos de alto nivel a partir de señales de nivel bajo y medio para la comprensión de la persona a partir de imágenes. En concreto, nos centramos en dos diferentes tareas en el ámbito de la moda: la segmentación semántica del vestido y la predicción del buen ajuste de la prenda a partir de imágenes con meta-datos con la finalidad de aconsejar al usuario sobre moda. En resumen, para extraer conocimiento a partir de imágenes con presencia de seres humanos es preciso construir modelos de alto nivel que integren señales de nivel medio y bajo. En general, el punto crítico para obtener resultados fiables es el empleo y la comprensión de rasgos fuertes. La aportación fundamental de esta tesis es la propuesta de una variedad de algoritmos de nivel bajo, medio y alto para el tratamiento de imágenes centradas en seres humanos que pueden integrarse en modelos de alto nivel, para mejor comprensión de los seres humanos a partir de fotografías, así como abordar problemas planteados por el buen ajuste de las prendas

Beyond Dichotomy: Dynamics of Choice in Compositional Space

The quantitative study of choice under conditions of uncertainty dates back to the earliest applications of probability to games of chance. Over time, theories of choice have transitioned away from the `oughts' of rational econometrics toward more face-valid descriptions of observed behaviors. Throughout this period, the problem of subjective probability has posed a consistent difficulty for theories of choice. The most successful approach for modeling these distortions is use of `log-odds,' which provides a powerful description of two-alternative choice as a power law function of relative outcome probability. The log-odds approach can be generalized using the framework of `compositional analysis.' The core statistical methodology of this framework is introduced and described, with an eye towards developing models of choice across any number of alternatives. The viability of these models is demonstrated on several previously published datasets. A series of experiments with rats explored the effect of changing the number of alternatives. Power-law models continued to provide an effective description of behavior, but subjective probabilities were also found to be less distorted when subjects made choices among a larger number of alternatives (eight at once) than among smaller numbers (four or six). This effect was robust against controls for age, order of experience, chamber configuration, and schedule richness. A working hypothesis is put forward based on an analysis of responses as a dynamical process: Subjects succeed at complex tasks by limiting their transitions between response alternatives to a highly stereotyped `default transition matrix,' making only slight deviations in order to adapt to changing task demands. This strategy is computationally efficient. However, severe mismatches between the schedule and a subject's default transition matrix are much more likely to occur when fewer alternatives are available, and behavior under such conditions is necessarily insensitive. Implications for other choice models are considered

Principles of Neural Network Architecture Design - Invertibility and Domain Knowledge

Neural networks architectures allow a tremendous variety of design choices. In this work, we study two principles underlying these architectures: First, the design and application of invertible neural networks (INNs). Second, the incorporation of domain knowledge into neural network architectures. After introducing the mathematical foundations of deep learning, we address the invertibility of standard feedforward neural networks from a mathematical perspective. These results serve as a motivation for our proposed invertible residual networks (i-ResNets). This architecture class is then studied in two scenarios: First, we propose ways to use i-ResNets as a normalizing flow and demonstrate the applicability for high-dimensional generative modeling. Second, we study the excessive invariance of common deep image classifiers and discuss consequences for adversarial robustness. We finish with a study of convolutional neural networks for tumor classification based on imaging mass spectrometry (IMS) data. For this application, we propose an adapted architecture guided by our knowledge of the domain of IMS data and show its superior performance on two challenging tumor classification datasets

Untangling hotel industry’s inefficiency: An SFA approach applied to a renowned Portuguese hotel chain

The present paper explores the technical efficiency of four hotels from Teixeira Duarte Group - a renowned Portuguese hotel chain. An efficiency ranking is established from these four hotel units located in Portugal using Stochastic Frontier Analysis. This methodology allows to discriminate between measurement error and systematic inefficiencies in the estimation process enabling to investigate the main inefficiency causes. Several suggestions concerning efficiency improvement are undertaken for each hotel studied.info:eu-repo/semantics/publishedVersio

Statistical methods and applications to animal breeding

This thesis comprises a collection of 39 research papers divided into three groups. The first group discusses the development of statistical methods, especially novel methods of variance component estimation, with general application. The second group examines the potential use of statistical methods in animal breeding studies, ranging from the construction of new experimental designs to the analysis of non-normal data. The third group reports on studies on animal breeding data in beef and dairy cattle.Group I is entitled "Statistical methods, including variance component estimation, with general application". The major theme of this group is the estimation of variance components. Some previous work based on methods for balanced data, gave rise to methods that were neither unique nor efficient and other methods gave results that are inconsistent with the analysis of variance for balanced data. A method was introduced, now known as REML (Residual Maximum Likelihood) that unifies the area. The method was introduced for the analysis of incomplete block designs with unequal block size but was found to have important applications in the analysis of groups of similar trials, time-series and animal breeding. Papers investigating REML estimation for multivariate data, time-series and detecting outliers are included. The relationship of REML to other methods is elucidated, especially for balanced and partially balanced designs. Computational strategies are discussed.The last two papers in the group illustrate a method of analysis of dial lei crosses that involves using multiple copies of the data. This idea of using multiple copies was shown also to be useful in the analysis of rectangular lattice designs and in the interpretation of some recently introduced neighbour analyses of field trials.The next group of papers, Group II, report on "Application of statistical methods to animal breeding studies". The work on variance components has some application in animal breeding and I have built on these links. Four papers consider efficient designs for estimation of genetic parameters, including designs for estimating heritability from data on two generations of data, for estimating maternal genetic variances, for estimating parent-offspri ng regression and for estimating multivariate genetic parameters. These designs can lead to substantial reductions in the variances of the estimates of the parameters, compared with classical designs, halving variances in some cases. Other papers have shown how to efficiently estimate heritability from unbalanced data, both from two generations of data and from more than two generations.Often in animal breeding experiments animals used as parents are not selected at random, but selected on phenotypic measurements, perhaps of relatives. This can cause bias in some methods of estimation. On the other hand REML estimates can take account of the selection process. Selection experiments and the estimation of realised heritability are discussed.REML estimation has found widespread acceptance by animal breeders, partly because some quantities arising in the methods were terms that animal breeders use in evaluating animals. It was shown how to improve one method of evaluation and methods of evaluating sires were reviewedSome work is included on multivariate evaluation. It is shown how the complex multivariate calculations can be reduced to simpler univariate calculations using a canonical transformation, how results on selection indices can be used to interpret multivariate predictions. A simple interpretation of quadratic selection indices is given.Other work considered some parallel problems with non-normal data. In particular for binary data, estimation of heritability, optimal designs for estimation of heritability and prediction of breeding values. It was shown how to estimate genotype frequencies using generalised linear model methods and > h? suggested how to evaluate animals worth and estimate genetic parameters when the data fits a generalised linear model.The last group, Group III, is entitled "Experimental studies". These include reports on a long term study of evaluation of breeds and cross-breeding in beef cattle in Zambia. The section also examines the genetic relationship between meat and milk production in British Friesian cattle. The validity of models used in dairy sire evaluation are investigated including the heterogeneity of heritability of milk yield at different levels of production and the use of a novel model for taking account of environmental variation within herds.GROUP I: STATISTICAL METHODS INCLUDING VARIANCE COMPONENT ESTIMATE WITH GENERAL APPLICATION01. R. THOMPSON. 1969. Iterative estimation of variance components for non-orthogonal data. Biometrics 25, 767-773. || 02. H.D. PATTERSON and R. THOMPSON. 1971. Recovery of inter-block information when block sizes are unequal. Biometrika 58, 545-554. || 03. H.D. PATTERSON and R. THOMPSON. 1975. Maximum likelihood estimation of components of variance. Proceedings of the 8th International Biometric Conference. Ed. L.C.A. Corsten and T. Postelnicu, 199-207. || 04. R. THOMPSON. 1980. Maximum likelihood estimation of variance components. Math. Operationsforsh. Statist. ljU 545-561. || 05. R. THOMPSON. 1978. The estimation of variance and covariance components with an application when records are subject to culling. Biometrics 29, 527-550. || 06. L.R. SCHAEFFER, J.W. WILTON and R. THOMPSON. 1978. Simultaneous estimation of variance and covariance components from multitrait mixed model equations. Biometrics 34, 199-208. || 07. D.M. COOPER and R. THOMPSON. 1977 . A note on the estimation of the parameters of the autoregressive-moving average process. Biometrika 64, 625-628. || 08. R. THOMPSON. 1985. A note on restricted maximum likelihood estimation with an alternative outlier model. J.R. Statist. Soc. B 47, 53-55. || 09. R. THOMPSON. 1975. A note on the W transformation. Technometrics J7, 511-512. || 10. R. THOMPSON and K. MEYER. 1986. Estimation of variance components : what is missing in the EM algorithm? J. Statist. Comput. Simul. 24 215-230. || 11. D.L. ROBINSON, R. THOMPSON and P.G.N. DIGBY. REML. 1982. A program for the analysis of non-orthogonal data by restricted maximum likelihood. COMPSTAT 1982, II. Eds. H. Cassinus, P. Ettinger and J.R. Mattieu. Physica-Verlag, Wien 231-232. || 12. R. THOMPSON. 1983. Dial lei crosses, partially balanced incomplete block designs with triangular association schemes and rectangular lattices. GENSTAT newsletter JJ3, 16-32. || 13. R. THOMPSON. 1984. The use of multiple copies of data in forming and interpreting analysis of variance. Experimental design, Statistical Methods and Genetic Statistics. Ed. K. Hinkelmann. Marcel Dekker, New York, 155-174.GROUP II: APPLICATION OF STATISTICAL METHODS TO ANIMAL BREEDING STUDIES14. R. THOMPSON. 1976. The estimation of maternal genetic variances. Biometrics 32 903-917. || 15. R. THOMPSON. 1976. Design of experiments to estimate heritability when observations are available on parents and offspring. Biometrics 32 283-304. || 16. W.G. HILL and R. THOMPSON. 1977. Design of experiments to estimate parent-offspring regression using selected parents. Anim. Prod. 24, 163-168. || 17. N.D. CAMERON and R. THOMPSON. 1986. Design of multivariate selection experiments to estimate genetic parameters. Theor. Appl. Genet. 72, 466-476. || 18. R. THOMPSON. 1977. The estimation of heritability with unbalanced data. I. Observations available on parents and offspring. Biometrics 33, 485-495. || 19. R. THOMPSON. 1977. The estimation of heritability with unbalanced data. II. Data available on more than two generations. Biometrics 33, 495-504. || 20. R. THOMPSON. 1977. The estimation of heritability with unbalanced data. III. Unpublished Appendices, 1-17. || 21. R. THOMPSON. 1976. Estimation of quantitative genetic parameters. Proceedings of the International Conference on Quantitative Genetics. Ed. 0. Kempthorne, E. Pollak and T. Bailey. Iowa State University press, Ames, Iowa, 639-657. (vii) || 22. W.G. HILL and R. THOMPSON. 1978. Probabilities of non-positive definite between group or genetic covariance matrices. Biometrics 34, 429-439. || 23. K. MEYER and R. THOMPSON. 1984. Bias in variance and covariance component estimators due to selection on a correlated trait. Z. Tierzucht. Zuchtungsbiol. 101, 33-50. || 24. R. THOMPSON. 1976. Relationship between the cumulative different and best linear unbiased predictor methods of evaluating bulls. Anim. Prod. 23^, 15-24. || 25. R. THOMPSON. 1979. Sire Evaluation. Biometrics 35, 339-353. || 26. R. THOMPSON. 1986. Estimation of realised heritability in a selected population using mixed model methods. Genet. Sel. Evol . 475-484. || 27. R. THOMPSON. 1972. The maximum likelihood approach to the estimate of liability. Anim. Hum. Genet. 36, 221-231. || 28. R. THOMPSON, B.J. McGUIRK and A.R. GILMOUR. 1985. Estimating the heritability of all-or-none and categorical traits by offspring-parent regression. Z. Tierzucht. Zuchtungsbiol. 102, 342-354. || 29. J.L. FOULLEY, D. GIANOLA and R. THOMPSON. 1983. Prediction of genetic merit from data on binary and quantitative variates with an application to calving difficulty, birth weight and pelvic opening. Genet. Sel. Evol. 15, 401-424. || 30. R. THOMPSON and R.J. BAKER. 1981. Composite link functions in generalised linear models. J.R. Statist. Soc. B. 30, 125-131. || 31. R. THOMPSON. 1980. A note on the estimation of economic values for selection indices. Anim. Prod. 31, 115-117.GROUP III: EXPERIMENTAL STUDIES32. W. THORPE, D.K.R. CRUICKSHANK and R. THOMPSON. 1980. Genetic and evironmental influences on beef cattle production in Zambia. Factors affecting weaner production from Angoni, Barotse and Boran dams. Anim. Prod. 30, 217-234. || 33. W. THORPE, D.K.R. CRUICKSHANK and R. THOMPSON. 1980. Genetic and environmental influences on beef cattle production in Zambia. 2. Sire weights for age of purebred and reciprocally crossbred progeny. Anim. Prod. 30, 235-243. || 34. W. THORPE, D.K.R. CRUICKSHANK and R. THOMPSON. 1980. Genetic and environmental influences on beef cattle production in Zambia. 3. Carcass characteristics of purebred and reciprocally crossbred progeny. Anim. Prod. 30, 245-252. || 35. W. THORPE, D.C.K. CRUICKSHANK and R. THOMPSON. 1982. Genetic and environmental influences on beef cattle in Zambia. 4. Weaner production from purebred and reciprocally crossbred progeny. Anim. Prod. 33, 165-177. || 36. W. THORPE, D.K.R. CRUICKSHANK and R. THOMPSON. 1979. The growth and carcass character!- sti cs of crosses of Hereford and Friesian with Angoni, Barotse and Boran cattle in Zambia. J. Agric. Sci., Camb. 93, 423-430. || 37. I.L. MASON, V.E. VIAL and R. THOMPSON. 1972. Genetic parameters of beef characteristics and the genetic relationship between meat and milk production in British Friesian cattle. Anim. Prod. 135-148. || 38. W.G. HILL, M.R. EDWARDS, M-K A. AHMED and R. THOMPSON. 1983. Heritability of milk yield and composition at different levels and variability of production. Anim. Prod. 36, 59-68. || 39. V.P.S. CHAUHAN and R. THOMPSON. 1986. Dairy sire evaluation using a "rolling months" model. Z. Tierzucht. Zuchtungsbiol 103, 321-333

Multivariate Statistical Machine Learning Methods for Genomic Prediction

This book is open access under a CC BY 4.0 license This open access book brings together the latest genome base prediction models currently being used by statisticians, breeders and data scientists. It provides an accessible way to understand the theory behind each statistical learning tool, the required pre-processing, the basics of model building, how to train statistical learning methods, the basic R scripts needed to implement each statistical learning tool, and the output of each tool. To do so, for each tool the book provides background theory, some elements of the R statistical software for its implementation, the conceptual underpinnings, and at least two illustrative examples with data from real-world genomic selection experiments. Lastly, worked-out examples help readers check their own comprehension. The book will greatly appeal to readers in plant (and animal) breeding, geneticists and statisticians, as it provides in a very accessible way the necessary theory, the appropriate R code, and illustrative examples for a complete understanding of each statistical learning tool. In addition, it weighs the advantages and disadvantages of each tool