Semantic Attributes for Transfer Learning in Visual Recognition

Angetrieben durch den Erfolg von Deep Learning Verfahren wurden in Bezug auf künstliche Intelligenz erhebliche Fortschritte im Bereich des Maschinenverstehens gemacht. Allerdings sind Tausende von manuell annotierten Trainingsdaten zwingend notwendig, um die Generalisierungsfähigkeit solcher Modelle sicherzustellen. Darüber hinaus muss das Modell jedes Mal komplett neu trainiert werden, sobald es auf eine neue Problemklasse angewandt werden muss. Dies führt wiederum dazu, dass der sehr kostenintensive Prozess des Sammelns und Annotierens von Trainingsdaten wiederholt werden muss, wodurch die Skalierbarkeit solcher Modelle erheblich begrenzt wird. Auf der anderen Seite bearbeiten wir Menschen neue Aufgaben nicht isoliert, sondern haben die bemerkenswerte Fähigkeit, auf bereits erworbenes Wissen bei der Lösung neuer Probleme zurückzugreifen. Diese Fähigkeit wird als Transfer-Learning bezeichnet. Sie ermöglicht es uns, schneller, besser und anhand nur sehr weniger Beispiele Neues zu lernen. Daher besteht ein großes Interesse, diese Fähigkeit durch Algorithmen nachzuahmen, insbesondere in Bereichen, in denen Trainingsdaten sehr knapp oder sogar nicht verfügbar sind. In dieser Arbeit untersuchen wir Transfer-Learning im Kontext von Computer Vision. Insbesondere untersuchen wir, wie visuelle Erkennung (z.B. Objekt- oder Aktionsklassifizierung) durchgeführt werden kann, wenn nur wenige oder keine Trainingsbeispiele existieren. Eine vielversprechende Lösung in dieser Richtung ist das Framework der semantischen Attribute. Dabei werden visuelle Kategorien in Form von Attributen wie Farbe, Muster und Form beschrieben. Diese Attribute können aus einer disjunkten Menge von Trainingsbeispielen gelernt werden. Da die Attribute eine doppelte, d.h. sowohl visuelle als auch semantische, Interpretation haben, kann Sprache effektiv genutzt werden, um den Übertragungsprozess zu steuern. Dies bedeutet, dass Modelle für eine neue visuelle Kategorie nur anhand der sprachlichen Beschreibung erstellt werden können, indem relevante Attribute selektiert und auf die neue Kategorie übertragen werden. Die Notwendigkeit von Trainingsbildern entfällt durch diesen Prozess jedoch vollständig. In dieser Arbeit stellen wir neue Lösungen vor, semantische Attribute zu modellieren, zu übertragen, automatisch mit visuellen Kategorien zu assoziieren, und aus sprachlichen Beschreibungen zu erkennen. Zu diesem Zweck beleuchten wir die attributbasierte Erkennung aus den folgenden vier Blickpunkten: 1) Anders als das gängige Modell, bei dem Attribute global gelernt werden müssen, stellen wir einen hierarchischen Ansatz vor, der es ermöglicht, die Attribute auf verschiedenen Abstraktionsebenen zu lernen. Wir zeigen zudem, wie die Struktur zwischen den Kategorien effektiv genutzt werden kann, um den Lern- und Transferprozess zu steuern und damit diskriminative Modelle für neue Kategorien zu erstellen. Mit einer gründlichen experimentellen Analyse demonstrieren wir eine deutliche Verbesserung unseres Modells gegenüber dem globalen Ansatz, insbesondere bei der Erkennung detailgenauer Kategorien. 2) In vorherrschend attributbasierten Transferansätzen überwacht der Benutzer die Zuordnung zwischen den Attributen und den Kategorien. Wir schlagen in dieser Arbeit vor, die Verbindung zwischen den beiden automatisch und ohne Benutzereingriff herzustellen. Unser Modell erfasst die semantischen Beziehungen, welche die Attribute mit Objekten koppeln, um ihre Assoziationen vorherzusagen und unüberwacht auszuwählen welche Attribute übertragen werden sollen. 3) Wir umgehen die Notwendigkeit eines vordefinierten Vokabulars von Attributen. Statt dessen schlagen wir vor, Enyzklopädie-Artikel zu verwenden, die Objektkategorien in einem freien Text beschreiben, um automatisch eine Menge von diskriminanten, salienten und vielfältigen Attributen zu entdecken. Diese Beseitigung des Bedarfs eines benutzerdefinierten Vokabulars ermöglicht es uns, das Potenzial attributbasierter Modelle im Kontext sehr großer Datenmengen vollends auszuschöpfen. 4) Wir präsentieren eine neuartige Anwendung semantischer Attribute in der realen Welt. Wir schlagen das erste Verfahren vor, welches automatisch Modestile lernt, und vorhersagt, wie sich ihre Beliebtheit in naher Zukunft entwickeln wird. Wir zeigen, dass semantische Attribute interpretierbare Modestile liefern und zu einer besseren Vorhersage der Beliebtheit von visuellen Stilen im Vergleich zu anderen Darstellungen führen

Dagstuhl News January - December 2006

"Dagstuhl News" is a publication edited especially for the members of the Foundation "Informatikzentrum Schloss Dagstuhl" to thank them for their support. The News give a summary of the scientific work being done in Dagstuhl. Each Dagstuhl Seminar is presented by a small abstract describing the contents and scientific highlights of the seminar as well as the perspectives or challenges of the research topic

The development of factory templates for the integrated virtual factory framework

Páginas numeradas: I-XVI, 17-123Tese de mestrado integrado. Engenharia Electrotécnica e de Computadores (Major Automação). Faculdade de Engenharia. Universidade do Porto. 201

The development of factory templates for the integrated virtual factory framework

Páginas numeradas: I-XVI, 17-123Tese de mestrado integrado. Engenharia Electrotécnica e de Computadores (Major Automação). Faculdade de Engenharia. Universidade do Porto. 201

The 1995 Goddard Conference on Space Applications of Artificial Intelligence and Emerging Information Technologies

This publication comprises the papers presented at the 1995 Goddard Conference on Space Applications of Artificial Intelligence and Emerging Information Technologies held at the NASA/Goddard Space Flight Center, Greenbelt, Maryland, on May 9-11, 1995. The purpose of this annual conference is to provide a forum in which current research and development directed at space applications of artificial intelligence can be presented and discussed

Comments on the cybernetics of stability and regulation in social systems

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The methods and principles of cybernetics are applied to a discussion of stability and regulation in social systems taking a global viewpoint. The fundamental but still classical notion of stability as applied to homeostatic and ultrastable systems is discussed, with a particular reference to a specific well-studied example of a closed social group (the Tsembaga studied by Roy Rappaport in New Guinea). The discussion extends to the problem of evolution in large systems and the question of regulating evolution is addressed without special qualifications. A more comprehensive idea of stability is introduced as the argument turns to the problem of evolution for viability in general. Concepts pertaining to the problem of evolution are exemplified by a computer simulation model of an abstractly defined ecosystem in which various dynamic processes occur allowing the study of adaptive and evolutionary behaviour. In particular, the role of coalition formation and cooperative behaviour is stressed as a key factor in the evolution of complexity. The model consists of a population of several species of dimensionless automata inhabiting a geometrically defined environment in which a commodity essential for metabolic requirements (food) appears. Automata can sense properties of their environment, move about it, compete for food, reproduce or combine into coalitions thus forming new and more complex species. Each species is associated with a specific genotype from which the species’ behavioural characteristics (its phenotype) are derived. Complexity and survival efficiency of species increases through coalition formation, an event which occurs when automata are faced with an “undecidable” situation that is resolvable only by forming a new and more complex organization. Exogenous manipulation of the food distribution pattern and other critical factors produces different environmental conditions resulting in different behaviour patterns of automata and in different evolutionary “pathways.” Eve-1, the computer program developed to implement this model, accepts a high-level command language which allows for the setting of parameters, definition of initial configurations, and control of output formats. Results of simulation are produced graphically and include various pertinent tables. The program was given a modular hierarchical structure which allows easy generation of new versions incorporating different sets of rules. The model strives to capture the essence of the evolution of complexity viewed as a general process rather than to describe the evolution of a particular “real” system. In this respect it is not context-specific, and the behaviours which are observable in different runs can receive various interpretation depending on specific identifications. Of these, biological, ecological, and sociological interpretations are the most obvious and the latter, in particular, is stressed.J. M. Kaplan Fund in New Yor

ApoE4 effects on the structural covariance brain networks topology in Mild Cognitive Impairment

The Apolipoprotein E isoform E4 (ApoE4) is consistently associated with an elevated risk of developing late-onset Alzheimer's Disease (AD). However, little is known about his potential genetic modulation on the structural covariance brain networks during prodromal stages like Mild Cognitive Impairment (MCI). The covariance phenomenon is based on the observation that regions correlating in morphometric descriptors are often part of the same brain system. In a first study, I assessed the ApoE4-related changes on the brain network topology in 256 MCI patients, using the regional cortical thickness to define the covariance network. The cross-sectional sample selected from the ADNI database was subdivided into ApoE4-positive (Carriers) and negative (non-Carriers). At the group-level, the results showed a significant decrease in characteristic path length, clustering index, local efficiency, global connectivity, modularity, and increased global efficiency for Carriers compared to non-Carriers. Overall, I found that ApoE4 in MCI shaped the topological organization of cortical thickness covariance networks. In the second project, I investigated the impact of ApoE4 on the single-subject gray matter networks in a sample of 200 MCI from the ADNI database. The patients were classified based on clinical outcome (stable MCI versus converters to AD) and ApoE4 status (Carriers versus non-Carriers). The effects of ApoE4 and disease progression on the network measures at baseline and rate of change were explored. The topological network attributes were correlated with AD biomarkers. The main findings showed that gray matter network topology is affected independently by ApoE4 and the disease progression (to AD) in late-MCI. The network measures alterations showed a more random organization in Carriers compared to non-Carriers. Finally, as additional research, I investigated whether a network-based approach combined with the graph theory is able to detect cerebrovascular reactivity (CVR) changes in MCI. Our findings suggest that this experimental approach is more sensitive to identifying subtle cerebrovascular alterations than the classical experimental designs. This study paves the way for a future investigation on the ApoE4-cerebrovascular interaction effects on the brain networks during AD progression. In summary, my thesis results provide evidence of the value of the structural covariance brain network measures to capture subtle neurodegenerative changes associated with ApoE4 in MCI. Together with other biomarkers, these variables may help predict disease progression, providing additional reliable intermediate phenotypes

Integration of multi-scale protein interactions for biomedical data analysis

With the advancement of modern technologies, we observe an increasing accumulation of biomedical data about diseases. There is a need for computational methods to sift through and extract knowledge from the diverse data available in order to improve our mechanistic understanding of diseases and improve patient care. Biomedical data come in various forms as exemplified by the various omics data. Existing studies have shown that each form of omics data gives only partial information on cells state and motivated jointly mining multi-omics, multi-modal data to extract integrated system knowledge. The interactome is of particular importance as it enables the modelling of dependencies arising from molecular interactions. This Thesis takes a special interest in the multi-scale protein interactome and its integration with computational models to extract relevant information from biomedical data. We define multi-scale interactions at different omics scale that involve proteins: pairwise protein-protein interactions, multi-protein complexes, and biological pathways. Using hypergraph representations, we motivate considering higher-order protein interactions, highlighting the complementary biological information contained in the multi-scale interactome. Based on those results, we further investigate how those multi-scale protein interactions can be used as either prior knowledge, or auxiliary data to develop machine learning algorithms. First, we design a neural network using the multi-scale organization of proteins in a cell into biological pathways as prior knowledge and train it to predict a patient's diagnosis based on transcriptomics data. From the trained models, we develop a strategy to extract biomedical knowledge pertaining to the diseases investigated. Second, we propose a general framework based on Non-negative Matrix Factorization to integrate the multi-scale protein interactome with multi-omics data. We show that our approach outperforms the existing methods, provide biomedical insights and relevant hypotheses for specific cancer types