Representation Learning: A Review and New Perspectives

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, auto-encoders, manifold learning, and deep networks. This motivates longer-term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning

A review on initialization methods for nonnegative matrix factorization: Towards omics data experiments

Nonnegative Matrix Factorization (NMF) has acquired a relevant role in the panorama of knowledge extraction, thanks to the peculiarity that non-negativity applies to both bases and weights, which allows meaningful interpretations and is consistent with the natural human part-based learning process. Nevertheless, most NMF algorithms are iterative, so initialization methods affect convergence behaviour, the quality of the final solution, and NMF performance in terms of the residual of the cost function. Studies on the impact of NMF initialization techniques have been conducted for text or image datasets, but very few considerations can be found in the literature when biological datasets are studied, even though NMFs have largely demonstrated their usefulness in better understanding biological mechanisms with omic datasets. This paper aims to present the state-of-the-art on NMF initialization schemes along with some initial considerations on the impact of initialization methods when microarrays (a simple instance of omic data) are evaluated with NMF mechanisms. Using a series of measures to qualitatively examine the biological information extracted by a given NMF scheme, it preliminary appears that some information (e.g., represented by genes) can be extracted regardless of the initialization scheme used

RISE: An Incremental Trust-Region Method for Robust Online Sparse Least-Squares Estimation

Many point estimation problems in robotics, computer vision, and machine learning can be formulated as instances of the general problem of minimizing a sparse nonlinear sum-of-squares objective function. For inference problems of this type, each input datum gives rise to a summand in the objective function, and therefore performing online inference corresponds to solving a sequence of sparse nonlinear least-squares minimization problems in which additional summands are added to the objective function over time. In this paper, we present Robust Incremental least-Squares Estimation (RISE), an incrementalized version of the Powell's Dog-Leg numerical optimization method suitable for use in online sequential sparse least-squares minimization. As a trust-region method, RISE is naturally robust to objective function nonlinearity and numerical ill-conditioning and is provably globally convergent for a broad class of inferential cost functions (twice-continuously differentiable functions with bounded sublevel sets). Consequently, RISE maintains the speed of current state-of-the-art online sparse least-squares methods while providing superior reliability.United States. Office of Naval Research (Grant N00014-12-1-0093)United States. Office of Naval Research (Grant N00014-11-1-0688)United States. Office of Naval Research (Grant N00014-06-1-0043)United States. Office of Naval Research (Grant N00014-10-1-0936)United States. Air Force Research Laboratory (Contract FA8650-11-C-7137

Archetypes for histogram-valued data

Il principale sviluppo innovativo del lavoro è quello di propone una estensione dell'analisi archetipale per dati ad istogramma. Per quanto concerne l'impianto metodologico nell'approccio all'analisi di dati ad istogramma, che sono di natura complessa, il presente lavora utilizza le intuizioni della "Symbolic Data Analysis" (SDA) e le relazioni intrinseche tra dati valutati ad intervallo e dati valutati ad istogramma. Dopo aver discusso la tecnica sviluppata in ambiente Matlab, il suo funzionamento e le sue proprietà su di un esempio di comodo, tale tecnica viene proposta, nella sezione applicativa, come strumento per effettuare una analisi di tipo "benchmarking" quantitativo. Nello specifico, si propongono i principali risultati ottenuti da una applicazione degli archetipi per dati ad istogramma ad un caso di benchmarking interno del sistema scolastico, utilizzando dati provenienti dal test INVALSI relativi all'anno scolastico 2015/2016. In questo contesto l'unità di analisi è considerata essere la singola scuola, definita operativamente attraverso le distribuzioni dei punteggi dei propri alunni valutate, congiuntamente, sotto forma di oggetti simbolici ad istogramma

Space transfer concepts and analysis for exploration missions. Implementation plan and element description document (draft final). Volume 5: Nuclear electric propulsion vehicle

The nuclear electric propulsion (NEP) concept design developed in support of the Space Transfer Concepts and Analysis for Exploration Missions (STCAEM) study is presented. The evolution of the NEP concept is described along with the requirements, guidelines, and assumptions for the design. Operating modes and options are defined and a systems description of the vehicle is presented. Artificial gravity configuration options and space and ground support systems are discussed. Finally, an implementation plan is presented which addresses technology needs, schedules, facilities and costs

Space transfer concepts and analysis for exploration missions. Implementation plan and element description document (draft final). Volume 4: Solar electric propulsion vehicle

This document presents the solar electric propulsion (SEP) concept design developed as part of the Space Transfer Concepts and Analysis for Exploration Missions (STCAEM) study. The evolution of the SEP concept is described along with the requirements, guidelines and assumptions for the design. Operating modes and options are defined and a systems description of the vehicle is presented. Artificial gravity configuration options and space and ground support systems are discussed. Finally, an implementation plan is presented which addresses technology needs, schedules, facilities, and costs

DETERMINISTIC SEA WAVE AND SHIP MOTION FORECASTING: FROM REMOTE WAVE SENSING TO PREDICTION ERROR ASSESSMENT

Presently, the assessment of the operational safety of ships and offshore structures is typically addressed within a statistical framework, both at design stage and for the specification of operational limits. Recently, however, the availability of new remote sensing technologies is paving the way for complementary approaches based on deterministic predictions of sea waves and ship motions. In this respect, the marine wave radar is considered as the key asset for the deterministic prediction of wave elevation. Indeed, the possibility of measuring the sea surface, almost instantaneously and for large areas, can be used to forecast the wave elevation at the location of the operating units. Eventually, the coupling of deterministic wave forecast with suitable ship motion models opens the possibility for giving anticipated prediction and guidance. The application of this emerging approach can be beneficial to those short-time offshore operations requiring sea wave or ship motions to be forecast in the time horizon of tens of seconds to minutes. This envisages the possibility of development of finely tuned early warning, hazard control and support decision systems. One of the main aspects of this chain of models, which is oftentimes overlooked, is the importance of providing the forecast with a consistent assessment of the prediction error. Moreover, the additional sources of uncertainty coming from the wave measurement and from the inversion of the wave radar images are also seldom accounted for. In this thesis, the whole chain of models, that from the wave radar measurement leads to the ship motion prediction, is investigated. The first step is the proposal of a novel technique for the inversion of wave radar images that can consistently account for those regions of the sea surface that cannot be uniformly illuminated because of the shadowing effect. The adoption of a linear least square fitting approach, provided with a regularization technique, allows the proposed inversion method the needed flexibility to address the shadowed regions as missing data. Afterwards, the assessment of the error associated with deterministic wave predictions is addressed. A novel semi-analytical procedure is proposed which allows estimating the ensemble variance of prediction error, in a simple and flexible way, naturally embedding the characteristics of the linear fitting and propagation procedures. The approach also allows the inclusion, in the estimation of prediction error, of the effect of measurement error coming from the radar inversion techniques. The same technique for the estimation of wave elevation prediction error is then extended to linear ship motions, using linear transfer functions. In fact, the developed framework can deal with any linear transformation of the wave elevation, resulting in the definition of a sound measure of the prediction error of linear responses. The method can be used to provide deterministic predictions with confidence intervals, as well as for a consistent setup of the whole forecasting chain. The developed models are then tested for a set of application examples considering both linear and nonlinear wave fields. In this latter respect, a high order spectral method has been implemented to provide more realistic wave elevation fields. Example applications regarding linear ship motions have also been carried out. The proposed inversion procedure has shown interesting results for synthetic radar images generated from both linear and moderately nonlinear wave fields. However, further investigations are needed to reduce the high computational cost required for the inversion. The proposed approach for wave and ship motion prediction error, instead, can represent a convenient novel sound method for the consistent setup of deterministic prediction procedures, remaining, however, limited to those scenarios where nonlinearities play a minor role

Emergent Design

Explorations in Systems Phenomenology in Relation to Ontology, Hermeneutics and the Meta-dialectics of Design SYNOPSIS A Phenomenological Analysis of Emergent Design is performed based on the foundations of General Schemas Theory. The concept of Sign Engineering is explored in terms of Hermeneutics, Dialectics, and Ontology in order to define Emergent Systems and Metasystems Engineering based on the concept of Meta-dialectics. ABSTRACT Phenomenology, Ontology, Hermeneutics, and Dialectics will dominate our inquiry into the nature of the Emergent Design of the System and its inverse dual, the Meta-system. This is an speculative dissertation that attempts to produce a philosophical, mathematical, and theoretical view of the nature of Systems Engineering Design. Emergent System Design, i.e., the design of yet unheard of and/or hitherto non-existent Systems and Metasystems is the focus. This study is a frontal assault on the hard problem of explaining how Engineering produces new things, rather than a repetition or reordering of concepts that already exist. In this work the philosophies of E. Husserl, A. Gurwitsch, M. Heidegger, J. Derrida, G. Deleuze, A. Badiou, G. Hegel, I. Kant and other Continental Philosophers are brought to bear on different aspects of how new technological systems come into existence through the midwifery of Systems Engineering. Sign Engineering is singled out as the most important aspect of Systems Engineering. We will build on the work of Pieter Wisse and extend his theory of Sign Engineering to define Meta-dialectics in the form of Quadralectics and then Pentalectics. Along the way the various ontological levels of Being are explored in conjunction with the discovery that the Quadralectic is related to the possibility of design primarily at the Third Meta-level of Being, called Hyper Being. Design Process is dependent upon the emergent possibilities that appear in Hyper Being. Hyper Being, termed by Heidegger as Being (Being crossed-out) and termed by Derrida as Differance, also appears as the widest space within the Design Field at the third meta-level of Being and therefore provides the most leverage that is needed to produce emergent effects. Hyper Being is where possibilities appear within our worldview. Possibility is necessary for emergent events to occur. Hyper Being possibilities are extended by Wild Being propensities to allow the embodiment of new things. We discuss how this philosophical background relates to meta-methods such as the Gurevich Abstract State Machine and the Wisse Metapattern methods, as well as real-time architectural design methods as described in the Integral Software Engineering Methodology. One aim of this research is to find the foundation for extending the ISEM methodology to become a general purpose Systems Design Methodology. Our purpose is also to bring these philosophical considerations into the practical realm by examining P. Bourdieu’s ideas on the relationship between theoretical and practical reason and M. de Certeau’s ideas on practice. The relationship between design and implementation is seen in terms of the Set/Mass conceptual opposition. General Schemas Theory is used as a way of critiquing the dependence of Set based mathematics as a basis for Design. The dissertation delineates a new foundation for Systems Engineering as Emergent Engineering based on General Schemas Theory, and provides an advanced theory of Design based on the understanding of the meta-levels of Being, particularly focusing upon the relationship between Hyper Being and Wild Being in the context of Pure and Process Being