Two Optimal Strategies for Active Learning of Causal Models from Interventional Data

From observational data alone, a causal DAG is only identifiable up to Markov equivalence. Interventional data generally improves identifiability; however, the gain of an intervention strongly depends on the intervention target, that is, the intervened variables. We present active learning (that is, optimal experimental design) strategies calculating optimal interventions for two different learning goals. The first one is a greedy approach using single-vertex interventions that maximizes the number of edges that can be oriented after each intervention. The second one yields in polynomial time a minimum set of targets of arbitrary size that guarantees full identifiability. This second approach proves a conjecture of Eberhardt (2008) indicating the number of unbounded intervention targets which is sufficient and in the worst case necessary for full identifiability. In a simulation study, we compare our two active learning approaches to random interventions and an existing approach, and analyze the influence of estimation errors on the overall performance of active learning

Reverse Engineering of Biological Systems

Gene regulatory network (GRN) consists of a set of genes and regulatory relationships between the genes. As outputs of the GRN, gene expression data contain important information that can be used to reconstruct the GRN to a certain degree. However, the reverse engineer of GRNs from gene expression data is a challenging problem in systems biology. Conventional methods fail in inferring GRNs from gene expression data because of the relative less number of observations compared with the large number of the genes. The inherent noises in the data make the inference accuracy relatively low and the combinatorial explosion nature of the problem makes the inference task extremely difficult. This study aims at reconstructing the GRNs from time-course gene expression data based on GRN models using system identification and parameter estimation methods. The main content consists of three parts: (1) a review of the methods for reverse engineering of GRNs, (2) reverse engineering of GRNs based on linear models and (3) reverse engineering of GRNs based on a nonlinear model, specifically S-systems. In the first part, after the necessary background and challenges of the problem are introduced, various methods for the inference of GRNs are comprehensively reviewed from two aspects: models and inference algorithms. The advantages and disadvantages of each method are discussed. The second part focus on inferring GRNs from time-course gene expression data based on linear models. First, the statistical properties of two sparse penalties, adaptive LASSO and SCAD, with an autoregressive model are studied. It shows that the proposed methods using these two penalties can asymptotically reconstruct the underlying networks. This provides a solid foundation for these methods and their extensions. Second, the integration of multiple datasets should be able to improve the accuracy of the GRN inference. A novel method, Huber group LASSO, is developed to infer GRNs from multiple time-course data, which is also robust to large noises and outliers that the data may contain. An efficient algorithm is also developed and its convergence analysis is provided. The third part can be further divided into two phases: estimating the parameters of S-systems with system structure known and inferring the S-systems without knowing the system structure. Two methods, alternating weighted least squares (AWLS) and auxiliary function guided coordinate descent (AFGCD), have been developed to estimate the parameters of S-systems from time-course data. AWLS takes advantage of the special structure of S-systems and significantly outperforms one existing method, alternating regression (AR). AFGCD uses the auxiliary function and coordinate descent techniques to get the smart and efficient iteration formula and its convergence is theoretically guaranteed. Without knowing the system structure, taking advantage of the special structure of the S-system model, a novel method, pruning separable parameter estimation algorithm (PSPEA) is developed to locally infer the S-systems. PSPEA is then combined with continuous genetic algorithm (CGA) to form a hybrid algorithm which can globally reconstruct the S-systems

Probabilistic Modeling of Process Systems with Application to Risk Assessment and Fault Detection

Three new methods of joint probability estimation (modeling), a maximum-likelihood maximum-entropy method, a constrained maximum-entropy method, and a copula-based method called the rolling pin (RP) method, were developed. Compared to many existing probabilistic modeling methods such as Bayesian networks and copulas, the developed methods yield models that have better performance in terms of flexibility, interpretability and computational tractability. These methods can be used readily to model process systems and perform risk analysis and fault detection at steady state conditions, and can be coupled with appropriate mathematical tools to develop dynamic probabilistic models. Also, a method of performing probabilistic inference using RP-estimated joint probability distributions was introduced; this method is superior to Bayesian networks in several aspects. The RP method was also applied successfully to identify regression models that have high level of flexibility and are appealing in terms of computational costs.Ph.D., Chemical Engineering -- Drexel University, 201

Analyse des leviers : effets de colinéarité et hiérarchisation des impacts dans les études de marché et sociales

AbstractLinear regression is used in Market Research but faces difficulties due to multicollinearity. Other methods have been considered.A demonstration of the equality between lmg-Shapley and and Johnson methods for Variance Decomposition has been proposed. Also this research has shown that the decomposition proposed by Fabbris is not identical to those proposed by Genizi and Johnson, and that the CAR scores of two predictors do not equalize when their correlation tends towards 1. A new method, weifila (weighted first last) has been proposed and published in 2015.Also we have shown that permutation importance using Random Forest enables to take into account non linear relationships and deserves broader usage in Marketing Research.Regarding Bayesian Networks, there are multiple solutions available and expert driven restrictions and decisions support the recommendation to be careful in their usage and presentation, even if they allow to explore possible structures and make simulations.In the end, weifila or random forests are recommended instead of lmg-Shapley knowing that the benefit of structural and conceptual models should not be underestimated.Keywords :Linear regression, Variable Importance, Shapley Value, Random Forests, Bayesian NetworksLa colinéarité rend difficile l’utilisation de la régression linéaire pour estimer l’importance des variables dans les études de marché. D’autres approches ont donc été utilisées.Concernant la décomposition de la variance expliquée, une démonstration de l’égalité entre les méthodes lmg-Shapley et celle de Johnson avec deux prédicteurs est proposée. Il a aussi été montré que la méthode de Fabbris est différente des méthodes de Genizi et Johnson et que les CAR scores de deux prédicteurs ne s’égalisent pas lorsque leur corrélation tend vers 1.Une méthode nouvelle, weifila (weighted first last) a été définie et publiée en 2015.L’estimation de l’importance des variables avec les forêts aléatoires a également été analysée et les résultats montrent une bonne prise en compte des non-linéarités.Avec les réseaux bayésiens, la multiplicité des solutions et le recours à des restrictions et choix d’expert militent pour utilisation prudente même si les outils disponibles permettent une aide dans le choix des modèles.Le recours à weifila ou aux forêts aléatoires est recommandé plutôt que lmg-Shapley sans négliger les approches structurelles et les modèles conceptuels.Mots clés :régression, décomposition de la variance, importance, valeur de Shapley, forêts aléatoires, réseaux bayésiens