A morphospace of functional configuration to assess configural breadth based on brain functional networks

The best approach to quantify human brain functional reconfigurations in response to varying cognitive demands remains an unresolved topic in network neuroscience. We propose that such functional reconfigurations may be categorized into three different types: i) Network Configural Breadth, ii) Task-to-Task transitional reconfiguration, and iii) Within-Task reconfiguration. In order to quantify these reconfigurations, we propose a mesoscopic framework focused on functional networks (FNs) or communities. To do so, we introduce a 2D network morphospace that relies on two novel mesoscopic metrics, Trapping Efficiency (TE) and Exit Entropy (EE), which capture topology and integration of information within and between a reference set of FNs. In this study, we use this framework to quantify the Network Configural Breadth across different tasks. We show that the metrics defining this morphospace can differentiate FNs, cognitive tasks and subjects. We also show that network configural breadth significantly predicts behavioral measures, such as episodic memory, verbal episodic memory, fluid intelligence and general intelligence. In essence, we put forth a framework to explore the cognitive space in a comprehensive manner, for each individual separately, and at different levels of granularity. This tool that can also quantify the FN reconfigurations that result from the brain switching between mental states.Comment: main article: 24 pages, 8 figures, 2 tables. supporting information: 11 pages, 5 figure

Extracting the dynamics of behavior in sensory decision-making experiments

Decision-making strategies evolve during training and can continue to vary even in well-trained animals. However, studies of sensory decision-making tend to characterize behavior in terms of a fixed psychometric function that is fit only after training is complete. Here, we present PsyTrack, a flexible method for inferring the trajectory of sensory decision-making strategies from choice data. We apply PsyTrack to training data from mice, rats, and human subjects learning to perform auditory and visual decision-making tasks. We show that it successfully captures trial-to-trial fluctuations in the weighting of sensory stimuli, bias, and task-irrelevant covariates such as choice and stimulus history. This analysis reveals dramatic differences in learning across mice and rapid adaptation to changes in task statistics. PsyTrack scales easily to large datasets and offers a powerful tool for quantifying time-varying behavior in a wide variety of animals and tasks

Optimal experimental design and its applications to biochemical engineering systems

This work is motivated by challenges in data-based modelling of complex systems due to limited information of sparse and noisy experimental data. Optimal experimental design (OED) techniques, which aim at devising necessary experiments to generate informative measurement data to facilitate model identification, have been investigated comprehensively.The limitations of existing experimental design approaches have been extensively discussed, based on which advanced experimental design methods and efficient numerical strategies have been developed for improved solutions.;Two case study biochemical systems have been used through the research investigation, one is an enzyme reaction system, the other one is a lab-scale enzymatic biodiesel production system. The main contributions of this PhD work can be summarised as follows:;Single objective experimental designs by considering one type of design factors, i.e. input intensity, measurement set selection, sampling profile design, respectively, has been formulated and numerical strategies to solve these optimisation problems have been described in detail. Implementations of these design methods to biochemical systems have demonstrated its efficiency in reducing parameter estimation errors.;A new OED strategy has been proposed to cope with OED problems including multiple design factors in one optimisation framework. An iterative two-layer design structure is developed. In the lower layer for observation design, the sampling profile and the measurement set selection are combined and formulated as a single integrated observation design problem, which is relaxed to a convex optimization problem that can be solved with a local method.;Thus the measurement set selection and the sampling profile can be determined simultaneously. In the upper layer for input design, the optimisation of input intensities is obtained through stochastic global searching. In this way, the multi-factor optimisation problem is solved through the integration of a stochastic method, for the upper layer, and a deterministic method, for the lower layer.;A new enzyme reaction model has been established which represents a typical class of enzymatic kinetically controlled synthesis process. This model contains important kinetic reaction features, moderate complexity, and complete model information. It can be used as a benchmark problem for development and comparison of OED algorithms. Systematic analysis has been performed in order to examine the system behaviours, and the dependence on model parameters, initial operation conditions.;Structural identifiability and practical identifiability of this system have been analysed and identifiable parameters determined. The design of experiment for the enzyme reactionsystem by considering different types of design variables have been investigated. The parameter estimation precision can be improved significantly by using the proposed OED techniques, compared to the non-designed condition.;The OED techniques are numerically investigated based on a lab-scale biodiesel production process with real experimental data through research collaboration with DTU in Denmark. The OED applications on this real system model allow to examine the effectiveness and efficiency of those new proposed OED methods. The measurement set selection and the sampling design of this system are developed which provide detailed instructions on how to improve experiments through OED.;Also, the sensitivity analysis and parameter identifiability analysis are conducted; and their impacts to experimental design are clearly identified.This work is motivated by challenges in data-based modelling of complex systems due to limited information of sparse and noisy experimental data. Optimal experimental design (OED) techniques, which aim at devising necessary experiments to generate informative measurement data to facilitate model identification, have been investigated comprehensively.The limitations of existing experimental design approaches have been extensively discussed, based on which advanced experimental design methods and efficient numerical strategies have been developed for improved solutions.;Two case study biochemical systems have been used through the research investigation, one is an enzyme reaction system, the other one is a lab-scale enzymatic biodiesel production system. The main contributions of this PhD work can be summarised as follows:;Single objective experimental designs by considering one type of design factors, i.e. input intensity, measurement set selection, sampling profile design, respectively, has been formulated and numerical strategies to solve these optimisation problems have been described in detail. Implementations of these design methods to biochemical systems have demonstrated its efficiency in reducing parameter estimation errors.;A new OED strategy has been proposed to cope with OED problems including multiple design factors in one optimisation framework. An iterative two-layer design structure is developed. In the lower layer for observation design, the sampling profile and the measurement set selection are combined and formulated as a single integrated observation design problem, which is relaxed to a convex optimization problem that can be solved with a local method.;Thus the measurement set selection and the sampling profile can be determined simultaneously. In the upper layer for input design, the optimisation of input intensities is obtained through stochastic global searching. In this way, the multi-factor optimisation problem is solved through the integration of a stochastic method, for the upper layer, and a deterministic method, for the lower layer.;A new enzyme reaction model has been established which represents a typical class of enzymatic kinetically controlled synthesis process. This model contains important kinetic reaction features, moderate complexity, and complete model information. It can be used as a benchmark problem for development and comparison of OED algorithms. Systematic analysis has been performed in order to examine the system behaviours, and the dependence on model parameters, initial operation conditions.;Structural identifiability and practical identifiability of this system have been analysed and identifiable parameters determined. The design of experiment for the enzyme reactionsystem by considering different types of design variables have been investigated. The parameter estimation precision can be improved significantly by using the proposed OED techniques, compared to the non-designed condition.;The OED techniques are numerically investigated based on a lab-scale biodiesel production process with real experimental data through research collaboration with DTU in Denmark. The OED applications on this real system model allow to examine the effectiveness and efficiency of those new proposed OED methods. The measurement set selection and the sampling design of this system are developed which provide detailed instructions on how to improve experiments through OED.;Also, the sensitivity analysis and parameter identifiability analysis are conducted; and their impacts to experimental design are clearly identified

Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling?

In recent years, a strong debate has emerged in the hydrologic literature regarding what constitutes an appropriate framework for uncertainty estimation. Particularly, there is strong disagreement whether an uncertainty framework should have its roots within a proper statistical (Bayesian) context, or whether such a framework should be based on a different philosophy and implement informal measures and weaker inference to summarize parameter and predictive distributions. In this paper, we compare a formal Bayesian approach using Markov Chain Monte Carlo (MCMC) with generalized likelihood uncertainty estimation (GLUE) for assessing uncertainty in conceptual watershed modeling. Our formal Bayesian approach is implemented using the recently developed differential evolution adaptive metropolis (DREAM) MCMC scheme with a likelihood function that explicitly considers model structural, input and parameter uncertainty. Our results demonstrate that DREAM and GLUE can generate very similar estimates of total streamflow uncertainty. This suggests that formal and informal Bayesian approaches have more common ground than the hydrologic literature and ongoing debate might suggest. The main advantage of formal approaches is, however, that they attempt to disentangle the effect of forcing, parameter and model structural error on total predictive uncertainty. This is key to improving hydrologic theory and to better understand and predict the flow of water through catchment

Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation

We tackle the problems of latent variables identification and ``out-of-support'' image generation in representation learning. We show that both are possible for a class of decoders that we call additive, which are reminiscent of decoders used for object-centric representation learning (OCRL) and well suited for images that can be decomposed as a sum of object-specific images. We provide conditions under which exactly solving the reconstruction problem using an additive decoder is guaranteed to identify the blocks of latent variables up to permutation and block-wise invertible transformations. This guarantee relies only on very weak assumptions about the distribution of the latent factors, which might present statistical dependencies and have an almost arbitrarily shaped support. Our result provides a new setting where nonlinear independent component analysis (ICA) is possible and adds to our theoretical understanding of OCRL methods. We also show theoretically that additive decoders can generate novel images by recombining observed factors of variations in novel ways, an ability we refer to as Cartesian-product extrapolation. We show empirically that additivity is crucial for both identifiability and extrapolation on simulated data.Comment: Appears in: Advances in Neural Information Processing Systems 37 (NeurIPS 2023). 39 page

Statistical methods for data with different dimensions

This thesis addresses the joint analysis of data with different dimensions, such as scalars, vectors, functions and images. This is of high practical and methodological relevance, as in the course of the technical progress, data with increasing complexity and dimensionality becomes available, requiring the extension of statistical models to new types of data and leading to the development of completely new statistical methods. In the first part of the thesis, multivariate functional principal component analysis (MFPCA) is developed for functional data on different dimensional domains. This is a novel method, as existing approaches for MFPCA are restricted to multivariate functional data on the same, one-dimensional interval. Using the new approach, principal components for data consisting e.g. of functions and images (i.e. functions on a two-dimensional domain) can be obtained, taking potential covariation in the elements into account. The thesis constructs a thorough theoretical basis for multivariate functional data on different dimensional domains and derives a theoretical relationship between univariate and multivariate functional principal component analysis for finite sample sizes. The results can be used to estimate multivariate functional principal components, eigenvalues and scores based on their univariate counterparts. It is shown how the method can be extended to univariate elements in general basis representations and to a weighted version of MFPCA to correct for differences in domain, range or variation of the elements. The approach is also applicable for sparse data or data with measurement error. The finite sample performance of the new method is evaluated in a simulation study with different levels of complexity. Moreover, asymptotic properties for large sample sizes are derived in two theorems, using results from perturbation theory and showing consistency of the proposed estimators. The estimation algorithm has been implemented in a publicly available R-package MFPCA, together with another R-package funData for representing functional data in an object-oriented manner. The thesis provides an introduction to the software and the underlying concepts. The new approach is illustrated in an application to a neuroimaging dataset. The aim here is to examine the relationship between trajectories of a neuropsychological test score over time and FDG-PET brain scans at baseline, that can be interpreted as functions on a three-dimensional domain, as the latter might be predictive of subsequent cognitive decline. The results show that estimates obtained from the new MFPCA method are meaningful from a medical point of view and provide new insights into the data. The second part of the thesis is concerned with scalar-on-image regression. This class of statistical methods models the relation of a scalar outcome and an image predictor, hence data with different dimensions and a complex dependence structure. It is representative for a broad class of statistical models for complex data, which intrinsically is unidentifiable, as in general the number of observations will be low compared to the number of pixels in the image. Strong model assumptions are thus required to obtain a unique solution, which is of course conditional on the hypotheses made on the true coefficient image. In the thesis, different models for scalar-on-image regression with different assumptions are compared with respect to their ability to give reliable and interpretable estimates. To this end, new measures for quantifying the influence of model assumptions are developed and analyzed in a simulation study for nine different scalar-on-image models. The relevance of the topic is illustrated in a practical neuroimaging application. It is shown that different models with different assumptions can lead to results that share common patterns, but can differ substantially in their details, as model assumptions can have a strong influence on the estimates. This can entail the risk of over-interpreting effects that are mainly driven by the model assumptions.Diese Doktorarbeit beschäftigt sich mit der gemeinsamen Analyse von Daten unterschiedlicher Dimension, wie beispielsweise Skalare, Vektoren, Funktionen und Bilder. Dies ist sowohl aus praktischer als auch aus methodischer Sicht relevant, da im Zuge des technischen Fortschritts Daten mit zunehmender Komplexität und Dimensionalität zur Verfügung stehen, die einerseits eine Erweiterung von statistischen Modellen auf neue Datentypen erfordern und andererseits zur Entwicklung völlig neuer statistischer Methoden führen. Im ersten Teil der Arbeit wird eine multivariate funktionale Hauptkomponentenanalyse (engl. multivariate functional principal component analysis, MFPCA) für funktionale Daten auf unterschiedlich-dimensionalen Trägern entwickelt. Es handelt sich hier um eine neuartige Methode, da bestehende Ansätze für MFPCA auf multivariate funktionale Daten auf einem gemeinsamen eindimensionalen Intervall beschränkt sind. Mit dem neu entwickelten Ansatz können Hauptkomponenten für Daten bestimmt werden, die z.B. aus Funktionen und Bildern (d.h. Funktionen auf einem zwei-dimensionalen Träger) bestehen, womit eventuell vorhandene Kovariation in den Elementen berücksichtigt werden kann. In der Arbeit werden die theoretischen Grundlagen für multivariate funktionale Daten auf unterschiedlich-dimensionalen Trägern gelegt. Für den Fall einer endlichen Stichprobe wird anschließend einen theoretischen Zusammenhang zwischen univariater und multivariater funktionaler Hauptkomponentenanalyse hergeleitet. Das Ergebnis kann zur Schätzung multivariater funktionaler Hauptkomponenten, Eigenwerte und Scores auf Basis der univariaten Analoga genutzt werden. Es wird gezeigt, wie die Methode auf univariate Elemente in allgemeinen Basisdarstellungen erweitert werden kann. Weiterhin wird eine gewichtete Version der MFPCA vorgestellt, mithilfe derer für Unterschiede im Träger, Wertebereich oder Variation der einzelnen Elemente korrigiert werden kann. Der neue Ansatz eignet sich auch für funktionale Daten mit wenig Beobachtungspunkten (engl. sparse data) oder Daten, die mit Messfehlern erhoben wurden. Für den Fall endlicher Stichproben wird die Leistungsfähigkeit der neuen Methode im Rahmen einer Simulationsstudie mit unterschiedlichen Komplexitätsgraden untersucht. Darüberhinaus werden die asymptotischen Eigenschaften für große Stichproben in zwei Theoremen unter Verwendung von Resultaten aus der Perturbationstheorie hergeleitet und es wird bewiesen, dass die vorgeschlagenen Schätzer konsistent sind. Der Schätzalgorithmus ist in dem öffentlich verfügbaren R-Paket MFPCA implementiert, gemeinsam mit einem weiteren R-Paket funData zur objektorientierten Darstellung funktionaler Daten. Die Arbeit enthält eine Einführung in die Software und die zugrundeliegenden Konzepte. Die neue Methode wird in einem Anwendungskapitel anhand eines Neuroimaging Datensatzes illustriert. Ziel der Untersuchung ist es, einen Zusammenhang zwischen den Ergebnissen eines neuropsychologischen Tests über den Studienverlauf und FDG-PET Gehirnscans herzustellen, die zu Beginn der Studie aufgenommen wurden, da Letztere prädiktiv für eine anschließende Verschlechterung der kognitiven Fähigkeiten sein können. Die Scans können dabei als Funktionen auf einem drei-dimensionalen Träger aufgefasst werden. Die Ergebnisse zeigen, dass die von der neuen MFPCA Methode gefundenen Schätzer medizinisch sinnvoll sind und neue Einblicke in die Daten ermöglichen. Der zweite Teil der Arbeit beschäftigt sich mit Skalar-auf-Bild Regression. Diese statistische Modellklasse beschreibt den Zusammenhang einer skalaren Zielgröße und einer Einflussgröße in Form eines Bildes, also Daten mit unterschiedlicher Dimension und einer komplexen Abhängigkeitsstruktur. Sie steht stellvertretend für eine breite Klasse statistischer Modelle für komplexe Daten, die von sich aus nicht identifizierbar ist, da im Allgemeinen die Anzahl der Beobachtungen im Verhältnis zur Anzahl der Pixel in einem Bild sehr klein ist. Es sind also starke Modellannahmen vonnöten, um eine eindeutige Lösung zu erhalten, die selbstverständlich durch die Annahmen an das wahre Koeffizientenbild bedingt wird. In dieser Arbeit werden unterschiedliche Modelle für Skalar-auf-Bild Regression mit unterschiedlichen Annahmen in Bezug auf ihre Fähigkeit, zuverlässige und interpretierbare Ergebnise zu erzielen, untersucht. Zu diesem Zweck werden neue Maße zur Quantifizierung des Einflusses von Modellannahmen entwickelt und in einer Simulationsstudie für neun verschiedene Skalar-auf-Bild Regressionsmodelle untersucht. Die Bedeutung der Thematik wird wiederum in einer praktischen Anwendung aus dem Neuroimaging-Bereich veranschaulicht. Es wird gezeigt, dass unterschiedliche Modelle mit unterschiedlichen Annahmen zu Ergebnissen führen können, die zwar ähnliche Muster aufweisen, sich in Details aber zum Teil deutlich unterscheiden, da die Modellannahmen einen starken Einfluss auf die Schätzungen haben können. Dies bringt die mögliche Gefahr mit sich, Effekte zu überinterpretieren, die hauptsächlich von den Modellannahmen getrieben sind

Fitting Structural Equation Models via Variational Approximations

Structural equation models are commonly used to capture the relationship between sets of observed and unobservable variables. Traditionally these models are fitted using frequentist approaches but recently researchers and practitioners have developed increasing interest in Bayesian inference. In Bayesian settings, inference for these models is typically performed via Markov chain Monte Carlo methods, which may be computationally intensive for models with a large number of manifest variables or complex structures. Variational approximations can be a fast alternative; however, they have not been adequately explored for this class of models. We develop a mean field variational Bayes approach for fitting elemental structural equation models and demonstrate how bootstrap can considerably improve the variational approximation quality. We show that this variational approximation method can provide reliable inference while being significantly faster than Markov chain Monte Carlo