Universally Sloppy Parameter Sensitivities in Systems Biology

Quantitative computational models play an increasingly important role in modern biology. Such models typically involve many free parameters, and assigning their values is often a substantial obstacle to model development. Directly measuring \emph{in vivo} biochemical parameters is difficult, and collectively fitting them to other data often yields large parameter uncertainties. Nevertheless, in earlier work we showed in a growth-factor-signaling model that collective fitting could yield well-constrained predictions, even when it left individual parameters very poorly constrained. We also showed that the model had a `sloppy' spectrum of parameter sensitivities, with eigenvalues roughly evenly distributed over many decades. Here we use a collection of models from the literature to test whether such sloppy spectra are common in systems biology. Strikingly, we find that every model we examine has a sloppy spectrum of sensitivities. We also test several consequences of this sloppiness for building predictive models. In particular, sloppiness suggests that collective fits to even large amounts of ideal time-series data will often leave many parameters poorly constrained. Tests over our model collection are consistent with this suggestion. This difficulty with collective fits may seem to argue for direct parameter measurements, but sloppiness also implies that such measurements must be formidably precise and complete to usefully constrain many model predictions. We confirm this implication in our signaling model. Our results suggest that sloppy sensitivity spectra are universal in systems biology models. The prevalence of sloppiness highlights the power of collective fits and suggests that modelers should focus on predictions rather than on parameters.Comment: Submitted to PLoS Computational Biology. Supplementary Information available in "Other Formats" bundle. Discussion slightly revised to add historical contex

The sloppy model universality class and the Vandermonde matrix

In a variety of contexts, physicists study complex, nonlinear models with many unknown or tunable parameters to explain experimental data. We explain why such systems so often are sloppy; the system behavior depends only on a few `stiff' combinations of the parameters and is unchanged as other `sloppy' parameter combinations vary by orders of magnitude. We contrast examples of sloppy models (from systems biology, variational quantum Monte Carlo, and common data fitting) with systems which are not sloppy (multidimensional linear regression, random matrix ensembles). We observe that the eigenvalue spectra for the sensitivity of sloppy models have a striking, characteristic form, with a density of logarithms of eigenvalues which is roughly constant over a large range. We suggest that the common features of sloppy models indicate that they may belong to a common universality class. In particular, we motivate focusing on a Vandermonde ensemble of multiparameter nonlinear models and show in one limit that they exhibit the universal features of sloppy models.Comment: New content adde

SDSS-III: Massive Spectroscopic Surveys of the Distant Universe, the Milky Way Galaxy, and Extra-Solar Planetary Systems

Building on the legacy of the Sloan Digital Sky Survey (SDSS-I and II), SDSS-III is a program of four spectroscopic surveys on three scientific themes: dark energy and cosmological parameters, the history and structure of the Milky Way, and the population of giant planets around other stars. In keeping with SDSS tradition, SDSS-III will provide regular public releases of all its data, beginning with SDSS DR8 (which occurred in Jan 2011). This paper presents an overview of the four SDSS-III surveys. BOSS will measure redshifts of 1.5 million massive galaxies and Lya forest spectra of 150,000 quasars, using the BAO feature of large scale structure to obtain percent-level determinations of the distance scale and Hubble expansion rate at z<0.7 and at z~2.5. SEGUE-2, which is now completed, measured medium-resolution (R=1800) optical spectra of 118,000 stars in a variety of target categories, probing chemical evolution, stellar kinematics and substructure, and the mass profile of the dark matter halo from the solar neighborhood to distances of 100 kpc. APOGEE will obtain high-resolution (R~30,000), high signal-to-noise (S/N>100 per resolution element), H-band (1.51-1.70 micron) spectra of 10^5 evolved, late-type stars, measuring separate abundances for ~15 elements per star and creating the first high-precision spectroscopic survey of all Galactic stellar populations (bulge, bar, disks, halo) with a uniform set of stellar tracers and spectral diagnostics. MARVELS will monitor radial velocities of more than 8000 FGK stars with the sensitivity and cadence (10-40 m/s, ~24 visits per star) needed to detect giant planets with periods up to two years, providing an unprecedented data set for understanding the formation and dynamical evolution of giant planet systems. (Abridged)Comment: Revised to version published in The Astronomical Journa

Prediction and Optimal Experimental Design in Systems Biology Models

In this dissertation we propose some approaches in model-building and model analysis techniques that can be used for typical systems biology models. In Chapter 2 we introduce a dynamical model for growth factor receptor signaling and down-regulation. We show how, by quantitatively fitting the model to experimental data, we can infer interactions that are needed to describe the dynamical behavior. We demonstrate that predictions need to be accompanied by uncertainty estimates for both model validation and hypothesis testing. We then introduce some of the techniques from the optimal experimental design literature to reduce the prediction uncertainty for dynamical variables of interest. In Chapter 3 we analyze the convergence properties of some of the Markov Chain Monte Carlo (MCMC) algorithms that can be used to give more rigorous uncertainty estimates for both parameters and dynamical variables within a model. We lay out a straightforward procedure which gives approximate convergence rates as a function of the tunable parameters of the MCMC method. We show that the method gives good estimates of convergence rates for the one dimensional probability distributions we examine, and it suggests optimal choices for the tunable parameters. We discover that variants of the basic MCMC algorithms which claim to have accelerated convergence often completely fail to converge geometrically in the tails of the probability distribution. In Chapter 4 we consider a different problem --- how to efficiently simulate stochastic dynamics within a biochemical network. We introduce a mixed dynamics simulation algorithm which describes the biochemical reactions where some of the species can be treated as continuous variables, but other species are naturally described as discrete stochastic variables. We then attempt to describe an approximation to the continuous dynamics in a situation where the discrete variables change on a much faster relative time scale, analogous to the quasi-equilibrium assumption made in fully deterministic systems. However our approximation method mostly fails to capture the true correction to the dynamics; we speculate as to the reasons for this

Alternate virtual populations elucidate the type I interferon signature predictive of the response to rituximab in rheumatoid arthritis

Abstract Background Mechanistic biosimulation can be used in drug development to form testable hypotheses, develop predictions of efficacy before clinical trial results are available, and elucidate clinical response to therapy. However, there is a lack of tools to simultaneously (1) calibrate the prevalence of mechanistically distinct, large sets of virtual patients so their simulated responses statistically match phenotypic variability reported in published clinical trial outcomes, and (2) explore alternate hypotheses of those prevalence weightings to reflect underlying uncertainty in population biology. Here, we report the development of an algorithm, MAPEL (Mechanistic Axes Population Ensemble Linkage), which utilizes a mechanistically-based weighting method to match clinical trial statistics. MAPEL is the first algorithm for developing weighted virtual populations based on biosimulation results that enables the rapid development of an ensemble of alternate virtual population hypotheses, each validated by a composite goodness-of-fit criterion. Results Virtual patient cohort mechanistic biosimulation results were successfully calibrated with an acceptable composite goodness-of-fit to clinical populations across multiple therapeutic interventions. The resulting virtual populations were employed to investigate the mechanistic underpinnings of variations in the response to rituximab. A comparison between virtual populations with a strong or weak American College of Rheumatology (ACR) score in response to rituximab suggested that interferon β (IFNβ) was an important mechanistic contributor to the disease state, a signature that has previously been identified though the underlying mechanisms remain unclear. Sensitivity analysis elucidated key anti-inflammatory properties of IFNβ that modulated the pathophysiologic state, consistent with the observed prognostic correlation of baseline type I interferon measurements with clinical response. Specifically, the effects of IFNβ on proliferation of fibroblast-like synoviocytes and interleukin-10 synthesis in macrophages each partially counteract reductions in synovial inflammation imparted by rituximab. A multianalyte biomarker panel predictive for virtual population therapeutic responses suggested population dependencies on B cell-dependent mediators as well as additional markers implicating fibroblast-like synoviocytes. Conclusions The results illustrate how the MAPEL algorithm can leverage knowledge of cellular and molecular function through biosimulation to propose clear mechanistic hypotheses for differences in clinical populations. Furthermore, MAPEL facilitates the development of multianalyte biomarkers prognostic of patient responses in silico

We argue that any va...

ABSTRACT: Successful predictions are among the most compelling validations of any model. Extracting falsifiable predictions from nonlinear multiparameter models is complicated by the fact that such models are commonly sloppy, possessing sensitivities to different parameter combinations that range over many decades. Here we discuss how sloppiness affects the sorts of data that best constrain model predictions, makes linear uncertainty approximations dangerous, and introduces computational difficulties in Monte-Carlo uncertainty analysis. We also present a useful test problem and suggest refinements to the standards by which models are communicated

Highly Porous Type II Collagen-Containing Scaffolds for Enhanced Cartilage Repair with Reduced Hypertrophic Cartilage Formation

The ability to regenerate damaged cartilage capable of long-term performance in an active joint remains an unmet clinical challenge in regenerative medicine. Biomimetic scaffold biomaterials have shown some potential to direct effective cartilage-like formation and repair, albeit with limited clinical translation. In this context, type II collagen (CII)-containing scaffolds have been recently developed by our research group and have demonstrated significant chondrogenic capacity using murine cells. However, the ability of these CII-containing scaffolds to support improved longer-lasting cartilage repair with reduced calcified cartilage formation still needs to be assessed in order to elucidate their potential therapeutic benefit to patients. To this end, CII-containing scaffolds in presence or absence of hyaluronic acid (HyA) within a type I collagen (CI) network were manufactured and cultured with human mesenchymal stem cells (MSCs) in vitro under chondrogenic conditions for 28 days. Consistent with our previous study in rat cells, the results revealed enhanced cartilage-like formation in the biomimetic scaffolds. In addition, while the variable chondrogenic abilities of human MSCs isolated from different donors were highlighted, protein expression analysis illustrated consistent responses in terms of the deposition of key cartilage extracellular matrix (ECM) components. Specifically, CI/II-HyA scaffolds directed the greatest cell-mediated synthesis and accumulation in the matrices of type II collagen (a principal cartilage ECM component), and reduced deposition of type X collagen (a key protein associated with hypertrophic cartilage formation). Taken together, these results provide further evidence of the capability of these CI/II-HyA scaffolds to direct enhanced and longer-lasting cartilage repair in patients with reduced hypertrophic cartilage formation