Sketch and project: randomized iterative methods for linear systems and inverting matrices

Probabilistic ideas and tools have recently begun to permeate into several fields where they had traditionally not played a major role, including fields such as numerical linear algebra and optimization. One of the key ways in which these ideas influence these fields is via the development and analysis of randomized algorithms for solving standard and new problems of these fields. Such methods are typically easier to analyze, and often lead to faster and/or more scalable and versatile methods in practice. This thesis explores the design and analysis of new randomized iterative methods for solving linear systems and inverting matrices. The methods are based on a novel sketch-and-project framework. By sketching we mean, to start with a difficult problem and then randomly generate a simple problem that contains all the solutions of the original problem. After sketching the problem, we calculate the next iterate by projecting our current iterate onto the solution space of the sketched problem. The starting point for this thesis is the development of an archetype randomized method for solving linear systems. Our method has six different but equivalent interpretations: sketch-and-project, constrain-and-approximate, random intersect, random linear solve, random update and random fixed point. By varying its two parameters – a positive definite matrix (defining geometry), and a random matrix (sampled in an i.i.d. fashion in each iteration) – we recover a comprehensive array of well known algorithms as special cases, including the randomized Kaczmarz method, randomized Newton method, randomized coordinate descent method and random Gaussian pursuit. We also naturally obtain variants of all these methods using blocks and importance sampling. However, our method allows for a much wider selection of these two parameters, which leads to a number of new specific methods. We prove exponential convergence of the expected norm of the error in a single theorem, from which existing complexity results for known variants can be obtained. However, we also give an exact formula for the evolution of the expected iterates, which allows us to give lower bounds on the convergence rate. We then extend our problem to that of finding the projection of given vector onto the solution space of a linear system. For this we develop a new randomized iterative algorithm: stochastic dual ascent (SDA). The method is dual in nature, and iteratively solves the dual of the projection problem. The dual problem is a non-strongly concave quadratic maximization problem without constraints. In each iteration of SDA, a dual variable is updated by a carefully chosen point in a subspace spanned by the columns of a random matrix drawn independently from a fixed distribution. The distribution plays the role of a parameter of the method. Our complexity results hold for a wide family of distributions of random matrices, which opens the possibility to fine-tune the stochasticity of the method to particular applications. We prove that primal iterates associated with the dual process converge to the projection exponentially fast in expectation, and give a formula and an insightful lower bound for the convergence rate. We also prove that the same rate applies to dual function values, primal function values and the duality gap. Unlike traditional iterative methods, SDA converges under virtually no additional assumptions on the system (e.g., rank, diagonal dominance) beyond consistency. In fact, our lower bound improves as the rank of the system matrix drops. By mapping our dual algorithm to a primal process, we uncover that the SDA method is the dual method with respect to the sketch-and-project method from the previous chapter. Thus our new more general convergence results for SDA carry over to the sketch-and-project method and all its specializations (randomized Kaczmarz, randomized coordinate descent...etc). When our method specializes to a known algorithm, we either recover the best known rates, or improve upon them. Finally, we show that the framework can be applied to the distributed average consensus problem to obtain an array of new algorithms. The randomized gossip algorithm arises as a special case. In the final chapter, we extend our method for solving linear system to inverting matrices, and develop a family of methods with specialized variants that maintain symmetry or positive definiteness of the iterates. All the methods in the family converge globally and exponentially, with explicit rates. In special cases, we obtain stochastic block variants of several quasi-Newton updates, including bad Broyden (BB), good Broyden (GB), Powell-symmetric-Broyden (PSB), Davidon-Fletcher-Powell (DFP) and Broyden-Fletcher-Goldfarb-Shanno (BFGS). Ours are the first stochastic versions of these updates shown to converge to an inverse of a fixed matrix. Through a dual viewpoint we uncover a fundamental link between quasi-Newton updates and approximate inverse preconditioning. Further, we develop an adaptive variant of the randomized block BFGS (AdaRBFGS), where we modify the distribution underlying the stochasticity of the method throughout the iterative process to achieve faster convergence. By inverting several matrices from varied applications, we demonstrate that AdaRBFGS is highly competitive when compared to the well established Newton-Schulz and approximate preconditioning methods. In particular, on large-scale problems our method outperforms the standard methods by orders of magnitude. The development of efficient methods for estimating the inverse of very large matrices is a much needed tool for preconditioning and variable metric methods in the big data era

Parallel algorithms for nonlinear optimization

Parallel algorithm design is a very active research topic in optimization as parallel computer architectures have recently become easily accessible. This thesis is about an approach for designing parallel nonlinear programming algorithms. The main idea is to benefit from parallelization in designing new algorithms rather than considering direct parallelizations of the existing methods. We give a general framework following our approach, and then, give distinct algorithms that fit into this framework. The example algorithms we have designed either use procedures of existing methods within a multistart scheme, or they are completely new inherently parallel algorithms. In doing so, we try to show how it is possible to achieve parallelism in algorithm structure (at different levels) so that the resulting algorithms have a good solution performance in terms of robustness, quality of steps, and scalability. We complement our discussion with convergence proofs of the proposed algorithms

Convex Optimization and Extensions, with a View Toward Large-Scale Problems

Machine learning is a major source of interesting optimization problems of current interest. These problems tend to be challenging because of their enormous scale, which makes it difficult to apply traditional optimization algorithms. We explore three avenues to designing algorithms suited to handling these challenges, with a view toward large-scale ML tasks. The first is to develop better general methods for unconstrained minimization. The second is to tailor methods to the features of modern systems, namely the availability of distributed computing. The third is to use specialized algorithms to exploit specific problem structure. Chapters 2 and 3 focus on improving quasi-Newton methods, a mainstay of unconstrained optimization. In Chapter 2, we analyze an extension of quasi-Newton methods wherein we use block updates, which add curvature information to the Hessian approximation on a higher-dimensional subspace. This defines a family of methods, Block BFGS, that form a spectrum between the classical BFGS method and Newton's method, in terms of the amount of curvature information used. We show that by adding a correction step, the Block BFGS method inherits the convergence guarantees of BFGS for deterministic problems, most notably a Q-superlinear convergence rate for strongly convex problems. To explore the tradeoff between reduced iterations and greater work per iteration of block methods, we present a set of numerical experiments. In Chapter 3, we focus on the problem of step size determination. To obviate the need for line searches, and for pre-computing fixed step sizes, we derive an analytic step size, which we call curvature-adaptive, for self-concordant functions. This adaptive step size allows us to generalize the damped Newton method of Nesterov to other iterative methods, including gradient descent and quasi-Newton methods. We provide simple proofs of convergence, including superlinear convergence for adaptive BFGS, allowing us to obtain superlinear convergence without line searches. In Chapter 4, we move from general algorithms to hardware-influenced algorithms. We consider a form of distributed stochastic gradient descent that we call Leader SGD, which is inspired by the Elastic Averaging SGD method. These methods are intended for distributed settings where communication between machines may be expensive, making it important to set their consensus mechanism. We show that LSGD avoids an issue with spurious stationary points that affects EASGD, and provide a convergence analysis of LSGD. In the stochastic strongly convex setting, LSGD converges at the rate O(1/k) with diminishing step sizes, matching other distributed methods. We also analyze the impact of varying communication delays, stochasticity in the selection of the leader points, and under what conditions LSGD may produce better search directions than the gradient alone. In Chapter 5, we switch again to focus on algorithms to exploit problem structure. Specifically, we consider problems where variables satisfy multiaffine constraints, which motivates us to apply the Alternating Direction Method of Multipliers (ADMM). Problems that can be formulated with such a structure include representation learning (e.g with dictionaries) and deep learning. We show that ADMM can be applied directly to multiaffine problems. By extending the theory of nonconvex ADMM, we prove that ADMM is convergent on multiaffine problems satisfying certain assumptions, and more broadly, analyze the theoretical properties of ADMM for general problems, investigating the effect of different types of structure

Fluid-electro-mechanical model of the human heart for supercomputers

The heart is a complex system. From the transmembrane cell activity to the spatial organization in helicoidal fibers, it includes several spatial and temporal scales. The heart muscle is surrounded by two main tissues that modulate how it deforms: the pericardium and the blood. The former constrains the epicardial surface and the latter exerts a force in the endocardium. The main function of this peculiar muscle is to pump blood to the pulmonary and systemic circulations. In this way, solid dynamics of the heart is as important as the induced fluid dynamics. Despite the work done in computational research of multiphysics heart modelling, there is no reference of a tightly-coupled scheme that includes electrophysiology, solid and fluid mechanics in a whole human heart. In this work, we propose, develop and test a fluid-electro-mechanical model of the human heart. To start, the heartbeat phenomenon is disassembled in the different composing problems. The first building block is the electrical activity of the myocytes, that induces the mechanical deformation of the myocardium. The contraction of the muscle reduces the intracavitary space, that pushes out the contained blood. At the same time, the inertia, pressure and viscous stresses in this fluid exerts a force on the solid wall. In this way, we can understand the heart as a fluid-electro-mechanical problem. All the models are implemented in Alya, the Barcelona Supercomputing Center simulation software. A multi-code approach is used, splitting the problem in a solid and a fluid domain. In the former, electrophysiology coupled with solid mechanics are solved. In the later, fluid dynamics in an arbitrary Lagrangian-Eulerian domain are computed. The equations are spatially discretized using the finite element method and temporally discretized using finite differences. Facilitated by the multi-code approach, a novel high performance quasi-Newton method is developed to deal with the intrinsic issues of fluid-structure interaction problems in iomechanics. All the schemes are optimized to run in massively parallel computers. A wide range of experiments are shown to validate, test and tune the numerical model. The different hypothesis proposed — as the critical effect of the atrium or the presence of pericardium — are also tested in these experiments. Finally, a normal heartbeat is simulated and deeply analyzed. This healthy computational heart is first diseased with a left bundle branch block. After this, its function is restored simulating a cardiac resynchronization therapy. Then, a third grade atrioventricular block is simulated in the healthy heart. In this case, the pathologic model is treated with a minimally invasive leadless intracardiac pacemaker. This requires to include the device in the geometrical description of the problem, solve the structural problem with the tissue, and the fluid-structure interaction problem with the blood. As final experiment, we test the parallel performance of the coupled solver. In the cases mentioned above, the results are qualitatively compared against experimental measurements, when possible. Finally, a first glance in a coupled fluid-electro-mechanical cardiovascular system is shown. This model is build adding a one dimensional model of the arterial network created by the Laboratório Nacional de Computação Científica in Petropolis, Brasil. Despite the artificial geometries used, the outflow curves are comparable with physiological observations. The model presented in this thesis is a step towards the virtual human heart. In a near future computational models like the presented in this thesis will change how pathologies are understood and treated, and the way biomedical devices are designed.El corazón es un sistema complejo. Desde la actividad celular hasta la organización espacial en fibras helicoidales, incluye gran cantidad de escalas espaciales y temporales. El corazón está rodeado principalmente por dos tejidos que modulan su deformación: el pericardio y la sangre. El primero restringe el movimiento del epicardio, mientras el segundo ejerce fuerza sobre el endocardio. La función principal de este músculo es bombear sangre a la circulación sistémica y a la pulmonar. Así, la deformación del miocardio es tan importante como la fluidodinámica inducida. Al día de hoy, solo se han propuesto modelos parciales del corazón. Ninguno de los modelos publicados resuelve electrofisiología, mecánica del sólido, y dinámica de fluidos en una geometría completa del corazón. En esta tesis, proponemos, desarrollamos y probamos un modelo fluido -electro -mecánico del corazón. Primero, el problema del latido cardíaco es descompuesto en los distintos subproblemas. El primer bloque componente es la actividad eléctrica de los miocitos, que inducen la deformación mecánica del miocardio. La contratación de este músculo, reduce el espacio intracavitario, que empuja la sangre contenida. Al mismo tiempo, la inercia, presión y fuerzas viscosas del fluido inducen una presión sobre la pared del sólido. De esta manera, podemos entender el latido cardíaco como un problema fluido-electro-mecánico. Los modelos son implementados en Alya, el software de simulación del Barcelona Supercomputing Center. Se utiliza un diseño multi-código, separando el problema según el dominio en sólido y fluido. En el primero, se resuelve electrofisiología acoplado con mecánica del sólido. En el segundo, fluido dinámica en un dominio arbitrario Lagrangiano-Euleriano. Las ecuaciones son discretizadas espacial y temporalmente utilizando elementos finitos y diferencias finitas respectivamente. Facilitado por el diseño multi-codigo, se desarrolló un novedoso método quasi-Newton de alta performance, pensado específicamente para lidiar con los problemas intrínsecos de interacción fluido-estructura en biomecánica. Todos los esquemas fueron optimizados para correr en ordenadores masivamente paralelos.Se presenta un amplio espectro de experimentos con el fin de validar, probar y ajustar el modelo numérico. Las diferentes hipótesis propuestas tales como el efecto producido por la presencia de las aurículas o el pericardio son también demostradas en estos experimentos. Finalmente un latido normal es simulado y sus resultados son analizados con profundidad. El corazón computacional sano es, primeramente enfermado de un bloqueo de rama izquierda. Posteriormente se restaura la función normal mediante la terapia de resincronización cardíaca. Luego se afecta al corazón de un bloqueo atrioventricular de tercer grado. Esta patología es tratada mediante la implantación de un marcapasos intracardíaco. Para esto, se requiere incluir el dispositivo en la descripción geométrica, resolver el problema estructural con el tejido y la interacción fluido-estructura con la sangre. Como experimento numérico final, se prueba el desempeño paralelo del modelo acoplado.Finalmente, se muestran resultados preliminares para un modelo fluido-electro-mecánico del sistema cardiovascular. Este modelo se construye agregando un modelo unidimensional del árbol arterial. A pesar de las geometrías artificiales usadas, la curva de flujo en la raíz aórtica es comparable con observaciones experimentales. El modelo presentado aquí representa un avance hacia el humano virtual. En un futuro, modelos similares, cambiarán la forma en la que se entienden y tratan las enfermedades y la forma en la que los dispositivos biomédicos son diseñados.Postprint (published version

Fluid-electro-mechanical model of the human heart for supercomputers

The heart is a complex system. From the transmembrane cell activity to the spatial organization in helicoidal fibers, it includes several spatial and temporal scales. The heart muscle is surrounded by two main tissues that modulate how it deforms: the pericardium and the blood. The former constrains the epicardial surface and the latter exerts a force in the endocardium. The main function of this peculiar muscle is to pump blood to the pulmonary and systemic circulations. In this way, solid dynamics of the heart is as important as the induced fluid dynamics. Despite the work done in computational research of multiphysics heart modelling, there is no reference of a tightly-coupled scheme that includes electrophysiology, solid and fluid mechanics in a whole human heart. In this work, we propose, develop and test a fluid-electro-mechanical model of the human heart. To start, the heartbeat phenomenon is disassembled in the different composing problems. The first building block is the electrical activity of the myocytes, that induces the mechanical deformation of the myocardium. The contraction of the muscle reduces the intracavitary space, that pushes out the contained blood. At the same time, the inertia, pressure and viscous stresses in this fluid exerts a force on the solid wall. In this way, we can understand the heart as a fluid-electro-mechanical problem. All the models are implemented in Alya, the Barcelona Supercomputing Center simulation software. A multi-code approach is used, splitting the problem in a solid and a fluid domain. In the former, electrophysiology coupled with solid mechanics are solved. In the later, fluid dynamics in an arbitrary Lagrangian-Eulerian domain are computed. The equations are spatially discretized using the finite element method and temporally discretized using finite differences. Facilitated by the multi-code approach, a novel high performance quasi-Newton method is developed to deal with the intrinsic issues of fluid-structure interaction problems in iomechanics. All the schemes are optimized to run in massively parallel computers. A wide range of experiments are shown to validate, test and tune the numerical model. The different hypothesis proposed — as the critical effect of the atrium or the presence of pericardium — are also tested in these experiments. Finally, a normal heartbeat is simulated and deeply analyzed. This healthy computational heart is first diseased with a left bundle branch block. After this, its function is restored simulating a cardiac resynchronization therapy. Then, a third grade atrioventricular block is simulated in the healthy heart. In this case, the pathologic model is treated with a minimally invasive leadless intracardiac pacemaker. This requires to include the device in the geometrical description of the problem, solve the structural problem with the tissue, and the fluid-structure interaction problem with the blood. As final experiment, we test the parallel performance of the coupled solver. In the cases mentioned above, the results are qualitatively compared against experimental measurements, when possible. Finally, a first glance in a coupled fluid-electro-mechanical cardiovascular system is shown. This model is build adding a one dimensional model of the arterial network created by the Laboratório Nacional de Computação Científica in Petropolis, Brasil. Despite the artificial geometries used, the outflow curves are comparable with physiological observations. The model presented in this thesis is a step towards the virtual human heart. In a near future computational models like the presented in this thesis will change how pathologies are understood and treated, and the way biomedical devices are designed.El corazón es un sistema complejo. Desde la actividad celular hasta la organización espacial en fibras helicoidales, incluye gran cantidad de escalas espaciales y temporales. El corazón está rodeado principalmente por dos tejidos que modulan su deformación: el pericardio y la sangre. El primero restringe el movimiento del epicardio, mientras el segundo ejerce fuerza sobre el endocardio. La función principal de este músculo es bombear sangre a la circulación sistémica y a la pulmonar. Así, la deformación del miocardio es tan importante como la fluidodinámica inducida. Al día de hoy, solo se han propuesto modelos parciales del corazón. Ninguno de los modelos publicados resuelve electrofisiología, mecánica del sólido, y dinámica de fluidos en una geometría completa del corazón. En esta tesis, proponemos, desarrollamos y probamos un modelo fluido -electro -mecánico del corazón. Primero, el problema del latido cardíaco es descompuesto en los distintos subproblemas. El primer bloque componente es la actividad eléctrica de los miocitos, que inducen la deformación mecánica del miocardio. La contratación de este músculo, reduce el espacio intracavitario, que empuja la sangre contenida. Al mismo tiempo, la inercia, presión y fuerzas viscosas del fluido inducen una presión sobre la pared del sólido. De esta manera, podemos entender el latido cardíaco como un problema fluido-electro-mecánico. Los modelos son implementados en Alya, el software de simulación del Barcelona Supercomputing Center. Se utiliza un diseño multi-código, separando el problema según el dominio en sólido y fluido. En el primero, se resuelve electrofisiología acoplado con mecánica del sólido. En el segundo, fluido dinámica en un dominio arbitrario Lagrangiano-Euleriano. Las ecuaciones son discretizadas espacial y temporalmente utilizando elementos finitos y diferencias finitas respectivamente. Facilitado por el diseño multi-codigo, se desarrolló un novedoso método quasi-Newton de alta performance, pensado específicamente para lidiar con los problemas intrínsecos de interacción fluido-estructura en biomecánica. Todos los esquemas fueron optimizados para correr en ordenadores masivamente paralelos.Se presenta un amplio espectro de experimentos con el fin de validar, probar y ajustar el modelo numérico. Las diferentes hipótesis propuestas tales como el efecto producido por la presencia de las aurículas o el pericardio son también demostradas en estos experimentos. Finalmente un latido normal es simulado y sus resultados son analizados con profundidad. El corazón computacional sano es, primeramente enfermado de un bloqueo de rama izquierda. Posteriormente se restaura la función normal mediante la terapia de resincronización cardíaca. Luego se afecta al corazón de un bloqueo atrioventricular de tercer grado. Esta patología es tratada mediante la implantación de un marcapasos intracardíaco. Para esto, se requiere incluir el dispositivo en la descripción geométrica, resolver el problema estructural con el tejido y la interacción fluido-estructura con la sangre. Como experimento numérico final, se prueba el desempeño paralelo del modelo acoplado.Finalmente, se muestran resultados preliminares para un modelo fluido-electro-mecánico del sistema cardiovascular. Este modelo se construye agregando un modelo unidimensional del árbol arterial. A pesar de las geometrías artificiales usadas, la curva de flujo en la raíz aórtica es comparable con observaciones experimentales. El modelo presentado aquí representa un avance hacia el humano virtual. En un futuro, modelos similares, cambiarán la forma en la que se entienden y tratan las enfermedades y la forma en la que los dispositivos biomédicos son diseñados

Control Theory-Inspired Acceleration of the Gradient-Descent Method: Centralized and Distributed

Mathematical optimization problems are prevalent across various disciplines in science and engineering. Particularly in electrical engineering, convex and non-convex optimization problems are well-known in signal processing, estimation, control, and machine learning research. In many of these contemporary applications, the data points are dispersed over several sources. Restrictions such as industrial competition, administrative regulations, and user privacy have motivated significant research on distributed optimization algorithms for solving such data-driven modeling problems. The traditional gradient-descent method can solve optimization problems with differentiable cost functions. However, the speed of convergence of the gradient-descent method and its accelerated variants is highly influenced by the conditioning of the optimization problem being solved. Specifically, when the cost is ill-conditioned, these methods (i) require many iterations to converge and (ii) are highly unstable against process noise. In this dissertation, we propose novel optimization algorithms, inspired by control-theoretic tools, that can significantly attenuate the influence of the problem's conditioning. First, we consider solving the linear regression problem in a distributed server-agent network. We propose the Iteratively Pre-conditioned Gradient-Descent (IPG) algorithm to mitigate the deleterious impact of the data points' conditioning on the convergence rate. We show that the IPG algorithm has an improved rate of convergence in comparison to both the classical and the accelerated gradient-descent methods. We further study the robustness of IPG against system noise and extend the idea of iterative pre-conditioning to stochastic settings, where the server updates the estimate based on a randomly selected data point at every iteration. In the same distributed environment, we present theoretical results on the local convergence of IPG for solving convex optimization problems. Next, we consider solving a system of linear equations in peer-to-peer multi-agent networks and propose a decentralized pre-conditioning technique. The proposed algorithm converges linearly, with an improved convergence rate than the decentralized gradient-descent. Considering the practical scenario where the computations performed by the agents are corrupted, or a communication delay exists between them, we study the robustness guarantee of the proposed algorithm and a variant of it. We apply the proposed algorithm for solving decentralized state estimation problems. Further, we develop a generic framework for adaptive gradient methods that solve non-convex optimization problems. Here, we model the adaptive gradient methods in a state-space framework, which allows us to exploit control-theoretic methodology in analyzing Adam and its prominent variants. We then utilize the classical transfer function paradigm to propose new variants of a few existing adaptive gradient methods. Applications on benchmark machine learning tasks demonstrate our proposed algorithms' efficiency. Our findings suggest further exploration of the existing tools from control theory in complex machine learning problems. The dissertation is concluded by showing that the potential in the previously mentioned idea of IPG goes beyond solving generic optimization problems through the development of a novel distributed beamforming algorithm and a novel observer for nonlinear dynamical systems, where IPG's robustness serves as a foundation in our designs. The proposed IPG for distributed beamforming (IPG-DB) facilitates a rapid establishment of communication links with far-field targets while jamming potential adversaries without assuming any feedback from the receivers, subject to unknown multipath fading in realistic environments. The proposed IPG observer utilizes a non-symmetric pre-conditioner, like IPG, as an approximation of the observability mapping's inverse Jacobian such that it asymptotically replicates the Newton observer with an additional advantage of enhanced robustness against measurement noise. Empirical results are presented, demonstrating both of these methods' efficiency compared to the existing methodologies

Coupling schemes and inexact Newton for multi-physics and coupled optimization problems

This work targets mathematical solutions and software for complex numerical simulation and optimization problems. Characteristics are the combination of different models and software modules and the need for massively parallel execution on supercomputers. We consider two different types of multi-component problems in Part I and Part II of the thesis: (i) Surface coupled fluid- structure interactions and (ii) analysis of medical MR imaging data of brain tumor patients. In (i), we establish highly accurate simulations by combining different aspects such as fluid flow and arterial wall deformation in hemodynamics simulations or fluid flow, heat transfer and mechanical stresses in cooling systems. For (ii), we focus on (a) facilitating the transfer of information such as functional brain regions from a statistical healthy atlas brain to the individual patient brain (which is topologically different due to the tumor), and (b) to allow for patient specific tumor progression simulations based on the estimation of biophysical parameters via inverse tumor growth simulation (given a single snapshot in time, only). Applications and specific characteristics of both problems are very distinct, yet both are hallmarked by strong inter-component relations and result in formidable, very large, coupled systems of partial differential equations. Part I targets robust and efficient quasi-Newton methods for black-box surface-coupling of parti- tioned fluid-structure interaction simulations. The partitioned approach allows for great flexibility and exchangeable of sub-components. However, breaking up multi-physics into single components requires advanced coupling strategies to ensure correct inter-component relations and effectively tackle instabilities. Due to the black-box paradigm, solver internals are hidden and information exchange is reduced to input/output relations. We develop advanced quasi-Newton methods that effectively establish the equation coupling of two (or more) solvers based on solving a non-linear fixed-point equation at the interface. Established state of the art methods fall short by either requiring costly tuning of problem dependent parameters, or becoming infeasible for large scale problems. In developing parameter-free, linear-complexity alternatives, we lift the robustness and parallel scalability of quasi-Newton methods for partitioned surface-coupled multi-physics simulations to a new level. The developed methods are implemented in the parallel, general purpose coupling tool preCICE. Part II targets MR image analysis of glioblastoma multiforme pathologies and patient specific simulation of brain tumor progression. We apply a joint medical image registration and biophysical inversion strategy, targeting at facilitating diagnosis, aiding and supporting surgical planning, and improving the efficacy of brain tumor therapy. We propose two problem formulations and decompose the resulting large-scale, highly non-linear and non-convex PDE-constrained optimization problem into two tightly coupled problems: inverse tumor simulation and medical image registration. We deduce a novel, modular Picard iteration-type solution strategy. We are the first to successfully solve the inverse tumor-growth problem based on a single patient snapshot with a gradient-based approach. We present the joint inversion framework SIBIA, which scales to very high image resolutions and parallel execution on tens of thousands of cores. We apply our methodology to synthetic and actual clinical data sets and achieve excellent normal-to-abnormal registration quality and present a proof of concept for a very promising strategy to obtain clinically relevant biophysical information. Advanced inexact-Newton methods are an essential tool for both parts. We connect the two parts by pointing out commonalities and differences of variants used in the two communities in unified notation