On Modeling Elastic and Inelastic Polarized Radiation Transport in the Earth Atmosphere with Monte Carlo Methods: On Modeling Elastic and Inelastic PolarizedRadiation Transport in the Earth Atmosphere withMonte Carlo Methods

The three dimensional Monte Carlo radiation transport model McArtim is extended to account for the simulation of the propagation of polarized radiation and the inelastic rotational Raman scattering which is the cause of the so called Ring effect. From the achieved and now sufficient precision of the calculated Ring effect new opportunities in optical absorption spectroscopy arise. In the calculation the method of importance sampling (IS) is applied. Thereby one obtains from an ensemble of Monte Carlo photon trajectories an intensity accounting for the elastic aerosol particle-, Cabannes- and the inelastic rotational Raman scattering (RRS) and simultaneously an intensity, for which Rayleigh scattering is treated as an elastic scattering process. By combining both intensities one obtains the so called filling-in (FI, which quantifies the filling-in of Fraunhofer lines) as a measure for the strength of the Ring effect with the same relative precision as the intensities. The validation of the polarized radiometric quantities and the Ring effect is made by comparison with partially published results of other radiation transport models. Furthermore the concept of discretisation of the optical domain into grid cells is extended by making grid cells arbitrarily joining into so called clusters, i.e. grid cell aggregates. Therewith the program is able to calculate derivatives of radiometrically or spectroscopically accessible quantities, namely the intensities at certain locations in the atmospheric radiation field and the light path integrals of trace gas concentrations associated thereto, i.e. the product of the DOAS (differential optical absorption spectroscopy) method, with respect to optical properties of aerosols and gases in connected spatial regions. The first and second order derivatives are validated through so called self-consistency tests. These derivatives allow the inversion of three dimensional tracegas and aerosol concentration profiles and pave the way down to 3D optical scattered light tomography. If such tomographic inversion scheme is based solely on spectral intensitites the available second order derivatives allows the consideration of the curvature in the cost function and therefore allows implementation of efficient optimisation algorithms. The influence of the instrument function on the spectra is analysed in order to mathematically assess the potential of DOAS to a sufficient degree. It turns out that the detailed knowledge of the instrument function is required for an advanced spectral analysis. Concludingly the mathematical separability of narrow band signatures of absorption and the Ring effect from the relatively broad band influence of the elastic scattering processes on the spectra is demonstrated which corresponds exactly to the DOAS principle. In that procedure the differential signal is obtained by approximately 4 orders of magnitude faster then by the separate modelling with and without narrow band structures. Thereby the fusion of the separated steps DOAS spectral analysis and subsequent radiation transport modeling becomes computationally feasible.:1.1. Radiation Transport Modeling and Atmospheric State Inversion 1.2. Vector RTE Solution Methods 1.3. Scope of the Thesis 1.4. Outline of the Thesis 2.1. General Structure 2.1.1. Chemical Composition of the Gas Phase 2.1.2. The Troposphere, Temperature and Pressure Vertical Structure 2.1.3. The Stratosphere 2.2. Aerosols and Clouds 2.2.1. Classification and Morphology 2.2.2. Water Related Particle Growth and Shrinking Processes 2.2.3. Size Spectra and Modes 3.1. Electromagnetic Waves 3.1.1. Maxwell\''s Equations 3.1.2. Measurement of Electromagnetic Waves 3.1.3. Polarization State of EM Waves 3.1.4. Stokes Vectors 3.2. Scattering and Absorption of EM Waves by Molecules and Particles 3.2.1. General Description of Scattering and Coordinate Systems 3.2.2. Molecular Scattering 3.2.3. Molecular Absorption Processes and Electronic Molecular States 3.2.4. Scattering On Spherical Particles - Mie Theory 3.3. Mathematical Description of Radiation Transport 3.3.1. Radiance and Irradiance 3.3.2. Absorption, Scattering and Extinction Coefficients 3.3.3. Optical Thickness and Transmission 3.3.4. Scattering 3.3.5. Incident (Ir)Radiance 3.3.6. The Black Surface Single Scattering Approximation 3.3.7. Radiative Transfer Equations 4.1. General Monte Carlo Methods 4.1.1. Numerical Integration 4.1.2. Importance Sampling and Zero Variance Estimates 4.1.3. Optimal Sampling 4.1.4. Sampling from Arbitrary Distributions 4.2. Path Generation or Collision Density Estimation 4.2.1. Discretization of the Optical Domain into Cells and Clusters 4.2.2. RTE Integral Form 4.2.3. Formal Solution of the IRTE 4.2.4. Overview on Monte Carlo RTE Solution Algorithms 4.2.5. Crude Monte Carlo 4.2.6. Sequential Importance Sampling (SIS) or Path Generation 4.3. Importance Sampling in Monte Carlo SIS Radiative Transfer 4.3.1. Weights for Alternate Kernels 4.3.2. Weights in the Calculation of RTE Functional Estimates 4.3.3. Application of IS to Mie Phase Functions Scatter Angle Sampling 5.1. Radiances, Intensities and the Reciprocity Theorem 5.1.1. Scalar Radiance Estimates 5.1.2. Backward Monte Carlo Scalar Radiance 5.1.3. Vector Radiances 5.2. Radiance Derivatives 5.2.1. Variables for Radiance Derivatives 5.3. Validation of Functionals 5.3.1. Validation of Vector Radiances 5.3.2. Validation of Radiance Derivatives 6.1. A Simply Structured Instrument Forward Model 6.2. Pure Atmospheric Spectra and Absorption 6.2.1. Direct Light Spectra 6.2.2. Scattered Sun Light Spectra 6.3. (D)OAS from the Perspective of Radiative Transfer Modeling 6.3.1. (Rest) Signatures of Weakly Absorbing Gases 6.3.2. Spectroscopic Measurements and Standard DOAS 6.4. DOAS Analysis Summary 6.4.1. DSCD Retrieval 6.4.2. Inversion 7.1. RRS-Modified RTE 7.1.1. RRS Cross Sections for Scattering out and into a Wavelength 7.1.2. Modification of the RTE Loss and Source Terms 7.2. Intensity Estimates Considering Rotational Raman Scattering 7.2.1. RRS in the Path Sampling Procedure 7.2.2. Adjoint RRS Correction Weights 7.2.3. Local Estimates of Intensities with RRS 7.2.4. Intensity Estimates 7.3. Ring Spectra 7.3.1. Elastic Biasing of the Local Estimates 7.3.2. Cumulative Weights and Local Estimates 7.3.3. Test of the Elastic Biasing 7.4. Validation 7.4.1. Comparison to an Analytic Single Scattering Code 7.4.2. Single Scattering Model Including Rotational Raman Scattering 7.4.3. Multiple Scattering Model Comparison 7.4.4. Comparison with A Measurement 7.4.5. Validation of Approximate Methods For Ring Effect Modeling 7.5. Summary and Discussion 8.1. Status and Summary 8.1.1. Ring-Effect and Absorption Corrected Radiances 8.1.2. Derivatives of Radiometric Quantities Accessible Through Spectroscopy 8.1.3. Polarization 8.1.4. Time Integrated Sensitivities for 3D UV/vis/NIR Remote Sensing 8.2. Outlook A.1. Zero Variance Estimates A.2. Free Path Length Sampling in a Homogeneous Medium A.3. Cumulative Differential Scatter Cross Sections A.3.1. Cardanic formulas A.3.2. Rayleigh and Raman Phase Functions A.3.3. Henyey-Greenstein Model A.3.4. Legendre Polynomial Phase Function Model A.3.5. Table Methods A.4. Greens Function in the Derivation of the IRTE A.5. Source Code For Stokes Vector Transformation Plot B.1. 1st Order Derivatives B.2. 2nd Order Derivatives B.3. Hessian of Integrals Depending on Many Variables C.1. Slit Function f Derivatives C.2. Signal Sn Derivatives C.3. Chi Square Spline Fitting C.3.1. Constrained Non-Linear Least Square Problem C.3.2. Spline Fitting C.3.3. Jacobians and Hessia

Inverse modeling of cloud-aerosol interactions – Part 1: Detailed response surface analysis

This is the final version of the article. Available from EGU via the DOI in this record.New methodologies are required to probe the sensitivity of parameters describing cloud droplet activation. This paper presents an inverse modeling-based method for exploring cloud-aerosol interactions via response surfaces. The objective function, containing the difference between the measured and model predicted cloud droplet size distribution is studied in a two-dimensional framework, and presented for pseudo-adiabatic cloud parcel model parameters that are pair-wise selected. From this response surface analysis it is shown that the susceptibility of cloud droplet size distribution to variations in different aerosol physiochemical parameters is highly dependent on the aerosol environment and meteorological conditions. In general the cloud droplet size distribution is most susceptible to changes in the updraft velocity. A shift towards an increase in the importance of chemistry for the cloud nucleating ability of particles is shown to exist somewhere between marine average and rural continental aerosol regimes. We also use these response surfaces to explore the feasibility of inverse modeling to determine cloud-aerosol interactions. It is shown that the "cloud-aerosol" inverse problem is particularly difficult to solve due to significant parameter interaction, presence of multiple regions of attraction, numerous local optima, and considerable parameter insensitivity. The identifiability of the model parameters will be dependent on the choice of the objective function. Sensitivity analysis is performed to investigate the location of the information content within the calibration data to confirm that our choice of objective function maximizes information retrieval from the cloud droplet size distribution. Cloud parcel models that employ a moving-centre based calculation of the cloud droplet size distribution pose additional difficulties when applying automatic search algorithms for studying cloud-aerosol interactions. To aid future studies, an increased resolution of the region of the size spectrum associated with droplet activation within cloud parcel models, or further development of fixed-sectional cloud models would be beneficial. Despite these improvements, it is demonstrated that powerful search algorithms remain necessary to efficiently explore the parameter space and successfully solve the cloud-aerosol inverse problem.We gratefully acknowledge the financial support of the Bert Bolin Centre for Climate research. We gratefully appreciate G. J. Roelofs, IMAU, Utrecht, the Netherlands, for providing us with the pseudo-adiabatic cloud parcel model used in this study. We gratefully acknowledge Hamish Struthers valuable discussions and his help to improve the readability of the manuscript. Some of the calculations made during the course of this study have been made possible using the LISA cluster from the SARA centre for parallel computing at the University of Amsterdam, the Netherlands. AS acknowledges support from an Office of Naval Research YIP award (N00014-10-1-0811).The authors acknowledge the Swedish Environmental Monitoring Program a

Inverse modelling of cloud-aerosol interactions – Part 2: Sensitivity tests on liquid phase clouds using a Markov chain Monte Carlo based simulation approach

This paper presents a novel approach to investigate cloud-aerosol interactions by coupling a Markov chain Monte Carlo (MCMC) algorithm to an adiabatic cloud parcel model. Despite the number of numerical cloud-aerosol sensitivity studies previously conducted few have used statistical analysis tools to investigate the global sensitivity of a cloud model to input aerosol physiochemical parameters. Using numerically generated cloud droplet number concentration (CDNC) distributions (i.e. synthetic data) as cloud observations, this inverse modelling framework is shown to successfully estimate the correct calibration parameters, and their underlying posterior probability distribution. <br></br> The employed analysis method provides a new, integrative framework to evaluate the global sensitivity of the derived CDNC distribution to the input parameters describing the lognormal properties of the accumulation mode aerosol and the particle chemistry. To a large extent, results from prior studies are confirmed, but the present study also provides some additional insights. There is a transition in relative sensitivity from very clean marine Arctic conditions where the lognormal aerosol parameters representing the accumulation mode aerosol number concentration and mean radius and are found to be most important for determining the CDNC distribution to very polluted continental environments (aerosol concentration in the accumulation mode >1000 cm<sup>−3</sup>) where particle chemistry is more important than both number concentration and size of the accumulation mode. <br></br> The competition and compensation between the cloud model input parameters illustrates that if the soluble mass fraction is reduced, the aerosol number concentration, geometric standard deviation and mean radius of the accumulation mode must increase in order to achieve the same CDNC distribution. <br></br> This study demonstrates that inverse modelling provides a flexible, transparent and integrative method for efficiently exploring cloud-aerosol interactions with respect to parameter sensitivity and correlation

Estimating model evidence using data assimilation

We review the field of data assimilation (DA) from a Bayesian perspective and show that, in addition to its by now common application to state estimation, DA may be used for model selection. An important special case of the latter is the discrimination between a factual model–which corresponds, to the best of the modeller's knowledge, to the situation in the actual world in which a sequence of events has occurred–and a counterfactual model, in which a particular forcing or process might be absent or just quantitatively different from the actual world. Three different ensemble‐DA methods are reviewed for this purpose: the ensemble Kalman filter (EnKF), the ensemble four‐dimensional variational smoother (En‐4D‐Var), and the iterative ensemble Kalman smoother (IEnKS). An original contextual formulation of model evidence (CME) is introduced. It is shown how to apply these three methods to compute CME, using the approximated time‐dependent probability distribution functions (pdfs) each of them provide in the process of state estimation. The theoretical formulae so derived are applied to two simplified nonlinear and chaotic models: (i) the Lorenz three‐variable convection model (L63), and (ii) the Lorenz 40‐variable midlatitude atmospheric dynamics model (L95). The numerical results of these three DA‐based methods and those of an integration based on importance sampling are compared. It is found that better CME estimates are obtained by using DA, and the IEnKS method appears to be best among the DA methods. Differences among the performance of the three DA‐based methods are discussed as a function of model properties. Finally, the methodology is implemented for parameter estimation and for event attribution

Aerosol formation and growth rates from chamber experiments using Kalman smoothing

Bayesian state estimation in the form of Kalman smoothing was applied to differential mobility analyser train (DMA-train) measurements of aerosol size distribution dynamics. Four experiments were analysed in order to estimate the aerosol size distribution, formation rate, and size-dependent growth rate, as functions of time. The first analysed case was a synthetic one, generated by a detailed aerosol dynamics model and the other three chamber experiments performed at the CERN CLOUD facility. The estimated formation and growth rates were compared with other methods used earlier for the CLOUD data and with the true values for the computer-generated synthetic experiment. The agreement in the growth rates was very good for all studied cases: estimations with an earlier method fell within the uncertainty limits of the Kalman smoother results. The formation rates also matched well, within roughly a factor of 2.5 in all cases, which can be considered very good considering the fact that they were estimated from data given by two different instruments, the other being the particle size magnifier (PSM), which is known to have large uncertainties close to its detection limit. The presented fixed interval Kalman smoother (FIKS) method has clear advantages compared with earlier methods that have been applied to this kind of data. First, FIKS can reconstruct the size distribution between possible size gaps in the measurement in such a way that it is consistent with aerosol size distribution dynamics theory, and second, the method gives rise to direct and reliable estimation of size distribution and process rate uncertainties if the uncertainties in the kernel functions and numerical models are known.Bayesian state estimation in the form of Kalman smoothing was applied to differential mobility analyser train (DMA-train) measurements of aerosol size distribution dynamics. Four experiments were analysed in order to estimate the aerosol size distribution, formation rate, and size-dependent growth rate, as functions of time. The first analysed case was a synthetic one, generated by a detailed aerosol dynamics model and the other three chamber experiments performed at the CERN CLOUD facility. The estimated formation and growth rates were compared with other methods used earlier for the CLOUD data and with the true values for the computer-generated synthetic experiment. The agreement in the growth rates was very good for all studied cases: estimations with an earlier method fell within the uncertainty limits of the Kalman smoother results. The formation rates also matched well, within roughly a factor of 2.5 in all cases, which can be considered very good considering the fact that they were estimated from data given by two different instruments, the other being the particle size magnifier (PSM), which is known to have large uncertainties close to its detection limit. The presented fixed interval Kalman smoother (FIKS) method has clear advantages compared with earlier methods that have been applied to this kind of data. First, FIKS can reconstruct the size distribution between possible size gaps in the measurement in such a way that it is consistent with aerosol size distribution dynamics theory, and second, the method gives rise to direct and reliable estimation of size distribution and process rate uncertainties if the un-certainties in the kernel functions and numerical models are known.Peer reviewe

Dimension reduction approaches for atmospheric remote sensing of greenhouse gases

Hiilidioksidi (CO2) ja metaani (CH4) ovat merkittävimmät ihmisperäiset kasvihuonekaasut, joilla molemmilla on huomattava vaikutus ilmastonmuutokseen ja ilmakehän lämpenemiseen. Näiden kaasujen pitoisuuksien epäsuorat kaukokartoitusmittaukset ovat oleellinen osa ihmisperäisten päästöjen kehityksen seurannassa. Näitä mittauksia tarvitaan myös arvioitaessa kasvihuonekaasujen vaikutusta ilmakehän prosesseihin. Edellämainitun tutkimuksen luotettavuus perustuu suurilta osin mittausten epävarmuuden arvioinnin paikkansapitävyyteen, minkä takaamiseksi käytetään korkeatasoista epävarmuusanalyysiä. Tämän väitöskirjatyön tavoitteena on kehittää ja ottaa käyttöön luotettavia ja laskennallisesti tehokkaita epävarmuusanalyysimenetelmiä sovellettuna kasvihuonekaasujen kaukokartoitukseen. Kehitetyt menetelmät perustuvat matemaattisesti käänteisongelmien teoriaan ja todennäköisyysteorian sovelluksiin. Käytämme erityisesti informaatioteoreettisia työkaluja pienentääksemme käänteisongelman ulottuvuutta. Tämä johtaa laskennalliseen ongelmaan, joka on huomattavasti nopeampi ratkaista. Työn sovelluskohteita ovat Nasan Orbiting Carbon Observatory 2 -satelliitin hiilidioksidipitoisuusmittaukset sekä Sodankylän Arktisessa Avaruuskeskuksessa sijaitsevan spektrometrin mittaamat metaanipitoisuudet. Jälkimmäisessä keskitymme sekä yksittäisiin mittauksiin että koko aikasarjan tutkimiseen ajalta 2009–2018. Kehitetyt menetelmät toimivat erittäin hyvin käsitellyissä sovelluksissa luoden pohjaa uusille operatiivisille epävarmuusanalyysialgoritmeille.Carbon dioxide (CO2) and methane (CH4) are two most significant anthropogenic greenhouse gases contributing to climate change and global warming. Indirect remote sensing measurements of atmospheric concentrations of these gases are crucial for monitoring manmade emissions and understanding their effects and related atmospheric processes. The reliability of these studies depends largely on robust uncertainty quantification of the measurements, which provides rigorous error estimates and confidence intervals for all results. The main goal of this work is to develop and implement rigorous, robust and computationally efficient means of uncertainty quantification for atmospheric remote sensing of greenhouse gases. We consider both CO2 measurements by NASA’s Orbiting Carbon Observatory 2 (OCO-2) and CH4 measurements by Sodankylä Arctic Space Center’s Fourier Transform Spectrometer (FTS), the latter being studied from the perspectives of both individual measurements, and the entire time series from 2009-2018. Our approach leverages recent mathematical results on dimension reduction to produce novel algorithms that are a step towards thorough and efficient operational uncertainty quantification in the field of atmospheric remote sensing. Mathematically, the process of inferring gas concentrations from indirect measurements is an ill-posed inverse problem, meaning that a well-defined solution doesn't exist without proper regularization. Bayesian approach utilizes probability theory to provide a regularized solution to the inverse problem as a posterior probability distribution. The posterior distribution is conventionally approximated using a Gaussian distribution, and results are reported as the mean of the distribution as a point estimate, and the corresponding variance as a measure of uncertainty. In reality, due to non-linear physics models used in the computations, the posterior is not well approximated by a Gaussian distribution, and ignoring its actual shape can lead to unpredictable errors and inaccuracies in the retrieval. Markov chain Monte Carlo (MCMC) methods offer a robust way to explore the actual properties of posterior distributions, but they tend to be computationally infeasible as the dimension of the state vector increases. In this work, the low intrinsic information content of remote sensing measurements is exploited to implement the Likelihood-Informed Subspace (LIS) dimension reduction method, which increases the computational efficiency of MCMC. Novel algorithms using LIS are implemented to abovementioned atmospheric CH4 profile and column-averaged CO2 concentration inverse problems