MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories

Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio, or equivalently the posterior function. We show that this approach can be formulated in terms of mutual information maximization between model parameters and simulated data. We use this equivalence to reinterpret existing approaches for amortized inference and propose two new methods that rely on lower bounds of the mutual information. We apply our framework to the inference of parameters of stochastic processes and chaotic dynamical systems from sampled trajectories, using artificial neural networks for posterior prediction. Our approach provides a unified framework that leverages the power of mutual information estimators for inference

Zagadnienia entropii w układach mezoskopowych

Praca rozpoczyna się krótkim wprowadzeniem do termodynamiki klasycznej. Ta część formułuje definicję entropii, jako wielkości postulowanej przez drugą zasadę termodynamiki. Druga część pracy poświęcona jest zagadnieniom związanym z interpretacją definicji entropii Gibbsa i Boltzmanna poza granicą termodynamiczną. W trzeciej części przedstawiono przykładowe zastosowania entropii informacyjnej poza mechaniką statystyczną.The thesis begins with the formulation of classical thermodynamics in the framework of differential geometry and thus provides a strict definition of thermodynamic entropy as postulated by the second law of thermodynamics.In the second part, several issues of interpretation of Boltzmann and Gibbs entropy definitions are addressed, as recent dispute concerning existence of negative (absolute) temperatures revealed that some of the cornerstones of statistical physics are still a matter of deliberations.Third part of the thesis introduces basic concepts in stochastic dynamics. Furthermore, we review some examples of application of entropy outside statistical mechanics. We analyse a simple model of signal transmission and reveal the role of noise in this systems by estimating the conveyed information