A Simple Derivation of the Refined Sphere Packing Bound Under Certain Symmetry Hypotheses

A judicious application of the Berry-Esseen theorem via suitable Augustin information measures is demonstrated to be sufficient for deriving the sphere packing bound with a prefactor that is $\mathit{\Omega}\left(n^{-0.5(1-E_{sp}'(R))}\right)$ for all codes on certain families of channels -- including the Gaussian channels and the non-stationary Renyi symmetric channels -- and for the constant composition codes on stationary memoryless channels. The resulting non-asymptotic bounds have definite approximation error terms. As a preliminary result that might be of interest on its own, the trade-off between type I and type II error probabilities in the hypothesis testing problem with (possibly non-stationary) independent samples is determined up to some multiplicative constants, assuming that the probabilities of both types of error are decaying exponentially with the number of samples, using the Berry-Esseen theorem.Comment: 20 page

Simulacija rijetkih događaja

Standardnim Monte Carlo algoritmom teško je precizno procijeniti vjerojatnosti iznimno rijetkih događaja zbog velike relativne pogreške takvih procjenitelja. Cilj ovog diplomskog rada bio je prikazati neke od simulacijskih tehnika kojima se taj problem može zaobići. Naglasak je stavljen na uzorkovanje po važnosti i uvjetne Monte Carlo metode kao tehnike redukcije varijance procjenitelja. Precizno su definirane mjere efikasnosti algoritama poput ograničenosti relativne pogreške i logaritamske efikasnosti. Opisano je nekoliko algoritama koji se uglavnom bave procjenom vjerojatnosti prelaska praga u slučajnim šetnjama u slučajevima distribucija skokova lakog i teškog repa te su dokazana njihova teorijska svojstva efikasnosti. Algoritmi u slučaju distribucija lakog repa temelje se na eksponencijalnoj promjeni mjere u uzorkovanju po važnosti, a posebno je obrađen Siegmundov algoritam za koji je nađena optimalna distribucija važnosti. Također je spomenuta poveznica s teorijom velikih devijacija. Algoritmi u slučaju teških repova znatno su manje razvijeni od onih u slučaju lakih repova, te je za njih često teško pronaći dobru distribuciju važnosti. U ovom radu obrađeni su neki primjeri algoritama u slučaju subeksponencijalnih distribucija (s naglaskom na Paretovu i Weibullovu distribuciju) koji se temelje na uvjetnoj Monte Carlo metodi ili uzorkovanju po važnosti. U radu se navodi više stvarnih primjera primjene ovih algoritama, poput vjerojatnosti propasti u osiguravajućim društvima. Na kraju su algoritmi implementirani u MATLAB-u te uspoređeni u simulacijama. Siegmundov algoritam pokazao se dobrim i u praksi, dok su za slučaj teških repova bolje rezultate dali algoritmi uvjetne Monte Carlo metode nego uzorkovanja po važnosti.The probability of exceptionally rare events is difficult to estimate accurately using the standard Monte Carlo algorithm due to a high relative error of such estimators. The aim of this master’s thesis is to present some simulation techniques which may overcome this problem, with the emphasis on the importance sampling and conditional Monte Carlo method as variance reduction techniques. Algorithm efficiency measures, such as the bounded relative error and logarithmic efficiency, are precisely defined. Several algorithms are described, mainly concerning hitting times probabilities in light and heavy tailed random walks, and their theoretical efficiency proven. For light tailed distributions, the algorithms are based on the exponential change of measure in the importance sampling. The Siegmund’s algorithm is presented, for which the optimal importance distribution has been determined. A brief explanation of large deviations approach to optimal exponential change of measure is also provided. Algorithms for heavy tailed distributions are substantially less developed and it is often difficult to find a good importance distribution. This paper presents some algorithms in the case of subexponential distributions (with the emphasis on Pareto and Weibull distribution), which are based on the conditional Monte Carlo method and importance sampling. Several examples of real life application of the algorithms are discussed, such as ruin probabilities in insurance. Finally, the algorithms have been implemented in MATLAB and compared in simulations. The Siegmund’s algorithm has also proven good in practice, whereas in heavy tailed case the conditional Monte Carlo algorithms have shown better performance than the importance sampling ones

Simulacija rijetkih događaja

Standardnim Monte Carlo algoritmom teško je precizno procijeniti vjerojatnosti iznimno rijetkih događaja zbog velike relativne pogreške takvih procjenitelja. Cilj ovog diplomskog rada bio je prikazati neke od simulacijskih tehnika kojima se taj problem može zaobići. Naglasak je stavljen na uzorkovanje po važnosti i uvjetne Monte Carlo metode kao tehnike redukcije varijance procjenitelja. Precizno su definirane mjere efikasnosti algoritama poput ograničenosti relativne pogreške i logaritamske efikasnosti. Opisano je nekoliko algoritama koji se uglavnom bave procjenom vjerojatnosti prelaska praga u slučajnim šetnjama u slučajevima distribucija skokova lakog i teškog repa te su dokazana njihova teorijska svojstva efikasnosti. Algoritmi u slučaju distribucija lakog repa temelje se na eksponencijalnoj promjeni mjere u uzorkovanju po važnosti, a posebno je obrađen Siegmundov algoritam za koji je nađena optimalna distribucija važnosti. Također je spomenuta poveznica s teorijom velikih devijacija. Algoritmi u slučaju teških repova znatno su manje razvijeni od onih u slučaju lakih repova, te je za njih često teško pronaći dobru distribuciju važnosti. U ovom radu obrađeni su neki primjeri algoritama u slučaju subeksponencijalnih distribucija (s naglaskom na Paretovu i Weibullovu distribuciju) koji se temelje na uvjetnoj Monte Carlo metodi ili uzorkovanju po važnosti. U radu se navodi više stvarnih primjera primjene ovih algoritama, poput vjerojatnosti propasti u osiguravajućim društvima. Na kraju su algoritmi implementirani u MATLAB-u te uspoređeni u simulacijama. Siegmundov algoritam pokazao se dobrim i u praksi, dok su za slučaj teških repova bolje rezultate dali algoritmi uvjetne Monte Carlo metode nego uzorkovanja po važnosti.The probability of exceptionally rare events is difficult to estimate accurately using the standard Monte Carlo algorithm due to a high relative error of such estimators. The aim of this master’s thesis is to present some simulation techniques which may overcome this problem, with the emphasis on the importance sampling and conditional Monte Carlo method as variance reduction techniques. Algorithm efficiency measures, such as the bounded relative error and logarithmic efficiency, are precisely defined. Several algorithms are described, mainly concerning hitting times probabilities in light and heavy tailed random walks, and their theoretical efficiency proven. For light tailed distributions, the algorithms are based on the exponential change of measure in the importance sampling. The Siegmund’s algorithm is presented, for which the optimal importance distribution has been determined. A brief explanation of large deviations approach to optimal exponential change of measure is also provided. Algorithms for heavy tailed distributions are substantially less developed and it is often difficult to find a good importance distribution. This paper presents some algorithms in the case of subexponential distributions (with the emphasis on Pareto and Weibull distribution), which are based on the conditional Monte Carlo method and importance sampling. Several examples of real life application of the algorithms are discussed, such as ruin probabilities in insurance. Finally, the algorithms have been implemented in MATLAB and compared in simulations. The Siegmund’s algorithm has also proven good in practice, whereas in heavy tailed case the conditional Monte Carlo algorithms have shown better performance than the importance sampling ones

Simulacija rijetkih događaja

Standardnim Monte Carlo algoritmom teško je precizno procijeniti vjerojatnosti iznimno rijetkih događaja zbog velike relativne pogreške takvih procjenitelja. Cilj ovog diplomskog rada bio je prikazati neke od simulacijskih tehnika kojima se taj problem može zaobići. Naglasak je stavljen na uzorkovanje po važnosti i uvjetne Monte Carlo metode kao tehnike redukcije varijance procjenitelja. Precizno su definirane mjere efikasnosti algoritama poput ograničenosti relativne pogreške i logaritamske efikasnosti. Opisano je nekoliko algoritama koji se uglavnom bave procjenom vjerojatnosti prelaska praga u slučajnim šetnjama u slučajevima distribucija skokova lakog i teškog repa te su dokazana njihova teorijska svojstva efikasnosti. Algoritmi u slučaju distribucija lakog repa temelje se na eksponencijalnoj promjeni mjere u uzorkovanju po važnosti, a posebno je obrađen Siegmundov algoritam za koji je nađena optimalna distribucija važnosti. Također je spomenuta poveznica s teorijom velikih devijacija. Algoritmi u slučaju teških repova znatno su manje razvijeni od onih u slučaju lakih repova, te je za njih često teško pronaći dobru distribuciju važnosti. U ovom radu obrađeni su neki primjeri algoritama u slučaju subeksponencijalnih distribucija (s naglaskom na Paretovu i Weibullovu distribuciju) koji se temelje na uvjetnoj Monte Carlo metodi ili uzorkovanju po važnosti. U radu se navodi više stvarnih primjera primjene ovih algoritama, poput vjerojatnosti propasti u osiguravajućim društvima. Na kraju su algoritmi implementirani u MATLAB-u te uspoređeni u simulacijama. Siegmundov algoritam pokazao se dobrim i u praksi, dok su za slučaj teških repova bolje rezultate dali algoritmi uvjetne Monte Carlo metode nego uzorkovanja po važnosti.The probability of exceptionally rare events is difficult to estimate accurately using the standard Monte Carlo algorithm due to a high relative error of such estimators. The aim of this master’s thesis is to present some simulation techniques which may overcome this problem, with the emphasis on the importance sampling and conditional Monte Carlo method as variance reduction techniques. Algorithm efficiency measures, such as the bounded relative error and logarithmic efficiency, are precisely defined. Several algorithms are described, mainly concerning hitting times probabilities in light and heavy tailed random walks, and their theoretical efficiency proven. For light tailed distributions, the algorithms are based on the exponential change of measure in the importance sampling. The Siegmund’s algorithm is presented, for which the optimal importance distribution has been determined. A brief explanation of large deviations approach to optimal exponential change of measure is also provided. Algorithms for heavy tailed distributions are substantially less developed and it is often difficult to find a good importance distribution. This paper presents some algorithms in the case of subexponential distributions (with the emphasis on Pareto and Weibull distribution), which are based on the conditional Monte Carlo method and importance sampling. Several examples of real life application of the algorithms are discussed, such as ruin probabilities in insurance. Finally, the algorithms have been implemented in MATLAB and compared in simulations. The Siegmund’s algorithm has also proven good in practice, whereas in heavy tailed case the conditional Monte Carlo algorithms have shown better performance than the importance sampling ones