Generalised Pinsker Inequalities

We generalise the classical Pinsker inequality which relates variational divergence to Kullback-Liebler divergence in two ways: we consider arbitrary f-divergences in place of KL divergence, and we assume knowledge of a sequence of values of generalised variational divergences. We then develop a best possible inequality for this doubly generalised situation. Specialising our result to the classical case provides a new and tight explicit bound relating KL to variational divergence (solving a problem posed by Vajda some 40 years ago). The solution relies on exploiting a connection between divergences and the Bayes risk of a learning problem via an integral representation.Comment: 21 pages, 3 figures, accepted to COLT 200

Partial tests, universal tests and decomposability

For a property P and a sub-property P', we say that P is P'-partially testable with q queries} if there exists an algorithm that distinguishes, with high probability, inputs in P' from inputs ε-far from P, using q queries. Some natural properties require many queries to test, but can be partitioned into a small number of subsets for which they are partially testable with very few queries, sometimes even a number independent of the input size. For properties over {0,1}, the notion of being thus partitionable ties in closely with Merlin-Arthur proofs of Proximity (MAPs) as defined independently in [14] a partition into r partially-testable properties is the same as a Merlin-Arthur system where the proof consists of the identity of one of the r partially-testable properties, giving a 2-way translation to an O(log r) size proof. Our main result is that for some low complexity properties a partition as above cannot exist, and moreover that for each of our properties there does not exist even a single sub-property featuring both a large size and a query-efficient partial test, in particular improving the lower bound set in [14]. For this we use neither the traditional Yao-type arguments nor the more recent communication complexity method, but open up a new approach for proving lower bounds. First, we use entropy analysis, which allows us to apply our arguments directly to 2-sided tests, thus avoiding the cost of the conversion in [14] from 2-sided to 1-sided tests. Broadly speaking we use "distinguishing instances" of a supposed test to show that a uniformly random choice of a member of the sub-property has "low entropy areas", ultimately leading to it having a low total entropy and hence having a small base set. Additionally, to have our arguments apply to adaptive tests, we use a mechanism of "rearranging" the input bits (through a decision tree that adaptively reads the entire input) to expose the low entropy that would otherwise not be apparent. We also explore the possibility of a connection in the other direction, namely whether the existence of a good partition (or MAP) can lead to a relatively query-efficient standard property test. We provide some preliminary results concerning this question, including a simple lower bound on the possible trade-off. Our second major result is a positive trade-off result for the restricted framework of 1-sided proximity oblivious tests. This is achieved through the construction of a "universal tester" that works the same for all properties admitting the restricted test. Our tester is very related to the notion of sample-based testing (for a non-constant number of queries) as defined by Goldreich and Ron in [13]. In particular it partially resolves an open problem raised by [13]

Parameter estimation for stochastic models of biochemical reactions

Parameter estimation is central for the analysis of models in Systems Biology. Stochastic models are of increasing importance. However, parameter estimation for stochastic models is still in the early phase of development and there is need for efficient methods to estimate model parameters from time course data which is intrinsically stochastic, only partially observed and has measurement noise. The thesis investigates methods for parameter estimation for stochastic models presenting one efficient method based on integration of ordinary differential equations (ODE) which allows parameter estimation even for models which have qualitatively different behavior in stochastic modeling compared to modeling with ODEs. Further methods proposed in the thesis are based on stochastic simulations. One of the methods uses the stochastic simulations for an estimation of the transition probabilities in the likelihood function. This method is suggested as an addition to the ODE-based method and should be used in systems with few reactions and small state spaces. The resulting stochastic optimization problem can be solved with a Particle Swarm algorithm. To this goal a stopping criterion suited to the stochasticity is proposed. Another approach is a transformation to a deterministic optimization problem. Therefore the polynomial chaos expansion is extended to stochastic functions in this thesis and then used for the transformation. The ODE-based method is motivated from a fast and efficient method for parameter estimation for systems of ODEs. A multiple shooting procedure is used in which the continuity constraints are omitted to allow for stochasticity. Unobserved states are treated by enlarging the optimization vector or using resulting values from the forward integration. To see how well the method covers the stochastic dynamics some test functions will be suggested. It is demonstrated that the method works well even in systems which have qualitatively different behavior in stochastic modeling than in modeling with ODEs. From a computational point of view, this method allows to tackle systems as large as those tackled in deterministic modeling