On the Bayes-optimality of F-measure maximizers

The F-measure, which has originally been introduced in information retrieval, is nowadays routinely used as a performance metric for problems such as binary classification, multi-label classification, and structured output prediction. Optimizing this measure is a statistically and computationally challenging problem, since no closed-form solution exists. Adopting a decision-theoretic perspective, this article provides a formal and experimental analysis of different approaches for maximizing the F-measure. We start with a Bayes-risk analysis of related loss functions, such as Hamming loss and subset zero-one loss, showing that optimizing such losses as a surrogate of the F-measure leads to a high worst-case regret. Subsequently, we perform a similar type of analysis for F-measure maximizing algorithms, showing that such algorithms are approximate, while relying on additional assumptions regarding the statistical distribution of the binary response variables. Furthermore, we present a new algorithm which is not only computationally efficient but also Bayes-optimal, regardless of the underlying distribution. To this end, the algorithm requires only a quadratic (with respect to the number of binary responses) number of parameters of the joint distribution. We illustrate the practical performance of all analyzed methods by means of experiments with multi-label classification problems

T-Duality, and the K-Theoretic Partition Function of TypeIIA Superstring Theory

We study the partition function of type IIA string theory on 10-manifolds of the form T^2 x X where X is 8-dimensional, compact, and spin. We pay particular attention to the effects of the topological phases in the supergravity action implied by the K-theoretic formulation of RR fields, and we use these to check the T-duality invariance of the partition function. We find that the partition function is only T-duality invariant when we take into account the T-duality anomalies in the RR sector, the fermionic path integral (including 4-fermi interaction terms), and 1-loop corrections including worldsheet instantons. We comment on applications of our computation to speculations about the role of the Romans mass in M-theory. We also discuss some issues which arise when one attempts to extend these considerations to checking the full U-duality invariance of the theory.Comment: 73 pages, harvmac, b-mod

Heights of one- and two-sided congruence lattices of semigroups

The height of a poset $P$ is the supremum of the cardinalities of chains in $P$. The exact formula for the height of the subgroup lattice of the symmetric group $\mathcal{S}_n$ is known, as is an accurate asymptotic formula for the height of the subsemigroup lattice of the full transformation monoid $\mathcal{T}_n$. Motivated by the related question of determining the heights of the lattices of left- and right congruences of $\mathcal{T}_n$, we develop a general method for computing the heights of lattices of both one- and two-sided congruences for semigroups. We apply this theory to obtain exact height formulae for several monoids of transformations, matrices and partitions, including: the full transformation monoid $\mathcal{T}_n$, the partial transformation monoid $\mathcal{PT}_n$, the symmetric inverse monoid $\mathcal{I}_n$, the monoid of order-preserving transformations $\mathcal{O}_n$, the full matrix monoid $\mathcal{M}(n,q)$, the partition monoid $\mathcal{P}_n$, the Brauer monoid $\mathcal{B}_n$ and the Temperley-Lieb monoid $\mathcal{TL}_n$

A methodology for producing reliable software, volume 1

An investigation into the areas having an impact on producing reliable software including automated verification tools, software modeling, testing techniques, structured programming, and management techniques is presented. This final report contains the results of this investigation, analysis of each technique, and the definition of a methodology for producing reliable software

On the Bayes-optimality of F-measure maximizers

The F-measure, which has originally been introduced in information retrieval, is nowadays routinely used as a performance metric for problems such as binary classification, multi-label classification, and structured output prediction. Optimizing this measure is a statistically and computationally challenging problem, since no closed-form solution exists. Adopting a decision-theoretic perspective, this article provides a formal and experimental analysis of different approaches for maximizing the F-measure. We start with a Bayes-risk analysis of related loss functions, such as Hamming loss and subset zero-one loss, showing that optimizing such losses as a surrogate of the F-measure leads to a high worst-case regret. Subsequently, we perform a similar type of analysis for F-measure maximizing algorithms, showing that such algorithms are approximate, while relying on additional assumptions regarding the statistical distribution of the binary response variables. Furthermore, we present a new algorithm which is not only computationally efficient but also Bayes-optimal, regardless of the underlying distribution. To this end, the algorithm requires only a quadratic (with respect to the number of binary responses) number of parameters of the joint distribution. We illustrate the practical performance of all analyzed methods by means of experiments with multi-label classification problems