Probabilistic Monads, Domains and Classical Information

Shannon's classical information theory uses probability theory to analyze channels as mechanisms for information flow. In this paper, we generalize results of Martin, Allwein and Moskowitz for binary channels to show how some more modern tools - probabilistic monads and domain theory in particular - can be used to model classical channels. As initiated Martin, et al., the point of departure is to consider the family of channels with fixed inputs and outputs, rather than trying to analyze channels one at a time. The results show that domain theory has a role to play in the capacity of channels; in particular, the (n x n)-stochastic matrices, which are the classical channels having the same sized input as output, admit a quotient compact ordered space which is a domain, and the capacity map factors through this quotient via a Scott-continuous map that measures the quotient domain. We also comment on how some of our results relate to recent discoveries about quantum channels and free affine monoids.Comment: In Proceedings DCM 2011, arXiv:1207.682

SCS 14: SCS Memo of Lawson Dated 7-12-76

Also accessible at https://www2.mathematik.tu-darmstadt.de/~logik/keimel/scs.htm

Domains and Probability Measures: A Topological Retrospective

Domain theory has seen success as a semantic model for high-level programming languages, having devised a range of constructs to support various effects that arise in programming. One of the most interesting - and problematic - is probabilistic choice, which traditionally has been modeled using a domain-theoretic rendering of sub-probability measures as valuations. In this talk, I will place the domain-theoretic approach in context, by showing how it relates to the more traditional approaches such as functional analysis and set theory. In particular, we show how the topologies that arise in the classic approaches relate to the domain-theoretic rendering. We also describe some recent developments that extend the domain approach to stochastic process theory

SCS 27: Closure Operators and Kernel Operators in CL

Also accessible at https://www2.mathematik.tu-darmstadt.de/~logik/keimel/scs.htm

Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm

NLP tasks are often limited by scarcity of manually annotated data. In social media sentiment analysis and related tasks, researchers have therefore used binarized emoticons and specific hashtags as forms of distant supervision. Our paper shows that by extending the distant supervision to a more diverse set of noisy labels, the models can learn richer representations. Through emoji prediction on a dataset of 1246 million tweets containing one of 64 common emojis we obtain state-of-the-art performance on 8 benchmark datasets within sentiment, emotion and sarcasm detection using a single pretrained model. Our analyses confirm that the diversity of our emotional labels yield a performance improvement over previous distant supervision approaches.Comment: Accepted at EMNLP 2017. Please include EMNLP in any citations. Minor changes from the EMNLP camera-ready version. 9 pages + references and supplementary materia