Fixed-to-Variable Length Distribution Matching

Fixed-to-variable length (f2v) matchers are used to reversibly transform an input sequence of independent and uniformly distributed bits into an output sequence of bits that are (approximately) independent and distributed according to a target distribution. The degree of approximation is measured by the informational divergence between the output distribution and the target distribution. An algorithm is developed that efficiently finds optimal f2v codes. It is shown that by encoding the input bits blockwise, the informational divergence per bit approaches zero as the block length approaches infinity. A relation to data compression by Tunstall coding is established.Comment: 5 pages, essentially the ISIT 2013 versio

Molar and local effects of the fixed-ratio changeover requirement on choice, changeovers, and visits: A parametric examination of the fixed-ratio changeover requirement

The distribution of behavior by organisms in choice situations is of long-standing interest to psychologists. The generalized matching relation accurately predicts choice between concurrent variable-interval schedules of reinforcement. Researchers have assumed, on weak grounds, that the effect of the changeover requirement on sensitivity to reinforcement--the exponent in the generalized matching equation--was consistent. This experiment considered the effects of the changeover requirement by parametrically manipulating the fixed-ratio schedule required to switch alternatives. Pigeons pecked either of two side-response keys in a standard three-key operant chamber for food, delivered according to independent variable-interval schedules. No changeover delay was used, instead completion of five fixed-ratio schedules (FR 0, 2, 6, 12, or 20) on the center-response key alternated the active side key. Five reinforcer ratio (1:1, 1:2, 2:1, 1:4, and 4:1) were paired with most FR schedules. A matching relation analysis indicated that for two pigeons response-allocation sensitivity generally overmatched for all but the FR 0 condition, which undermatched. The other two pigeons\u27 sensitivity increased to overmatching when FR 12 was in force. Excepting FR 0 conditions, time-allocation sensitivity, on the other hand, decreased from extreme overmatching toward matching as the changeover requirement increased. Reliable changes in response rates to the two alternatives account for the results. A positive relation between the conditional probability of switching and run length is reported. That is, the greater the number of consecutive pecks to an alternative, the greater the likelihood of switching. This result suggests that behavior is controlled in part by local reinforcement contingencies. I speculate that factors that increase visit duration may increase local control of switching. The procedure encourages a foraging interpretation. The FR changeover requirement can be considered functionally equivalent to travel between patches. An analysis of visit measures supported earlier evidence that residence measures increase as travel between patches increases. These results together with the matching results suggest that behavior ecologists and operant psychologists are working on similar problems and the traditional tools of operant psychology can be used to simulate travel, an important component of foraging in the wild

Posterior Matching Scheme for Gaussian Multiple Access Channel with Feedback

Posterior matching is a method proposed by Ofer Shayevitz and Meir Feder to design capacity achieving coding schemes for general point-to-point memoryless channels with feedback. In this paper, we present a way to extend posterior matching based encoding and variable rate decoding ideas for the Gaussian MAC with feedback, referred to as time-varying posterior matching scheme, analyze the achievable rate region and error probabilities of the extended encoding-decoding scheme. The time-varying posterior matching scheme is a generalization of the Shayevitz and Feder's posterior matching scheme when the posterior distributions of the input messages given output are not fixed over transmission time slots. It turns out that the well-known Ozarow's encoding scheme, which obtains the capacity of two-user Gaussian channel, is a special case of our extended posterior matching framework as the Schalkwijk-Kailath's scheme is a special case of the point-to-point posterior matching mentioned above. Furthermore, our designed posterior matching also obtains the linear-feedback sum-capacity for the symmetric multiuser Gaussian MAC. Besides, the encoding scheme in this paper is designed for the real Gaussian MAC to obtain that performance, which is different from previous approaches where encoding schemes are designed for the complex Gaussian MAC. More importantly, this paper shows potential of posterior matching in designing optimal coding schemes for multiuser channels with feedback.Comment: submitted to the IEEE Transactions on Information Theory. A shorter version has been accepted to IEEE Information Theory Workshop 201

A survey of max-type recursive distributional equations

In certain problems in a variety of applied probability settings (from probabilistic analysis of algorithms to statistical physics), the central requirement is to solve a recursive distributional equation of the form X =^d g((\xi_i,X_i),i\geq 1). Here (\xi_i) and g(\cdot) are given and the X_i are independent copies of the unknown distribution X. We survey this area, emphasizing examples where the function g(\cdot) is essentially a ``maximum'' or ``minimum'' function. We draw attention to the theoretical question of endogeny: in the associated recursive tree process X_i, are the X_i measurable functions of the innovations process (\xi_i)?Comment: Published at http://dx.doi.org/10.1214/105051605000000142 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Optimal Feedback Communication via Posterior Matching

In this paper we introduce a fundamental principle for optimal communication over general memoryless channels in the presence of noiseless feedback, termed posterior matching. Using this principle, we devise a (simple, sequential) generic feedback transmission scheme suitable for a large class of memoryless channels and input distributions, achieving any rate below the corresponding mutual information. This provides a unified framework for optimal feedback communication in which the Horstein scheme (BSC) and the Schalkwijk-Kailath scheme (AWGN channel) are special cases. Thus, as a corollary, we prove that the Horstein scheme indeed attains the BSC capacity, settling a longstanding conjecture. We further provide closed form expressions for the error probability of the scheme over a range of rates, and derive the achievable rates in a mismatch setting where the scheme is designed according to the wrong channel model. Several illustrative examples of the posterior matching scheme for specific channels are given, and the corresponding error probability expressions are evaluated. The proof techniques employed utilize novel relations between information rates and contraction properties of iterated function systems.Comment: IEEE Transactions on Information Theor