Experiments on predictability of word in context and information rate in natural language

Based on data from a large-scale experiment with human subjects, we conclude that the logarithm of probability to guess a word in context (unpredictability) depends linearly on the word length. This result holds both for poetry and prose, even though with prose, the subjects don't know the length of the omitted word. We hypothesize that this effect reflects a tendency of natural language to have an even information rate

Method and apparatus for decoding compatible convolutional codes

This invention relates to learning decoders for decoding compatible convolutional codes. The decoder decodes signals which have been encoded by a convolutional coder and allows performance near the theoretical limit of performance for coded data systems. The decoder includes a sub-bit shift register wherein the received sub-bits are entered after regeneration and shifted in synchronization with a clock signal recovered from the received sub-bit stream. The received sub-bits are processed by a sub-bit decision circuit, entered into a sub-bit shift register, decoded by a decision circuit, entered into a data shift register, and updated to reduce data errors. The bit decision circuit utilizes stored sub-bits and stored data bits to determine subsequent data-bits. Data errors are reduced by using at least one up-date circuit

Information Theory and the Earth's Density Distribution

An argument for using the information theory approach as an inference technique in solid earth geophysics. A spherically symmetric density distribution is derived as an example of the method. A simple model of the earth plus knowledge of its mass and moment of inertia lead to a density distribution which was surprisingly close to the optimum distribution. Future directions for the information theory approach in solid earth geophysics as well as its strengths and weaknesses are discussed