Solving Multiclass Learning Problems via Error-Correcting Output Codes

Multiclass learning problems involve finding a definition for an unknown function f(x) whose range is a discrete set containing k &gt 2 values (i.e., k ``classes''). The definition is acquired by studying collections of training examples of the form [x_i, f (x_i)]. Existing approaches to multiclass learning problems include direct application of multiclass algorithms such as the decision-tree algorithms C4.5 and CART, application of binary concept learning algorithms to learn individual binary functions for each of the k classes, and application of binary concept learning algorithms with distributed output representations. This paper compares these three approaches to a new technique in which error-correcting codes are employed as a distributed output representation. We show that these output representations improve the generalization performance of both C4.5 and backpropagation on a wide range of multiclass learning tasks. We also demonstrate that this approach is robust with respect to changes in the size of the training sample, the assignment of distributed representations to particular classes, and the application of overfitting avoidance techniques such as decision-tree pruning. Finally, we show that---like the other methods---the error-correcting code technique can provide reliable class probability estimates. Taken together, these results demonstrate that error-correcting output codes provide a general-purpose method for improving the performance of inductive learning programs on multiclass problems.Comment: See http://www.jair.org/ for any accompanying file

Concatenated Codes for Amplitude Damping

We discuss a method to construct quantum codes correcting amplitude damping errors via code concatenation. The inner codes are chosen as asymmetric Calderbank-Shor-Steane (CSS) codes. By concatenating with outer codes correcting symmetric errors, many new codes with good parameters are found, which are better than the amplitude damping codes obtained by any previously known construction.Comment: 5 page

Coding Theory and Algebraic Combinatorics

This chapter introduces and elaborates on the fruitful interplay of coding theory and algebraic combinatorics, with most of the focus on the interaction of codes with combinatorial designs, finite geometries, simple groups, sphere packings, kissing numbers, lattices, and association schemes. In particular, special interest is devoted to the relationship between codes and combinatorial designs. We describe and recapitulate important results in the development of the state of the art. In addition, we give illustrative examples and constructions, and highlight recent advances. Finally, we provide a collection of significant open problems and challenges concerning future research.Comment: 33 pages; handbook chapter, to appear in: "Selected Topics in Information and Coding Theory", ed. by I. Woungang et al., World Scientific, Singapore, 201

Improving the Sphere-Packing Bound for Binary Codes over Memoryless Symmetric Channels

A lower bound on the minimum required code length of binary codes is obtained. The bound is obtained based on observing a close relation between the Ulam's liar game and channel coding. In fact, Spencer's optimal solution to the game is used to derive this new bound which improves the famous Sphere-Packing Bound.Comment: 5 pages,3 figures, Presented at the Forty-Seventh Annual Allerton Conference on Communication, Control, and Computing, Sep. 200

On Optimal Binary One-Error-Correcting Codes of Lengths 2m−42^m-4 and 2m−32^m-3

Best and Brouwer [Discrete Math. 17 (1977), 235-245] proved that triply-shortened and doubly-shortened binary Hamming codes (which have length $2^m-4$ and $2^m-3$, respectively) are optimal. Properties of such codes are here studied, determining among other things parameters of certain subcodes. A utilization of these properties makes a computer-aided classification of the optimal binary one-error-correcting codes of lengths 12 and 13 possible; there are 237610 and 117823 such codes, respectively (with 27375 and 17513 inequivalent extensions). This completes the classification of optimal binary one-error-correcting codes for all lengths up to 15. Some properties of the classified codes are further investigated. Finally, it is proved that for any $m \geq 4$, there are optimal binary one-error-correcting codes of length $2^m-4$ and $2^m-3$ that cannot be lengthened to perfect codes of length $2^m-1$.Comment: Accepted for publication in IEEE Transactions on Information Theory. Data available at http://www.iki.fi/opottone/code