There are five main contributions of this dissertation. The first contribution is new closed-form expressions for channel capacity of a new class of channel matrices. The second contribution is the discovery of the structure for optimal binary quantizer and the associated methods for finding an optimal quantizer that maximizes mutual information between the input and output for a given input distribution. The third contribution is the discovery of the structure for an optimal K-ary quantizer that maximizes the mutual information subject to an arbitrary constraint on the output distribution. The fourth contribution is the joint design of an optimal quantizer that maximizes the mutual information over both the input distribution and the quantization parameters for an arbitrary binary noisy channel with a given noise density. The last contribution is the development and analysis of novel efficient classification algorithms for finding the minimum impurity partition using mutual information as the metric
