Skip to main content
Article thumbnail
Location of Repository

Most Informative Dimension Reduction

By Amir Globerson and Naftali Tishby


Finding effective low dimensional features from empirical co-occurrence data is one of the most fundamental problems in machine learning and complex data analysis. One principled approach to this problem is to represent the data in low dimension with minimal loss of the information contained in the original data. In this paper we present a novel information theoretic principle and algorithm for extracting low dimensional representations, or feature-vectors, that capture as much as possible of the mutual information between the variables. Unlike previous work in this direction, here we do not cluster or quantize the variables, but rather extract continuous feature functions directly from the co-occurrence matrix, using a converging iterative projection algorithm. The obtained features serve, in a well defined way, as approximate sucient statistics that capture the information in a joint sample of the variables. Our approach is both simpler and more general than clustering or mixture models and is applicable to a wide range of problems, from document categorization to bioinformatics and analysis of neural codes

Year: 2002
OAI identifier: oai:CiteSeerX.psu:
Provided by: CiteSeerX
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • (external link)
  • (external link)
  • Suggested articles

    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.