Self-Organization with Information Theoretic Learning

Unsupervised learning algorithms seek to discover the structure in the data using solely the samples. The most important classes of unsupervised algorithms are clustering, principal curves and vector quantization, and each has been derived independently of the others. This talk presents a new selforganizing principle called the Principle of Relevant Information (PRI) that was motivated by fundamental ideas of information theory. We will also show how each of these classes of unsupervised algorithms are special cases of the PRI, weighting differently the minimization of entropy and the distance to the original data set. Examples will also be provided.