Counting Distinctions: On the Conceptual Foundations of Shannon's Information Theory
University of California at Riverside
March 26, 2009
Synthese, Vol. 168, pp. 119-149, May 2009
Ordinary "propositional" logic can be interpreted as the logic of subsets. The concept of a partition on a universe set U is dual to the concept of a subset of the universe set in the category-theoretic sense of duality between epimorphisms and monomorpisms. The dual to the notion of an element being in a subset is that of a distinction being made by a partition (i.e., a pair of elements being in distinct blocks of the partition). Probability theory started by moving beyond the logic of subsets on a universe to assign a "probability" to each subset of a finite universe which was the number of elements in the subset normalized by the size of the universe set (using the Laplacian assumption of each element having equal probability). This paper works out the corresponding conceptual development starting with the logic of partitions. Each partition on U is assigned a "logical entropy" which is the number of distinctions made by the partition normalized by the number of ordered pairs in UxU. This notion of logical entropy is then precisely related to Shannon's notion of entropy showing that information theory is conceptually based on the logical notion of distinctions.
Number of Pages in PDF File: 31
Keywords: information theory, Shannon's entropy, logical entropy, logic of partitions
JEL Classification: D80Accepted Paper Series
Date posted: March 27, 2009 ; Last revised: April 21, 2009
© 2013 Social Science Electronic Publishing, Inc. All Rights Reserved.
This page was processed by apollo6 in 0.359 seconds