Counting Distinctions: On the Conceptual Foundations of Shannon's Information Theory

Synthese, Vol. 168, pp. 119-149, May 2009

31 Pages Posted: 27 Mar 2009 Last revised: 21 Apr 2009

Date Written: March 26, 2009


Ordinary "propositional" logic can be interpreted as the logic of subsets. The concept of a partition on a universe set U is dual to the concept of a subset of the universe set in the category-theoretic sense of duality between epimorphisms and monomorpisms. The dual to the notion of an element being in a subset is that of a distinction being made by a partition (i.e., a pair of elements being in distinct blocks of the partition). Probability theory started by moving beyond the logic of subsets on a universe to assign a "probability" to each subset of a finite universe which was the number of elements in the subset normalized by the size of the universe set (using the Laplacian assumption of each element having equal probability). This paper works out the corresponding conceptual development starting with the logic of partitions. Each partition on U is assigned a "logical entropy" which is the number of distinctions made by the partition normalized by the number of ordered pairs in UxU. This notion of logical entropy is then precisely related to Shannon's notion of entropy showing that information theory is conceptually based on the logical notion of distinctions.

Keywords: information theory, Shannon's entropy, logical entropy, logic of partitions

JEL Classification: D80

Suggested Citation

Ellerman, David, Counting Distinctions: On the Conceptual Foundations of Shannon's Information Theory (March 26, 2009). Synthese, Vol. 168, pp. 119-149, May 2009, Available at SSRN:

David Ellerman (Contact Author)

University of Ljubljana ( email )

School of Social Science
Ljubljana, CA


Here is the Coronavirus
related research on SSRN

Paper statistics

Abstract Views
PlumX Metrics