Counting Distinctions: On the Conceptual Foundations of Shannon's Information Theory

Synthese, Vol. 168, pp. 119-149, May 2009

31 Pages Posted: 27 Mar 2009 Last revised: 21 Apr 2009

Date Written: March 26, 2009

Abstract

Ordinary "propositional" logic can be interpreted as the logic of subsets. The concept of a partition on a universe set U is dual to the concept of a subset of the universe set in the category-theoretic sense of duality between epimorphisms and monomorpisms. The dual to the notion of an element being in a subset is that of a distinction being made by a partition (i.e., a pair of elements being in distinct blocks of the partition). Probability theory started by moving beyond the logic of subsets on a universe to assign a "probability" to each subset of a finite universe which was the number of elements in the subset normalized by the size of the universe set (using the Laplacian assumption of each element having equal probability). This paper works out the corresponding conceptual development starting with the logic of partitions. Each partition on U is assigned a "logical entropy" which is the number of distinctions made by the partition normalized by the number of ordered pairs in UxU. This notion of logical entropy is then precisely related to Shannon's notion of entropy showing that information theory is conceptually based on the logical notion of distinctions.

Keywords: information theory, Shannon's entropy, logical entropy, logic of partitions

JEL Classification: D80

Suggested Citation

Ellerman, David, Counting Distinctions: On the Conceptual Foundations of Shannon's Information Theory (March 26, 2009). Synthese, Vol. 168, pp. 119-149, May 2009, Available at SSRN: https://ssrn.com/abstract=1368904

David Ellerman (Contact Author)

University of Ljubljana ( email )

School of Social Science
Ljubljana, CA
Slovenia

HOME PAGE: http://www.ellerman.org

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
120
Abstract Views
813
rank
253,866
PlumX Metrics