[jay santos]

Originally Posted by Sleeping in EQ

This gem is from the reknowned physicist and mathematician, Norbert Wiener:

Messages themselves are a form of pattern and organization. Indeed, it is possible to treat sets of messages as having an entropy like sets of states of the external world. Just as entropy is a measure of disorganization, the information carried by a set of messages is a measure of organization. In fact, it is possible to interpret the information carried by a message as essentially the negative of its entropy, and the negative logarithm of its probability. That is, the more probable the message, the less information it gives. Cliches, for example, are less illuminating than great poems. (1954, 21).

Good quote, but I disagree with the inference. The probability of the message is not in how many times it is repeated but how many times it is created independently.