Why some messages are more interesting than others

“The sun rised today” = not news
“The sun didn’t rise today” – interesting!


Claude Shannon in his article about Communication Theory (1948) wrote that information is related to surprise.
Specifically, that a message informing us of an event that has probability p to happen conveys

‐log(base 2) p bits of information.

For example, if an event has 50% probability to happen, then it has ‐log 0.5 = 1 bit of information associated.

If the event has only 10% probability to happen, then it has -log 0.1 = 3.3 bits of information.

Shannon’s diagram of a communication system – from Wikipedia

When we pick up a newspaper, we are looking for maximum information, so more `surprising’ events make for better news.

See also what the physicist John Wheeler wrote in 1990:

“… every it — every particle, every field of force — derives its function, its meaning, its very existence entirely from yes-or-no questions, binary choices, _bits_.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s