“The sun rised today” = not news
“The sun didn’t rise today” – interesting!
Claude Shannon in his article about Communication Theory (1948) wrote that information is related to surprise.
Specifically, that a message informing us of an event that has probability p to happen conveys
‐log(base 2) p bits of information.
For example, if an event has 50% probability to happen, then it has ‐log 0.5 = 1 bit of information associated.
If the event has only 10% probability to happen, then it has -log 0.1 = 3.3 bits of information.
When we pick up a newspaper, we are looking for maximum information, so more `surprising’ events make for better news.
“… every it — every particle, every field of force — derives its function, its meaning, its very existence entirely from yes-or-no questions, binary choices, _bits_.”