[T]his is very different from the Newtonian picture where Newton presupposes there's a river of time flowing, that is there before anything is put in… - Julian Barbour

The concepts of message and probability enable one, for a definite source of <math>N</math> messages, to define Shannon’s information. If <math>p_i,\quad i = 1, 2, ..., N</math>, is the relative probability of message <math>i</math> and <math>\log p_i</math> is its base-2 logarithm, then the information <math>I</math> of the given source is(1) <math>\quad I = - \sum_{k=1}^N p_i \log p_i</math>.The minus sign makes <math>I</math> positive because all probabilities, which are necessarily greater than or equal zero, are less than unity (their sum being<math>\textstyle \sum_{i}^N p_i = 1</math>, so that their logarithms are all negative.

[T]hings, not information, are primary. ... 'bit' derives from 'it'.

[W]hat we call yesterday is self-contained and has its experience of being yesterday, and today has memories of yesterday; and therefor I say that it's later... but each is completely self-contained, and there's no reason why you should put one... here, and another one there...