The concepts of message and probability enable one, for a definite source of <math>N</math> messages, to define Shannon’s information. If <math>p_i,\… - Julian Barbour

" "

The concepts of message and probability enable one, for a definite source of <math>N</math> messages, to define Shannon’s information. If <math>p_i,\quad i = 1, 2, ..., N</math>, is the relative probability of message <math>i</math> and <math>\log p_i</math> is its base-2 logarithm, then the information <math>I</math> of the given source is(1) <math>\quad I = - \sum_{k=1}^N p_i \log p_i</math>.The minus sign makes <math>I</math> positive because all probabilities, which are necessarily greater than or equal zero, are less than unity (their sum being<math>\textstyle \sum_{i}^N p_i = 1</math>, so that their logarithms are all negative.