Originally Posted by

**mrsi** Here's the question

---------------------------------

Let I be the unit interval, i.e. $\displaystyle I = \{x\ |\ 0 \leqslant x \leqslant 1\}$, and let $\displaystyle I^ N$ be the N-fold cross product of I with itself, i.e., the unit N-box. Let P be that subset of $\displaystyle I^ N$ consisting of points with positive coordinates summing to 1, i.e.

$\displaystyle P=\{(p_{1},p_{2},...,p_{N})\ |\ \sum\limits_{i=1}^Np_i=1,\ p_{i}>0\}$

Let R be the real numbers and define the information function $\displaystyle H : P \to R$ by

$\displaystyle H(p_{1},p_{2},...,p_{N})=-\sum\limits_{i=1}^{N} p_{i}\log{p_{i}}$

[I have modified that to correct several typos.]

H gives the amount of information in a communication system with N alternative messages where the ith message is transmitted with probability pi. (Interesting fact: If the base of the logarithm is chosen to be 2, then the unit of information is the bit, and corresponds to the amount of information in one yes-no question.) Prove that if all the messages are transmitted with equal probability, then the amount of information is equal to log N.

--------------

Here's what I did

Let $\displaystyle p_{i}=\frac{1}{N}$

$\displaystyle -\sum\limits_{i=1}^{N} p_{i}\log{p_{i}}=[(-\frac{1}{2}\log\frac{1}{2})+(-\frac{1}{3}\log\frac{1}{3})+...+(-\frac{1}{N}\log\frac{1}{N})]$