# Math Help - Calculating Entropy using Shannons Formula

1. ## Calculating Entropy using Shannons Formula

Hey guys,
I am real need of some help here. Basically I have a test tomorrow and the following question was on last years test and its completely over my head as I haven't done Maths in years.
I have to use the following formula:

To calculate the following:
An image, 500 pixels by 400 pixels, consists of 80000 Blue, 6000 Orange, 8000 Red, 2000 Yellow, 50000 Indigo, 40000 Green, 4000 Violet pixels, 10000 Turquoise pixels.
(iv) Calculate Entropy in bits based on Shannon’s formula. (Show how you derived the result)
--
Any help will be greatly appreciated as I'm basically fooked for this test.

2. Originally Posted by Dougster
Hey guys,
I am real need of some help here. Basically I have a test tomorrow and the following question was on last years test and its completely over my head as I haven't done Maths in years.
I have to use the following formula:

To calculate the following:
An image, 500 pixels by 400 pixels, consists of 80000 Blue, 6000 Orange, 8000 Red, 2000 Yellow, 50000 Indigo, 40000 Green, 4000 Violet pixels, 10000 Turquoise pixels.
(iv) Calculate Entropy in bits based on Shannon’s formula. (Show how you derived the result)
For each colour, $P_k$ for that colour is the probability that a pixel will be that colour. So for example $P_{\text{blue}}=\frac{80}{200} = 0.4$, because there are 80k blue pixels and 200k pixels altogether.

You then have to find $\log_2(P_{\text{blue}})$. But $\log_2x = \frac{\log x}{\log2}$. Therefore $-P_{\text{blue}}\log_2(P_{\text{blue}}) = -0.4\frac{\log0.4}{\log2} \approx -0.4\frac{-0.3979}{0.3010} = 0.5288$ (to 4 decimal places).

Having found $-P_{\text{blue}}\log_2(P_{\text{blue}})$, you then have to do the same for each of the other seven colours, and add the results to get the entropy.

3. Originally Posted by Opalg
For each colour, $P_k$ for that colour is the probability that a pixel will be that colour. So for example $P_{\text{blue}}=\frac{80}{200} = 0.4$, because there are 80k blue pixels and 200k pixels altogether.

You then have to find $\log_2(P_{\text{blue}})$. But $\log_2x = \frac{\log x}{\log2}$. Therefore $-P_{\text{blue}}\log_2(P_{\text{blue}}) = -0.4\frac{\log0.4}{\log2} \approx -0.4\frac{-0.3979}{0.3010} = 0.5288$ (to 4 decimal places).

Having found $-P_{\text{blue}}\log_2(P_{\text{blue}})$, you then have to do the same for each of the other seven colours, and add the results to get the entropy.
Thanks very much for your reply.

I've got the hang of calculating the probabilities but it is when log comes into it I get confused, basically because I don't even know what log is

Would you be able to simplify the middle of your reply into what buttons to press on the calculator. Once I can see how one is done (blue), then I'd have no problems with the rest.

Thanks again.

4. Originally Posted by Dougster
Would you be able to simplify the middle of your reply into what buttons to press on the calculator.
I guess the answer may depend on what sort of calculator you have. I use an ancient Casio. If I want to find $\frac{\log0.4}{\log2}$ then I press these buttons:

$\boxed{\mathstrut 0.4}\quad \boxed{\mathstrut \log}\quad \boxed{\mathstrut \div}\quad \boxed{\mathstrut 2}\quad \boxed{\mathstrut \log}\quad \boxed{\mathstrut =}$.

In that sequence, "log" means log to the base 10. You can replace both "log"s by "ln" (natural log, or log to base e), and the result will be the same.

Caution. Your results may vary. Different calculators use different sequences of operations.