Results 1 to 4 of 4

Math Help - Calculating Entropy using Shannons Formula

  1. #1
    Newbie
    Joined
    Jan 2009
    Posts
    2

    Calculating Entropy using Shannons Formula

    Hey guys,
    I am real need of some help here. Basically I have a test tomorrow and the following question was on last years test and its completely over my head as I haven't done Maths in years.
    I have to use the following formula:

    To calculate the following:
    An image, 500 pixels by 400 pixels, consists of 80000 Blue, 6000 Orange, 8000 Red, 2000 Yellow, 50000 Indigo, 40000 Green, 4000 Violet pixels, 10000 Turquoise pixels.
    (iv) Calculate Entropy in bits based on Shannon’s formula. (Show how you derived the result)
    --
    Any help will be greatly appreciated as I'm basically fooked for this test.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    MHF Contributor
    Opalg's Avatar
    Joined
    Aug 2007
    From
    Leeds, UK
    Posts
    4,041
    Thanks
    7
    Quote Originally Posted by Dougster View Post
    Hey guys,
    I am real need of some help here. Basically I have a test tomorrow and the following question was on last years test and its completely over my head as I haven't done Maths in years.
    I have to use the following formula:

    To calculate the following:
    An image, 500 pixels by 400 pixels, consists of 80000 Blue, 6000 Orange, 8000 Red, 2000 Yellow, 50000 Indigo, 40000 Green, 4000 Violet pixels, 10000 Turquoise pixels.
    (iv) Calculate Entropy in bits based on Shannon’s formula. (Show how you derived the result)
    For each colour, P_k for that colour is the probability that a pixel will be that colour. So for example P_{\text{blue}}=\frac{80}{200} = 0.4, because there are 80k blue pixels and 200k pixels altogether.

    You then have to find \log_2(P_{\text{blue}}). But \log_2x = \frac{\log x}{\log2}. Therefore -P_{\text{blue}}\log_2(P_{\text{blue}}) = -0.4\frac{\log0.4}{\log2} \approx -0.4\frac{-0.3979}{0.3010} = 0.5288 (to 4 decimal places).

    Having found -P_{\text{blue}}\log_2(P_{\text{blue}}), you then have to do the same for each of the other seven colours, and add the results to get the entropy.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Jan 2009
    Posts
    2
    Quote Originally Posted by Opalg View Post
    For each colour, P_k for that colour is the probability that a pixel will be that colour. So for example P_{\text{blue}}=\frac{80}{200} = 0.4, because there are 80k blue pixels and 200k pixels altogether.

    You then have to find \log_2(P_{\text{blue}}). But \log_2x = \frac{\log x}{\log2}. Therefore -P_{\text{blue}}\log_2(P_{\text{blue}}) = -0.4\frac{\log0.4}{\log2} \approx -0.4\frac{-0.3979}{0.3010} = 0.5288 (to 4 decimal places).

    Having found -P_{\text{blue}}\log_2(P_{\text{blue}}), you then have to do the same for each of the other seven colours, and add the results to get the entropy.
    Thanks very much for your reply.

    I've got the hang of calculating the probabilities but it is when log comes into it I get confused, basically because I don't even know what log is

    Would you be able to simplify the middle of your reply into what buttons to press on the calculator. Once I can see how one is done (blue), then I'd have no problems with the rest.

    Thanks again.
    Follow Math Help Forum on Facebook and Google+

  4. #4
    MHF Contributor
    Opalg's Avatar
    Joined
    Aug 2007
    From
    Leeds, UK
    Posts
    4,041
    Thanks
    7
    Quote Originally Posted by Dougster View Post
    Would you be able to simplify the middle of your reply into what buttons to press on the calculator.
    I guess the answer may depend on what sort of calculator you have. I use an ancient Casio. If I want to find \frac{\log0.4}{\log2} then I press these buttons:

     \boxed{\mathstrut 0.4}\quad \boxed{\mathstrut \log}\quad \boxed{\mathstrut \div}\quad \boxed{\mathstrut 2}\quad \boxed{\mathstrut \log}\quad \boxed{\mathstrut =}.

    In that sequence, "log" means log to the base 10. You can replace both "log"s by "ln" (natural log, or log to base e), and the result will be the same.

    Caution. Your results may vary. Different calculators use different sequences of operations.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Formula For Calculating Ratings of Products?
    Posted in the Pre-Calculus Forum
    Replies: 1
    Last Post: November 18th 2011, 05:46 AM
  2. Calculating the expected value for my formula (LaPlace)
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: May 15th 2011, 01:16 PM
  3. Formula for calculating the distance....
    Posted in the Geometry Forum
    Replies: 5
    Last Post: September 18th 2007, 07:38 PM
  4. Formula for calculating recurring payments?
    Posted in the Business Math Forum
    Replies: 2
    Last Post: August 3rd 2007, 10:40 AM
  5. Query about Information Entropy formula
    Posted in the Advanced Statistics Forum
    Replies: 4
    Last Post: May 29th 2007, 04:55 AM

Search Tags


/mathhelpforum @mathhelpforum