Results 1 to 2 of 2

Math Help - joint entropy

  1. #1
    Super Member
    Joined
    Aug 2009
    Posts
    639

    joint entropy

    given that x={0,1}, y={0,1}, and that p(0,0)=p(1,1)=a/2 and p(0,1)=p(1,0)=(1-a)/2,why is it that the joint entropy can never be zero?
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Junior Member
    Joined
    Oct 2011
    Posts
    35
    Thanks
    3

    Re: joint entropy

    The joint entropy is:

    -\sum_i p_i \log(p_i) = -2 \frac{a}{2} \log\left(\frac{a}{2}\right) - 2 \frac{1-a}{2} \log\left(\frac{1-a}{2}\right) = 2 H\left(\frac{a}{2}\right)

    where H(x) = -x\log(x)-(1-x)\log(1-x) is the so-called binary entropy function, which goes to zero only for x=0 or x=1. So your entropy expression can only go to zero for a=0 or a=2.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. entropy
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: May 3rd 2012, 01:19 PM
  2. Replies: 0
    Last Post: August 3rd 2010, 07:02 PM
  3. Replies: 5
    Last Post: December 5th 2009, 11:30 PM
  4. Entropy
    Posted in the Discrete Math Forum
    Replies: 2
    Last Post: October 6th 2008, 10:48 AM
  5. Entropy
    Posted in the Advanced Applied Math Forum
    Replies: 4
    Last Post: December 27th 2007, 04:58 AM

Search Tags


/mathhelpforum @mathhelpforum