# joint entropy

• May 3rd 2012, 03:25 PM
alexandrabel90
joint entropy
given that x={0,1}, y={0,1}, and that p(0,0)=p(1,1)=a/2 and p(0,1)=p(1,0)=(1-a)/2,why is it that the joint entropy can never be zero?
• May 3rd 2012, 04:17 PM
jens
Re: joint entropy
The joint entropy is:

$-\sum_i p_i \log(p_i) = -2 \frac{a}{2} \log\left(\frac{a}{2}\right) - 2 \frac{1-a}{2} \log\left(\frac{1-a}{2}\right) = 2 H\left(\frac{a}{2}\right)$

where $H(x) = -x\log(x)-(1-x)\log(1-x)$ is the so-called binary entropy function, which goes to zero only for $x=0$ or $x=1$. So your entropy expression can only go to zero for $a=0$ or $a=2$.