I'm having a tough time with this problem:
I managed to do the proof (part a) no problem. However, I don't know where to start when it comes to part b.
If somebody could help me get started that'd be great.
All you need to do is check if P(k+1) is bigger or less than P(k).
This is determined by the multiplier:
$\displaystyle \frac{p}{1-p}\frac{n-k}{k+1}$
and you need to check when
$\displaystyle \frac{p}{1-p}\frac{n-k}{k+1} < 1$
because that's when the P() terms start to decrease.
I still don't see how to do it. Lets say I want to determine if P(k=0) is less than or equal to P(k=1). This would allow me to show that the function starts by increasing. However, if you plug the values in, you get $\displaystyle \frac{p^2(n^2-n)}{2(1-p)^2}$ and $\displaystyle \frac{pn}{1-p}$. I don't see how to determine which one is greater without knowing the values.
That being sad, I also don't know how to determine when $\displaystyle \frac{p}{1-p}\frac{n-k}{k+1} < 1$ for the same reason. It's too abstract, I don't see how you can determine it without knowing some of the values.
It's algebra:
$\displaystyle \frac{p}{1-p}\frac{n-k}{k+1} < 1$
multiply out the denominators:
$\displaystyle p(n-k) < (1+k)(1-p)$
rearrange:
$\displaystyle pn-pk < 1+k -p-pk$
$\displaystyle pn+p-1 < k $
$\displaystyle k > pn+p-1 $
So the multiplier is less than 1 when k > pn+p-1. That means the largest P(k) happened before this...