Hello all! Having a problem with general MLE concepts. See if you can help me with this.

Let X1,...Xn be a random sample of size n where Xi~BIN(1,p). How do I find the Asyptotic distribution of the MLE of p?

Thanks for any help!

Printable View

- Mar 16th 2011, 08:46 PMcangerAsymptotic distributions of MLEs
Hello all! Having a problem with general MLE concepts. See if you can help me with this.

Let X1,...Xn be a random sample of size n where Xi~BIN(1,p). How do I find the Asyptotic distribution of the MLE of p?

Thanks for any help! - Mar 16th 2011, 10:43 PMmatheagle
the MLE is $\displaystyle \hat P$ which is an sample mean of Bernoulli's

So $\displaystyle {\hat P-P\over \sqrt{{pq\over n}}}\to N(0,1)$

and by Slutsky's Theorem

$\displaystyle {\hat P-P\over \sqrt{{\hat p\hat q\over n}}}\to N(0,1)$ as well. - Mar 17th 2011, 06:12 AMVolga
That looks like the formula $\displaystyle \sqrt{n}(\hat{\theta}-\theta)\sim{N}(0,I_Y^{-1}(\theta))$ adapted to Bin(1,p) for n observations, or, $\displaystyle \sqrt{I_Y(\theta)}(\hat{\theta}-\theta)\sim{N}(0,1)$ for large n

(with MLE=$\displaystyle \hat{p}$ and Fisher information $\displaystyle I_Y=\frac{1}{p(1-p)}$ for one observation, so $\displaystyle I_Y=\frac{n}{p(1-p)}$ for n observations)

So, does the former ($\displaystyle \sqrt{n}(\hat{\theta}-\theta)->N(0,I_Y^{-1}(\theta))$) comes from Slutsky theorem? I just never seen this name before, and I was also given the above assymptotic formula without explaining where it comes from. - Mar 17th 2011, 05:27 PMmatheagle
The Central Limit Theorem gives you

$\displaystyle {\hat P-P\over \sqrt{{pq\over n}}}\to N(0,1)$

replacing the p's and q's with their sample versions also converges in distribution to

a standard normal since

$\displaystyle \hat P\to P$ and $\displaystyle 1-\hat P\to 1-P$ by either law of large numbers

all you need is convergence in probability http://en.wikipedia.org/wiki/Slutsky's_theorem - Mar 17th 2011, 09:04 PMVolga
Ah! then, that assymptotic formula with Fisher information in it looks just like a version of the Central Limit Theorem. (edited: but of course it would, because it originates from CLT, I just didn't see the proof so I didn't see that it does. Sorry for off-top).