# Thread: Distribution of a MLE -

1. ## Distribution of a MLE -

Anyone have a pointer to some literature (online) of this. Having a bully of a time trying to come to some understanding here of what is going on. In particular were looking at likelihood functions and (according to instructor), to determine the pdf of an MLE, we set the derivative of the likelihood function equal to 0 and solve for the MLE - a result that makes no sense to me.

Just curious if anyone has an example of the derivation of a distribution for some MLE, as the book we are using is pretty sparse in this regards and has no examples of such an exercise.

2. Originally Posted by ANDS!
Anyone have a pointer to some literature (online) of this. Having a bully of a time trying to come to some understanding here of what is going on. In particular were looking at likelihood functions and (according to instructor), to determine the pdf of an MLE, we set the derivative of the likelihood function equal to 0 and solve for the MLE - a result that makes no sense to me.

Just curious if anyone has an example of the derivation of a distribution for some MLE, as the book we are using is pretty sparse in this regards and has no examples of such an exercise.
With a bit of hand waving over prior distributions (proper or improper) you may regard the likelihood function as the Bayesian posterior distribution of the parameters in the model you are doing the estimation for, which to some minds is more significant than the maximum of this distribution.

You seem to be confusing the posterior distribution (likelihood function) of the model parameters with the means of finding the maximum. To find the maximum you differentiate with respect to the parameters set the derivative to zero and solve the resulting equations for the parameters. This is a standard calculus procedure for finding the extrema of a function. (For technical reasons we often maximise the log-likelihood but that gives the same result as maximising the likelihood for these problems)

CB

3. I'll give some distributional results on the MLE. As Captain Black stated, what you are actually doing when you set the derivative of the log likelihood to 0 is deriving the MLE, not the pdf of the MLE.

Under suitable regularity conditions, MLEs are asymptotically normal with mean equal to their associated parameter and variance-covariance matrix equal to the inverse fisher information matrix. They are also efficient in the sense that the asymptotic variance attains the Cramer-Rao lower bound (again, under suitable regularity conditions). I think Casella and Berger handwave their way to this result in chapter 10 of their introductory statistical theory text, so you can look there for more details. This is an asymptotic result; I don't believe there are any particularly useful exact results that hold with reasonable generality.

4. The most likely value of a parameter, given some vector of observations, isn't what I was looking for. Literally (without attempting a Bayesian approach) I was looking for something on - in a frequentist framework I suppose - determining the distribution of an MLE; it seems for the particularl problem I was looking at, this was possible because of the nature of the parameter of interest, but would not necessarily always be so.