# Thread: density for order stat, sufficient stat, MLE question

1. ## density for order stat, sufficient stat, MLE question

Pls take a look, this is an exam practice question but no answer is provided.

Question.

Let $\displaystyle Y_1, ... Y_n$ be a random sample from a distribution with density function

$\displaystyle f_Y(y;\theta)=3\theta^{-3}y^2, 0\leq{y}\leq{\theta}$
$\displaystyle f_Y(y;\theta)=0$ otherwise

where $\displaystyle \theta>0$ is an unknown parameter.

(a) Let $\displaystyle X=max_{1\leq{i}\leq{n}}Y_i$. Find the density function of X.

(b) find a simple sufficient statistic for $\displaystyle \theta$.

(c) find the maximum likelihood estimator $\displaystyle \hat{\theta}$ for $\displaystyle \theta$.

(d) compute the mean square error of the maximum likelihood estimator $\displaystyle \hat{\theta}$.

(a) I take it X is the maximum Y in the sample ie $\displaystyle Y_{(n)}$ so I find the density of it

$\displaystyle P(Y_{(n)}\leq{y})=P(Y_1\leq{y},... Y_n\leq{y})=[P(Y\leq{y})}^n=[F_Y(y)]^n=(3\theta^{-3}\frac{y^3}{3})^n=\theta^{-3n}y^{3n}$

$\displaystyle f_X(y)=f_{Y_{(n)}}(y)=\frac{d}{dy}F_{Y_{(n)}}(y)=n f_Y(y)[F_y(y)]^n=n*3\theta^{-3}y^2*\theta^{-3n}y^{3n}=3n\theta^{-3-3n}y^{2+3n}$

(b) to find sufficient statistics for $\displaystyle \theta$, I will use Factorisation theorem, ie factorising the joint density function of the sample

$\displaystyle f_Y(y;\theta)=(3\theta^{-3})^n\prod_{i=1}^nY_i^2I_{Y_{(1)}\geq{0}}(y)I_{Y_{ (n)}\leq\theta}(y)=(3\theta^{-3})^nI_{Y_{(n)}\leq\theta}(y)*\prod_{i=1}^nY_i^2I_ {Y_{(1)}\geq{0}}(y)$

the second component depends on the sample only; the first component depends on $\displaystyle \theta$ and on the sample through $\displaystyle Y_{(n)}(y)$ and can be a sufficient statistic for $\displaystyle \theta$.

?? does this makes sense ??

(c) here is where I am stuck and cannot move on to (d)

To find MLE for $\displaystyle \theta$, I will find maximum of log-likelihood of the density of Y

$\displaystyle L_Y(y;\theta)=(3\theta^{-3})^n\prod_{i=1}^nY_i^2$ subject to a constraint $\displaystyle 0\leq{y}\leq{\theta}$

$\displaystyle l_Y(y;\theta)=ln(L_Y)=ln[(3\theta^{-3})^n]+ln[\prod_{i=1}^nY_i^2]=n\ln(3\theta^{-3})+\Sigma_{i=1}^n\ln{Y_i^2}$

$\displaystyle \frac{\partial}{\partial\theta}(l_Y)=n\frac{1}{3\t heta^{-3}}(-9\theta^{-4})=-\frac{3n}{\theta^{-3}}\theta^{-4}=-\frac{3n}{\theta}$

this can be equal to zero only if n=0 and this is not what I am looking for here...

Can you help me to spot where I've gone wrong?

thanks!

2. Remember one thing: whenever the range of the variable in the pdf depends upon the unknown parameter, MLE can NOT be obtained by just differentiation.
Here, the range of $\displaystyle y$ depends on $\displaystyle \theta$ (since $\displaystyle y\leq\theta$)

MLE is such a value of the unknown parameter which maximizes the pdf. What do you think, for what value of $\displaystyle \theta$ the pdf will be maximum?

3. Originally Posted by Sambit
Remember one thing: whenever the range of the variable in the pdf depends upon the unknown parameter, MLE can NOT be obtained by just differentiation.
Here, the range of $\displaystyle y$ depends on $\displaystyle \theta$ (since $\displaystyle y\leq\theta$)

MLE is such a value of the unknown parameter which maximizes the pdf. What do you think, for what value of $\displaystyle \theta$ the pdf will be maximum?
Oh, good point )))

So, my likelihood function

$\displaystyle L_Y(\theta;y)=(3\theta^{-3})^n\prod_{i=1}^nY_i^2I_{Y_{(1)}\geq{0}}I_{Y_{(n) }\leq{\theta}}$

will reach maximum value (given fixed Y and $\displaystyle \theta>0$) if $\displaystyle \theta=Y_{(n)}$.

4. Further hint: the pdf attains maximum when $\displaystyle \theta$ is minimum. Do you follow that?

5. Yeah....you got it right.....

6. Originally Posted by Sambit
Further hint: the pdf attains maximum when $\displaystyle \theta$ is minimum. Do you follow that?
Yes I saw that (negative power of theta) but its lower bound is zero...

I must admit I can only do straightforward MLEs (and only on a clear day) so I am going blind here ))). I've posted a suggestion above. Appreciate your comment! This could be a good learning question for me.

7. Originally Posted by Sambit
Yeah....you got it right.....
thanks!! will proceed to (d) then.

8. (d) find MSE for $\displaystyle \hat{\theta}=Y_{(n)}=X$. Now I can use the density for X that I worked out in part (a) - truly if there is a rifle on the stage, there must be a moment in the performance when it will shoot ))))

$\displaystyle MSE_{\theta}(X)=E[(X-\theta)^2]=[Bias_{\theta}(X)]^2+Var_{\theta}(X)$

So I am planning to proceed as follows:

$\displaystyle Bias(X)=Bias (Y_{(n)})=\int_0^{\theta}(y-\theta)3n\theta^{-3-3n}y^{2+3n}dy$

$\displaystyle Var(X)=E(X^2)-E(X)^2$

$\displaystyle E(X)=E(Y_{(n)})=\int_0^{\theta}y3n\theta^{-3-3n}y^{2+3n}dy$

$\displaystyle E(X^2)=E(Y_{(n)}^2)=\int_0^{\theta}y^23n\theta^{-3-3n}y^{2+3n}dy$

Can you tell me if I am on the right path??...

9. To simplify the calculus, you can let $\displaystyle W_n=Y_{(n)}/\theta$

Then $\displaystyle F_W(w)=F_{Y{(n)}}(\theta w)=w^{3n}$ on 0<w<1.

This makes all these calculations easier.
This is a Beta random variable by the way
with $\displaystyle \alpha=3n$ and $\displaystyle \beta=1$
so you need not do any computation at all.

10. Originally Posted by matheagle
This is a Beta random variable by the way
with $\displaystyle \alpha=3n$ and $\displaystyle \beta=1$
so you need not do any computation at all.
ah! another challenge ))) let me try it, the things you've taught me so far proved to be valuable indeed

11. So, if $\displaystyle W=\frac{Y_{(n)}}{\theta}, F_W(w)=F(\theta{w})=w^{3n}, f_W(w)=3nw^{3n-1}, 0\leq{w}\leq{1}$ and $\displaystyle W\sim{Beta}(3n,1)$

Then we use Beta distribution to find mean and variance of W:

Mean=$\displaystyle E(W)=\frac{\alpha}{\alpha+\beta}=\frac{3n}{3n+1}$

$\displaystyle E(Y_{(n)})=E(\theta{W})=\theta{E}(W)=\frac{\theta{ 3n}}{3n+1}$

Variance=$\displaystyle Var(W)=\frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+ \beta+1)}=\frac{3n}{(3n+1)^2(3n+2)}$

$\displaystyle Var(Y_{(n)})=Var(\theta{W})=\theta^2Var(W)=\frac{\ theta^23n}{(3n+1)^2(3n+2)}$

(and I have checked the E and Var above by direct integration for w)

$\displaystyle Bias(Y_{(n)})=E(Y_{(n)}-\theta))=E(W\theta-\theta)=-E(\theta(1-W))=$

$\displaystyle =-\int_0^1\theta(1-w)w^{3n}dw=-\theta\int_0^1w^{3n}(1-w)dw=-\theta{B}(3n+1,2)=$

$\displaystyle =-\theta\frac{\Gamma(3n+1)\Gamma(2)}{\Gamma(3n+1+2)} =-\theta\frac{(3n+1-1)!(2-1)!}{(3n+1+2-1)!}=-\theta\frac{(3n)!}{(3n+2)!}=-\frac{\theta}{(3n+1)(3n+2)}$

$\displaystyle MSE_{\theta}(Y_{(n)})=\frac{\theta^2}{(3n+1)^2(3n+ 2)^2}+\frac{\theta^2}{(3n+1)^2(3n+2)}=\frac{\theta ^2(3n+3)}{(3n+1)^2(3n+2)}$

How's that?

12. Originally Posted by Volga
ah! another challenge ))) let me try it, the things you've taught me so far proved to be valuable indeed
if you keep saying that
someone may have a cow here

13. I only just glanced at your calculations and it looks like you're missing a square of the term in the denominator of the MSE.

14. Originally Posted by matheagle
I only just glanced at your calculations and it looks like you're missing a square of the term in the denominator of the MSE.
Thanks! you are right. But I cannot go back to that post and edit it, so this error (I think it is rather a typo) will be forever left on my record on this forum ))) whoever else reads this post, be aware...