Thread: Question on Bias and Estimators

Hi all, this is my problem, hopefully the image is clear


To be honest, I'm stuck at finding the bias. I understand I need to work out the expectation (though that seems difficult, so help would be welcome), but what does it mean by subtracting sigma, when sigma is unknown?

Thanks

Hey Mick.

Since you have been given a distribution for S^2 in terms of chi-square, you want to find the expectation of S by itself.

Recall that the expectation of a function of a random variable E[g(X)] for a continuous distribution is Integral (-infinity,infinity) g(x)f(x)dx. Now X = S^2. Let g(X) = SQRT(X) and now given the expectation theorem, you can find out what E[SQRT(S^2)] = E[S] is.

Also be aware that n and sigma^2 are constants and not random variables so E[S^2] = sigma^2/(n-1)*E[chi-square_n_distribution].

Also remember that the expectation of a known distribution always returns some number whether its an actual number or just an expression in terms of non-random constants.

Thanks for the reply. So I'm at the integral of g(x)f(x), you've defined g(X)=SQRT(X) which I understand. But where do you take f(x) from? I guess either the normal distribution function or the inside of the square root of S?

f(x) will be the chi-square distribution with (n-1) degrees of freedom and E[SQRT(S^2)] = sigma^2/(n-1) * E[g(X)] where X ~ chi-square(n-1).

Thanks, I'm making progress I think, albeit very slow!

I've worked through the integration trying to best follow the one on this page, about half way down: Chi-square distribution

But there's some parts of it I don't understand. So I'm going to post a picture of my workings, and if you could tell me what's badly wrong, or just point me how to continue it, I'd be really grateful. It looks like it can be integrated by parts, but I'm really bad at integration, and I've never been good with gamma distribution either. Thanks

Once you get to this step, use the fact that the integral of the PDF over the whole domain is equal to 1. Now you haven't got a 1/(Gamma(k)*theta^k) in there so you need to balance the equation by factoring in this term, but that's all there is to do it to evaluate the integral.

More specifically if r = 1/(Gamma(k)*theta^k) and I is the integral term (without the constant c) then r*I = 1 if theta = 2 and k = n/2 so this means I = 1/r and then you can go from there.

Also recall that you are trying to calculate E[S^2] where E[S^2] = [sigma^2/(n-1)]*E[SQRT(X)] <- chi-square so don't forget to factor in the sigma^2/(n-1) in the end (and the goal is to show that E[S^2] = sigma^2 for unbiased-ness or something else for biased-ness).

I'm sorry, I'm just confused. Can you tell me what I should be aiming for, what this integral should equal because I think once I've got this the rest of the question should be straightforward. Thanks

You want to show that E[S^2] = sigma^2 for S^2 to be an unbiased estimator of sigma^2 and we know that E[S^2] = sigma^2/(n-1) * E[SQRT(X)] where X ~ Chi-Square with n-1 degrees of freedom.

Now in your working out you get E[SQRT(X)] down to the form of a gamma distribution PDF multiplied by some constant, but since you have the PDF in the integral, you can balance it by calculating for the appropriate constant c*Integral = 1 since integrating a PDF over the whole domain will always give 1. So if you balance this you will get Integral = 1/c for some constant (Hint: look at the definition of the Gamma distribution to find out what the c is in your case).

Then once you get the final result for E[SQRT(X)] above, substitute in and see if you get E[S^2] = sigma^2 or something else and make a conclusion based on that.